Which Data Sources Should You Care About For AI Search?

0
1
Which Data Sources Should You Care About For AI Search?


Tier Typical use Source Evidence status What the evidence says Reference 1 Web & search discovery Google Search Confirmed + current Grounding with Google Search connects Gemini to real-time web content, returning inline citations to source URLs. Google — Gemini API docs, Grounding with Google Search 1 Bing Search Confirmed + current Microsoft documents Bing results being used to enhance Copilot responses. Not re-verified in this pass. Microsoft Bing 3 Common Crawl Confirmed historical GPT-3 used filtered Common Crawl as roughly 60% of its sampling mixture; LLaMA 1 reported 67%. Common Crawl 3 Historical web corpora (C4 etc.) Confirmed historical C4 is a cleaned derivative of Common Crawl; LLaMA reported C4 at 15% of its pretraining mixture. TensorFlow Datasets — C4 4 Web grounding services Strong evidence / likely Category inference covering third-party grounding/retrieval intermediaries. No single canonical source. — 1 Products & shopping Google Merchant Center Confirmed + current Merchant feed data underpins Google’s shopping surfaces. Retained on user instruction; Google Shopping removed as it is a surface, not a source. Google Merchant Center Help 2 Merchant / retail feeds (OpenAI) Confirmed + current Merchants share a secure, regularly refreshed CSV/JSON feed of identifiers, descriptions, pricing, inventory, media and fulfilment so ChatGPT can surface products accurately. Refreshes accepted as often as every 15 minutes. OpenAI Developers — Agentic Commerce, product feeds 4 Microsoft Merchant Center Strong evidence / likely Equivalent commercial feed infrastructure; inferred parallel to Google Merchant Center rather than separately evidenced. — 4 Marketplace feeds Strong evidence / likely Category inference. Shopify catalog data is already integrated into ChatGPT, which supports the pattern. OpenAI Help — Shopping with ChatGPT Search 1 Local & places Google Maps Confirmed + current Grounding with Google Maps is a documented tool alongside Search grounding, giving models geospatial context. Google Cloud — Grounding API 1 Google Business Profile Confirmed + current Business profile data feeds Google’s local surfaces. Carried from the source table; not separately re-verified. — 1 Yelp Confirmed + current Yelp licenses reviews, photos and business information to OpenAI for real-time local recommendations. Beyond grounding it also drives actions: ChatGPT users can book a table or join a waitlist, and Request a Quote lets users contact providers in-chat. Yelp’s 10-Q confirms it is live. Axios; Yelp blog; Yelp 10-Q FY2026 4 OpenStreetMap Strong evidence / likely Widely used open geospatial corpus; inferred rather than confirmed for any named model. — 4 Foursquare Strong evidence / likely Left in Tier 4 deliberately: the OpenAI deal is Yelp’s, and no equivalent evidence exists for Foursquare. — 4 Tripadvisor Strong evidence / likely Category inference for review/travel data. No confirmed deal identified in this pass. — 1 Knowledge & reference Wikipedia Confirmed + current Explicitly present in GPT-3’s disclosed mixture and LLaMA (June-Aug 2022 dumps, 20 languages); also widely used as a live reference/RAG corpus. Wikimedia dumps 1 Wikimedia Confirmed + current Same corpus family as Wikipedia. Licensing is unusually clear: principally CC BY-SA with attribution/share-alike obligations. Wikimedia dumps 4 Wikidata Strong evidence / likely Structured entity layer; strongly implied by knowledge-graph use but not separately confirmed. — 1 Community / Q&A / social Reddit Confirmed + current The Google deal gave access to the Reddit Data API for ‘real-time, structured, unique content’, and allows Reddit content to be displayed across Google products — i.e. live grounding, not only training. Tom’s Guide (Google/Reddit deal) 2 Reddit Confirmed + current Same deal, training side: Google may use Reddit posts to train its AI models and improve services such as Search; reported at roughly $60m/yr. NOTE: Reddit is reportedly weighing whether to renew — treat as unstable. Fortune; Neowin/WSJ on renewal doubt 4 Social platforms Strong evidence / likely Category inference covering platform-wide social corpora. — 4 Forums / communities Strong evidence / likely Category inference. Overlaps Reddit but generalised to non-Reddit forums. — 1 News & publisher content Live publisher pages Confirmed + current Reached at inference time via search grounding rather than pretraining; retrieval selection and crawlability govern inclusion. Google — Grounding with Google Search 2 Licensed publisher content Confirmed + current OpenAI has multiple explicit licensing partnerships (FT, Axel Springer, AP, News Corp). Terms differ per partner on training vs grounding vs attribution. OpenAI — FT content partnership 2 Publisher partnerships Confirmed + current Axel Springer’s deal includes otherwise paywalled material in answers; AP licensed part of its text archive. OpenAI — Axel Springer partnership 3 Historical news corpora Confirmed historical Archive material absorbed in pretraining; distinct from live licensed access. — 2 Developer / technical GitHub Confirmed + current LLaMA used GitHub’s public BigQuery dataset, restricted to Apache/BSD/MIT projects; The Pile separately includes GitHub. Public visibility is not an open licence. GitHub 2 Stack Overflow Confirmed + current Named in licensing-deal mapping alongside Reddit and Shutterstock as a data platform powering multiple buyers. LLM Pulse — AI content licensing deals mapped 2 Technical docs Confirmed + current Vendor documentation corpora; widely used but not tied to a single disclosed agreement. — 4 npm / PyPI registries Strong evidence / likely WEAKEST ENTRY IN THE TABLE. Relabelled from ‘package registries’ to name examples. No disclosed agreement or documented retrieval use found — consider cutting. — 1 Travel & commerce actions Google Hotel Center feeds Confirmed + current When Gemini or AI Mode show hotel options with real-time prices, that data comes from the Google Hotels feed. In Aug 2026 Google added hotel booking inside AI Mode completed with Google Pay, so this is grounding plus actions. TechCrunch — AI Mode travel update 4 Booking / partner feeds Strong evidence / likely Booking Holdings and IHG are reported as participants in Google’s agentic booking pilot, which supports the direction but stops short of a documented feed spec. InfosTourisme (IHG/Booking pilot) 4 OTA / commerce sources Strong evidence / likely Category inference. OTAs run their own rate feeds into these surfaces. — 4 Reservation / inventory APIs Strong evidence / likely Category inference covering booking/inventory endpoints exposed to agents. —