Where does AI get its information? And how to get cited

0
1
Where does AI get its information? And how to get cited


AI systems get information using multiple sources and methods, making each a pathway to higher AI visibility. Help AI learn about your brand through any of them — or better still, more than one of them — and you’re more likely to earn mentions and citations.

Let’s look at where AI systems like ChatGPT and Google’s AI Overviews get information from and how to give them more details about your brand.

The four ways AI gets its information

The four ways AI gets its information are training, retrieving live online information, leveraging licensing partnerships, and user-initiated actions to pull data:

1. Training

AI companies train their models on data to help those models learn to understand and generate content. This training data is the foundational knowledge layer that models draw on to answer prompts.

AI companies collect LLM training data from sources like:

  • Publicly available online content on your site and third-party ones like industry publications, blogs, code repositories, and Wikipedia
  • Content licensed from third parties like Reddit
  • Content provided by the AI company’s in-house staff, including trainers and researchers
  • AI chat histories of users who consent to the AI company using their conversations for training
  • Data generated by the AI company’s models for training purposes

Models have knowledge cutoff dates, after which they’re no longer trained on new data. To reduce the risk of outdated answers, AI systems often draw on additional sources of information that I’ll touch on later.

For example, when I asked ChatGPT how much rain Hawaii got from Hurricane Lala with web search turned off, ChatGPT replied that it was unaware of that hurricane. 

ChatGPT response saying no Hurricane Lala affected Hawaii and suggesting the user may mean Hurricane Lane in 2018

It was only when I enabled web search that ChatGPT acknowledged Hurricane Lala and the rainfall totals across different parts of the state.

ChatGPT response claiming Hurricane Lala hit Hawaii in 2026, with rainfall totals and cited sources

How to show up where AI models are trained

You can show up where AI models are trained by actively building a presence on publicly available online content and licensed third-party content. Here’s how:

  • Allow AI crawlers to access your site. Check that your site’s robots.txt file doesn’t contain disallow rules telling crawlers like OpenAI’s GPTBot not to collect AI training data from your site.
  • Present your brand and offerings clearly and consistently on your site. Your site’s content should reflect the brand narrative you want AI systems to know.
  • Build your brand’s presence on third-party sites. For example, create a Google Business Profile and online directory listings. And run digital public relations campaigns to secure coverage in industry publications.

These tactics won’t have an impact on existing models due to their cutoff dates. But your efforts can shape what newer models — which are trained on more recent data — know about your brand.

2. Retrieving live online information

AI systems retrieve live online information to supplement training data and improve their answers’ accuracy.

For example, ChatGPT retrieved live information from various sites to provide me with the latest U.S. stock market news:

ChatGPT stock market news response with cited sources including The Wall Street Journal, Reuters, and BBH

AI companies use proprietary crawlers to fetch live online data. They also get live web information through search engines, which crawl and index content independently.

This table lists some of AI systems’ known sources of live online data:

AI system

Known sources of live online data

ChatGPT

  • A proprietary search index built with OAI-SearchBot, OpenAI’s search crawler
  • Google’s search index, which is built with the Googlebot search crawler
  • Bing’s search index, which is built with the Bingbot search crawler

AI Overviews and Google AI Mode

Claude

  • A proprietary search index built with Claude-SearchBot, Anthropic’s search crawler

How to help AI systems retrieve your site content

Help AI systems retrieve your site content by allowing crawlers from AI companies and search engines to access your website by verifying that your robots.txt file doesn’t have disallow rules targeting these crawlers.

Next, use Google Search Console and Bing Webmaster Tools to check if Google and Bing have indexed your pages. If they haven’t, AI companies won’t be able to retrieve these pages’ content from the search engines’ indexes.

In Google Search Console, for example, click the drop-down menu under “Indexing” in the left sidebar and select “Pages” to view the number of pages Google has and hasn’t indexed.

You can also view specific URLs that are and aren’t indexed. To view your indexed pages, click “View data about indexed pages.”

Google Search Console Indexed Pages report listing example URLs and their last crawl dates

Or, use Semrush’s Site Audit tool to schedule regular scans of your website for issues affecting crawlability and other website performance factors. You can also get email notifications with the scan results.

Semrush Site Audit settings showing a weekly Monday crawl schedule and email notification option

View the technical issues that Site Audit detected on your site by navigating to the “Issues” tab. To narrow the report’s scope to crawlability issues, click “Crawlability.”

Semrush Site Audit Issues report filtered to Crawlability, showing errors and warnings

Click “How to fix” next to any issue to learn how to address it.

Semrush Site Audit issue details explaining an incorrect sitemap URL and how to fix it

After that, optimize your content to help AI systems match it to relevant prompts. Here are some tips for doing that:

  • Optimize for query fan-out: Identify prompts’ underlying intents and address them in your content
  • Write self-contained sections: Each section should provide a complete answer to its heading without needing context from other sections
  • Keep your content updated: AI systems prefer citing fresh content that provides the latest information on topics

Finally, work to get more brand mentions on third-party sites. Because AI systems ground their answers in live information not just from your site, but also from the wider internet.

Brand mentions on external sites should also corroborate your website’s information about your brand to help AI systems build a consistent picture about you across sources.

3. Leveraging licensing partnerships

Several AI companies have struck licensing partnerships allowing their AI systems to use partner organizations’ content for training and live retrieval.

For example, OpenAI and Google have partnered with Reddit and Stack Overflow. OpenAI also licenses content from organizations like Yelp and Time.

Content from partner organizations often appears prominently in AI systems’ answers. In this conversation, for instance, ChatGPT heavily cited Reddit when I asked for recommendations for “cheap eats in austin texas according to locals”:

ChatGPT response recommending cheap Austin restaurants using Reddit citations

How to improve your presence in partner companies’ content

Improve your presence in partner companies’ content by identifying your target AI system’s partner companies, then taking concrete steps to appear in their content.

Let’s say you want to grow your visibility on ChatGPT. Since OpenAI licenses content from Reddit, Yelp, and Time, you could improve your presence in these companies’ content by participating in Reddit discussions, creating a Yelp directory listing, and pitching journalists at Time for coverage.

As much as possible, ensure the content about your brand on third-party sites uses the language and framing you want AI systems to adopt when talking about you.

4. User-initiated actions to pull data

Users can prompt their AI systems to pull data from other tools and sources to enrich their answers.

For example, Claude lets me upload files I’ve exported from other tools, in formats like JPG, PDF, and CSV. I can then prompt Claude to analyze the files’ content and take actions with them:

Claude groups 100 wedding photography prompts into themes based on searcher intent

I can also connect online tools to Claude and prompt it to pull data from them directly.

Claude uses the Semrush integration to analyze ASICS organic keywords, traffic, and SEO strategy

Making a tool’s data accessible to AI systems is more about helping users process that data than it is about boosting the connected tool’s visibility in AI systems.

But this data accessibility may increase the tool’s value to users, giving them more reason to keep using it.

How to help AI systems get data from tools you build

If you build tools, work with developers to add features enabling AI systems to get data from them.

For example, allow users to export tool data in AI systems’ supported file formats.

You can also build a Model Context Protocol (MCP) server that connects your tool’s data via an application programming interface (API). This way, users can connect an AI system via the MCP to query your tool for data.

Track your AI visibility

Track your AI visibility with a tool like Semrush’s AI Visibility Toolkit, so you know whether your efforts to help AI get information on your brand have translated to more AI mentions and citations.

When you enter your domain into the AI Visibility Toolkit, the Visibility Overview report shows your number of mentions, citations, and cited pages over time.

Semrush AI Visibility Overview for stripe.com showing mentions, citations, cited pages, and six-month trends

The Visibility Overview report also breaks down your visibility by AI search platform. Use this data to assess the effectiveness of targeted measures, like the removal of disallow rules telling OpenAI’s bots to avoid crawling your site.

Semrush AI Visibility report showing mentions by ChatGPT, AI Overview, AI Mode, Gemini, and country

Next, go to the Perception report to see AI tools’ sentiment toward your brand and the narratives driving positive and negative sentiment.

Semrush brand perception report showing Stripe sentiment, brand strengths, and areas for improvement

If your organization is large and has especially complex needs, Semrush’s Enterprise AIO solution would be a better fit.