Is my website in ChatGPT's training data?

While ChatGPT hasn't published its manifest, its training corpora draws from CommonCrawl (and other publicly-available datasets). A check of whether your site is being pulled into CommonCrawl and FineWeb is the best public sign that it may be included.

What's the difference between CommonCrawl and FineWeb?

CommonCrawl is a raw monthly web crawl with roughly 2 billion URLs. FineWeb is HuggingFace's quality-filtered subset of CommonCrawl (~200M URLs) using the same filtering heuristics modern LLMs use — presence in FineWeb is a stronger signal than CommonCrawl alone.

How often are these datasets updated?

CommonCrawl's crawl data is released monthly followed by FineWeb's updates. Meanwhile, LLMs are trained on data from a fixed point in time — this report shows the datasets behind today's major models.

Can I remove my content from existing training datasets?

No — you can't change data in datasets that already exist. If you'd like to block user agents such as CCBot, GPTBot, PerplexityBot, and others in robots.txt from crawling you in the future, or you can reach out to dataset curators and ask them to remove your data.

How do I get my site included in future crawls?

CommonCrawl focuses on pages listed on sitemaps and those with inbound link references. To improve crawl prioritization, allow CCBot in your robots.txt, submit an up-to-date sitemap, regularly publish new content, and acquire links from reputable domains.

Why are some of my pages present but not others?

As crawling bots botter around the web, they see pages linked to, and crawl those first. Robots also skip pages that are orphaned, behind a login screen, or those that are blocked via a robots.txt file. Crawling bots also will crawl pages at the top of your site more easily than pages buried deep within them.

Does training-data presence guarantee AI citations?

No. While the model's presence on your page indicates it may have been trained on its content, citations are based more on freshness, topical authority, query match, and content structure.

How is this different from Profound or Indexly?

Profound and Indexly monitor what AI assistants say about your brand. This report tells you the upstream cause: whether your content was in the training data. Both matter — this is the diagnostic that comes first.

Free onboarding call with one of our founders. Join our discord.

Is your website in ChatGPT's training data?

Enter a URL. ACME.BOT scans CommonCrawl and FineWeb — the public datasets behind today's largest LLMs — and returns a per-URL report.

Scans CommonCrawl (2B+ URLs) and FineWeb (200M+ URLs) · Free preview, no signup required.