Map Any Site's Internal Link Graph.

A network created by a site's pages linking internally among themselves. ACME.BOT crawls a single URL, follows all internal links on the page, and returns a structured map for every page — including title, meta description, and outgoing links with anchor text — as well as auto-classifying them as a dead end, orphan, or hub.

ACME.BOT begins crawling the URL provided and extracts the page's meta description, title, and internal links. It then follows all internal links until it runs out or it hits max_pages.

Screaming Frog is a Java desktop app — require installation, limited to 500 URLs scanning for free. ACME.BOT runs in any browser — no signup needed — and gives structured JSON graph and a clickable visualization. It's better suited for quick structural scans than a full audit.

Domain of pages, each listing path/title/meta description. Also includes outgoing internal links with anchor text, and counts of pages crawled, discovered, and errored. Pages are classified as hubs, orphans, or dead-ends and also auto-grouped based on first path segment.

Hubs are pages with many internal links pointing to it — often a homepage, a blog index, pillar page, or page with relevant content to the topic. Orphans have zero internal links pointing to it — no other page that’s crawled has an internal link pointing to it. Alternatively, dead-ends have zero internal links pointing to other pages, which means they eliminate potential crawl paths to deeper content.

Internal linking is the structure of inter-page links within a site. Its importance comes from the way search engines use internal links to discover pages and distribute authority to them. For example, orphan pages don't typically rank, hubs transmit significant ranking power, and navigation depth determines the crawl priority of search engines.

Yes — you can see the full crawl JSON by looking at the results screen. You can also schedule crawls, diff runs, or export as CSV inside the ACME.BOT platform.

Each crawl walks up to a few hundred pages, controlled by the max_pages value. For larger sites, run multiple crawls from different starting URLs, or use the ACME.BOT platform.

Yes. ACME.BOT rate-limits requests by IP and caps each crawl by max_pages to prevent abuse. Submissions are anonymous — ACME.BOT provisions a separate identity per tool, no API key or account needed.

No, the crawler only follows internal links — those in the same domain as the starting URL. It ignores external links to focus the crawl on the structure of the site.

Yes. The crawler uses a standard user agent and follows the same access rules as a search engine. Pages disallowed by robots.txt won't be loaded, but they'll appear in the errors list.

Our crawler parses links from the raw HTML it fetches from a site's pages. However, since it's a server-side crawler, it will return fewer links from sites that navigate clients fully client-side — such as SPAs. To address this with SPAs, crawl its sitemap or pre-rendered build.

The platform conducts recurring crawls, flags broken links and redirect chains, diffs each run, and connects structural data to keyword plans and content. The free tool visualizes the graph, while the platform acts on it.