Map Any Site's Internal Link Graph.

A network created by a site's pages linking internally among themselves. ACME.BOT crawls a single URL, follows all internal links on the page, and returns a structured map for every page (title, meta description, and outgoing links with anchor text), classifying each one as a dead end, orphan, or hub.

ACME.BOT begins crawling the URL provided and extracts the page's meta description, title, and internal links. It then follows all internal links until it runs out or it hits max_pages.

Screaming Frog is a Java desktop app: you install it, and the free tier stops at 500 URLs. ACME.BOT runs in any browser with no signup and returns a structured JSON graph plus a clickable visualization. It suits quick structural scans rather than a full audit.

Domain of pages, each listing path/title/meta description. Also includes outgoing internal links with anchor text, and counts of pages crawled, discovered, and errored. Pages are classified as hubs, orphans, or dead-ends and also auto-grouped based on first path segment.

Hubs are pages with many internal links pointing at them, usually a homepage, a blog index, or a pillar page. Orphans have none: no other crawled page links to them. Dead-ends are the opposite, pages that link out to nothing, so a crawler reaching them has no path deeper into the site.

Internal linking is the structure of inter-page links within a site. Its importance comes from the way search engines use internal links to discover pages and distribute authority to them. For example, orphan pages don't typically rank, hubs transmit significant ranking power, and navigation depth determines the crawl priority of search engines.

Yes. The results screen shows the full crawl JSON. Scheduled crawls, run diffs, and CSV export live in the ACME.BOT platform.

Each crawl walks up to a few hundred pages, controlled by the max_pages value. For larger sites, run multiple crawls from different starting URLs, or use the ACME.BOT platform.

Yes. There's a daily cap on scans, and each crawl is capped by max_pages to prevent abuse. Submissions are anonymous. ACME.BOT provisions a separate identity per tool, so no API key or account is needed.

No. The crawler only follows internal links, meaning links on the same domain as the starting URL. External links are ignored so the crawl stays focused on site structure.

Yes. The ACME.BOT crawler uses a standard user agent and follows the same access rules as a search engine. Pages disallowed by robots.txt won't be loaded, but they'll appear in the errors list.

The crawler parses links from the raw HTML it fetches. Because it runs server-side, it finds fewer links on sites that build their navigation entirely in the browser, such as single-page apps. For those, crawl the sitemap or a pre-rendered build instead.

The platform conducts recurring crawls, flags broken links and redirect chains, diffs each run, and connects structural data to keyword plans and content. The free tool visualizes the graph, while the platform acts on it.