Public Site Recon

Find PDFs, forgotten docs, sitemap-only URLs, and historical leftovers.

Enter a domain or URL. This version crawls live internal pages, parses robots.txt and sitemaps, and can enrich results with the Wayback Machine and Common Crawl. It is built for public discovery, not brute-force probing.

Run a scan

Domain or URL

Scope

Max HTML pages

Max depth

Default file filter

Include Wayback history Good for older public files and URLs that are no longer linked.

Include Common Crawl Stronger historical coverage, but slower and more variable than Wayback.

Include subdomains Useful for docs, blog, help, legacy, or file-serving subdomains.

PDFs are already included. The better version of this tool is about provenance: where a URL was found, whether it is linked live, whether it only exists in history, and how risky/sloppy it looks.

Ready. Run a scan to build a report.