Crawl bigger documentation sites, and see exactly what made it in
Web sources now read documentation sites that publish AI-friendly versions of their pages, cover up to 500 pages, list every page the crawler found with its result, and re-crawl one page without the whole site.