Robots.txt validator and URL tester.

Check your crawl rules, find errors, and see why a page is allowed or blocked.

Add your robots.txt

UTF-8 plain text · 500 KiB (512,000 bytes) max. This tool supports 20,000 lines and 8,192 bytes per line.

Test a page (optional)

Nothing is saved. Paste and uploads stay in your browser; URL loading contacts that website directly.

Your robots.txt check

    Tests supplied rules, not live crawlability or indexing. Page URLs and sitemap references are never fetched.

    Know which rule wins.

    This robots.txt validator helps website owners review crawl rules before publishing them. Paste a file, upload plain text or load a public robots.txt where browser access is permitted. Test a page address for a selected Google crawler and see the relevant group, matched rule and source line. Fix syntax errors before relying on the test.

    A specific crawler group takes precedence over *. Matching paths are case-sensitive; the longest rule wins, with Allow winning ties. Crawl blocking does not guarantee removal from search results.

    Google’s robots.txt guidance ↗ · Check your XML sitemap

    Make useful community answers discoverable.

    Jatra gives customer discussions and resources a home on your domain. Book a demo call to explore a community that adds lasting, searchable knowledge to your website.

    Explore Jatra’s SEO & AEO features ↗
    Book a demo call