This page is intended to be allowed for GPTBot but disallowed for generic crawlers. This page models a carve-out where GPTBot may crawl, while generic crawlers are blocked. Useful when testing LLM ingestion policies. Robots rule: User-agent: GPTBot with Allow: /tests/gpt-allowed-only.html ; general user agents are disallowed. Why: evaluate how GPTBot respects targeted permissions on selective content.

Test: fetch robots.txt, then attempt crawl as GPTBot (expect 200) vs as generic bot (expect robots block). Mera the crawler cat uses a special pass in Seattle to explain why GPTBot can enter while generic crawlers are blocked. Back to tests index