How many websites pass the Ai Not Blocked check?

Based on 499 audited sites · Updated 2026-05-08

78.8%of audited sites pass this check

95% confidence interval: 75%–82.1%

About This Check

Checks whether robots.txt blocks common AI/LLM crawlers. Blocking AI agents can reduce your visibility in LLM-based discovery systems.

Why This Matters

LLM Discoverability

Blocking known AI user-agents (GPTBot, ClaudeBot, etc.) directly prevents those models from citing your content, summarising your pages, or recommending your site in responses.

SEO Impact

Blanket AI agent blocks can extend to Google's AI-driven tools such as Google-Extended; misconfigurations can inadvertently impair standard indexing as well.

Status Breakdown

78.8%pass(393)

4.6%warn(23)

16.6%fail(83)

Compliance Trend

Daily pass rate across benchmark snapshots (up to 90 days).

Industry Breakdown

How different industries perform on this check.

Other42.9% passavg score 67.4n=7

Energy100% passavg score 73.7n=3

Gaming33.3% passavg score 79.3n=3

Retail100% passavg score 77.2n=5

Travel100% passavg score 72n=6

Finance84.6% passavg score 75.7n=26