How many websites pass the Xrobots Noindex check?

Based on 9 audited sites · Updated 2026-05-08

100%of audited sites pass this check

About This Check

Checks whether the X-Robots-Tag response header contains noindex, which prevents indexing at the HTTP header level.

Why This Matters

LLM Discoverability

Header-level noindex directives apply to all content types (HTML, PDF, images) and are respected by AI crawlers at the same level as meta tags, making them the preferred mechanism for server-rendered or dynamically generated pages.

SEO Impact

X-Robots-Tag is the only way to suppress indexing of non-HTML resources such as PDFs; for HTML pages it has identical effect to the meta robots noindex tag.

Status Breakdown

100%pass(9)

0%warn(0)

0%fail(0)

Compliance Trend

Daily pass rate across benchmark snapshots (up to 90 days).

Industry Breakdown

How different industries perform on this check.

Technology100% passavg score 74.6n=3

Media & Entertainment100% passavg score 73.8n=5