Agent-Readiness AuditAbout AgentReadyBot

AgentReadyBot is the crawler behind the Agent-Readiness Audit. It runs only when someone submits a site on the scan page, fetches a fixed set of well-known files from that site's origin, and produces a report on how well the site serves AI agents. It does not crawl, index, or store page content.

Identification

User-Agent
AgentReadyBot/0.1 (+https://agentready.sbkinship.com/bot)
Trigger
Human-initiated scans only; the same origin is scanned at most once per 10 minutes.
Per request
5s timeout, at most 3 redirects, first 512KB of each body.
Per scan
About 22 requests, all to the submitted origin. No JavaScript is executed.
Signature
Every request except the two probes below carries an RFC 9421 signature (Web Bot Auth) with Signature-Agent: "https://agentready.sbkinship.com". Verify it against the key directory at /.well-known/http-message-signatures-directory.

What it fetches

Only / and these well-known paths, plus a same-origin sitemap or OpenAPI document if the site's own robots.txt or API catalog declares one. Nothing else, and never anything a user typed.

AI crawler access probe

One check asks whether the site blocks AI crawlers. To find out, AgentReadyBot requests / two extra times with these User-Agent strings. Only the header is imitated: no signatures, IP ranges or other identity of those crawlers are forged, and the responses are used solely to report the status code.

Opting out

AgentReadyBot reads your robots.txt before anything else. Add this group and every future scan of your origin stops after that one request:

User-agent: AgentReadyBot
Disallow: /

Blocking is honoured immediately; there is no cache of permission.