Is Your Site AI-Agent Ready? Free Scorers Are Only a Snapshot

Aerospace is an industry where a misplaced decimal can end a program. Its documentation is exhaustive, version-controlled, and reviewed by people paid to be precise. So it lands hard that when AeroScore, a scoring engine that grades documentation portals on AI-agent readiness, ran six major aerospace documentation portals through its checks, 0 of 6 passed.

If the most documentation-obsessed industry on earth can go 0 for 6, your site is worth a look. The good news: the criteria that AeroScore and similar tools test are concrete, cheap to check, and mostly free to fix. The catch, which I will get to, is that passing the test is not the same as being read.

What "AI-agent ready" actually means

Strip away the jargon and readiness comes down to a simple question: when an AI system arrives at your site, can it find, parse, and quote your content without tripping over your own setup? Three things decide the answer.

A discoverable llms.txt. This is a plain-text file at /llms.txt that lists your important pages in markdown, giving AI crawlers a predictable map of your site. If you have never set one up, our guide to llms.txt walks through it. Scorers check that the file exists, sits at the right URL, and is well-formed.

A robots.txt that does not block the agents you want. Plenty of sites quietly disallow GPTBot, ClaudeBot, or PerplexityBot in a line someone copied from a template two years ago. A readiness scorer flags those rules so a marketing team does not discover, months later, that it locked out the exact crawlers feeding AI search results.

Structured content instead of PDF blobs. This is where aerospace fails. Critical specs live inside PDFs, scanned manuals, and JavaScript-rendered viewers. An AI agent that fetches the page gets a link to a binary it will not open, or an empty shell that renders nothing without executing scripts. Content trapped in a PDF might as well not exist for a crawler that reads HTML and moves on. Semantic HTML with real headings, real text, and schema markup is what gets parsed and cited.

CiteReady, a free tool that checks whether AI search engines can cite your site, tests the same family of signals: robots.txt, schema, and llms.txt presence. Run it and an AeroScore-style scorer on any URL and you get a configuration grade in seconds. That is a genuinely useful first step, and I recommend doing it before you read another word.

The 5-minute self-audit any site can run

You do not need a tool to catch the biggest gaps. Walk this checklist on your own site right now:

1. Load yoursite.com/llms.txt in a browser. If it 404s, you have no AI-readable map. That is the single most common miss.
2. Open yoursite.com/robots.txt and search for GPTBot, ClaudeBot, Google-Extended, and PerplexityBot. Any Disallow: / under those agents means you are invisible to that system by choice.
3. Pick your most important page and view its source. If the substance is a linked PDF, or only appears after scripts run, an agent probably cannot read it.
4. Check for JSON-LD schema. Search the page source for application/ld+json. No structured data means AI systems have to guess what the page is about.
5. Confirm a meta description and clean heading structure. These are cheap signals that help both search and AI summarization.

Fix what this surfaces and you will clear most of what a free scorer grades. Which raises the obvious next question.

The wedge: configuration is not visibility

Here is the limitation nobody selling a free grader leads with. A configuration audit checks whether your site could be read by AI agents. It says nothing about whether they are.

Those are different questions with different answers. Your llms.txt can be present and perfectly formed while no crawler has fetched it in a month. Your robots.txt can welcome GPTBot while GPTBot ignores your site entirely, because nothing links to it and it never found you. You can pass every checklist item on the page and still be a ghost in the systems that matter. A green score measures your setup. It does not measure reality.

This is the gap between a checklist and a heartbeat monitor. One confirms the patient is dressed correctly for surgery. The other confirms the patient is alive.

Every item on the left is worth fixing. None of them proves an agent showed up.

What continuous bot-traffic monitoring adds

The other half of readiness is watching what actually hits your server. When you log and classify the AI crawlers reaching your site, three things become visible that no config audit can show.

Trend data. A one-time score is a photograph. Traffic over time is a film. You see whether ClaudeBot visits are climbing after you published a cornerstone guide, whether Perplexity discovered your pricing page, or whether a whole class of agent never arrives. Crawlytics charts this per bot, per page, and per day, with a 14-day projection so you read direction, not just yesterday.

Regression alerts. The most expensive readiness failures are the ones you cause by accident. A developer ships a new robots.txt, a CDN rule changes, a migration drops your llms.txt, and AI crawl volume quietly falls off a cliff. A config scan you ran last quarter will not catch it. Continuous monitoring flags the drop when it happens, so a deploy that locks out GPTBot becomes a same-day alert instead of a quarter of lost citations.

Per-page AI-crawler heatmaps. Aggregate numbers hide the story. What you want to know is which URLs the agents actually read and which they skip. A heatmap of AI-crawler hits per page can tell you your docs are getting fetched but your product pages are not, or that a section you thought mattered draws zero agent interest. That is the difference between guessing and knowing which content earns AI attention. It also shows whether your llms.txt is honored in practice: you can watch whether crawlers follow the map you published or wander off it.

If you want the deeper version of this, we wrote a full walkthrough on tracking AI bots crawling your site and on running a proper AI search visibility audit.

Use the free scorers first, then measure

None of this makes AeroScore or CiteReady a waste of time. The opposite is true. They are the right place to start, because you cannot debug visibility while your configuration is broken. If your llms.txt 404s and your robots.txt blocks ClaudeBot, no amount of traffic monitoring will help, because there is nothing to measure yet. Fix the config first. Free scorers make that fast.

The mistake is stopping there and assuming a passing grade means you are done. Configuration is the entry ticket. Whether agents actually walk in is a separate fact you can only learn by watching the door. Two steps, in order: a scorer certifies you can be crawled, then monitoring proves you are.

Crawlytics runs both ends. Its agent-readiness grader gives you the configuration score for free, in the same family as AeroScore and CiteReady, and its dashboard then shows the live bot traffic behind that score. For a broader look at the category, we compared the options in best agent-readiness tools and covered Cloudflare's readiness score specifically. Once the config is clean, the live demo shows what the traffic side looks like.

Frequently asked questions

What does it mean for a website to be AI-agent ready?

An AI-agent ready website can be found, parsed, and quoted by AI systems without tripping over its own setup. In practice that means a discoverable /llms.txt file, a robots.txt that does not block crawlers like GPTBot and ClaudeBot, and content in structured HTML rather than trapped in PDFs or script-rendered viewers. Readiness is about removing the technical barriers between your content and the agents that want to read it.

Is a free AI readiness score enough?

A free score is a strong first step but not the finish line. Tools like AeroScore and CiteReady check your configuration: whether your files exist, whether your robots rules are sane, whether you have schema. They cannot tell you whether AI crawlers actually visit, which pages they read, or whether your llms.txt is honored. Configuration is not the same as visibility. Fix the config with a free scorer, then use bot-traffic monitoring to confirm agents show up.

Why did 0 of 6 aerospace documentation portals pass AeroScore?

According to the AeroScore results shared on Hacker News, none of the six major aerospace portals tested met the readiness criteria. Aerospace documentation tends to live in PDFs, scanned manuals, and heavy JavaScript viewers, which AI crawlers struggle to read. Precise, thorough documentation is not the same as agent-readable documentation. A crawler that reads HTML sees a link to a binary and moves on, so meticulous content stays invisible.

What is the difference between AeroScore and CiteReady?

Both are free configuration scorers for AI readiness. AeroScore is a scoring engine that grades documentation portals on signals like llms.txt presence, robots.txt configuration, and PDF-heaviness, and its aerospace run found 0 of 6 portals passing. CiteReady checks whether AI search engines can cite your site by testing robots.txt, schema, and llms.txt presence. They overlap heavily and both are worth running. Neither measures live bot traffic.

How do I know if AI bots are actually crawling my site?

You have to look at your server traffic and identify the AI crawlers by user agent. GPTBot, ClaudeBot, PerplexityBot, and others announce themselves in request logs. A tool like Crawlytics classifies these visits automatically and shows which bots hit which pages, how often, and whether the trend is rising or falling. That live data is the only way to confirm the readiness work you did is producing actual agent visits.