Google calls llms.txt 'completely fine' — but that's not the same as Google using it. Here's what the guidance actually says and what it means for your site.
Google's position on llms.txt is the shortest useful thing Google has said about AI content formats in years: it's "completely fine." That came via a spokesperson response covered by Search Engine Journal, and it closes the question that was quietly stalling a lot of site owners: can I ship this without getting penalized?
The answer is yes. Full stop. But "completely fine" hides a subtlety worth spelling out, because the reason most people pause before shipping isn't actually about Google.
Google has cleared the file from a policy standpoint. It does not violate webmaster guidelines, it does not manipulate search signals, and Googlebot will not treat its presence as a spam indicator. That's the "completely fine" part.
What Google hasn't said — and hasn't done — is announce that it reads llms.txt as part of its ranking pipeline. As of mid-2026, Google Search still indexes pages through standard web crawling. AI Overviews pull from that web index. The llms.txt format was not designed for Google Search, and Google has not claimed otherwise.
So the permission is real. The SEO boost is not. Those are two separate things, and conflating them is the most common mistake in how this guidance gets reported.
The statement was framed around whether llms.txt creates risk for site owners. Google's answer: no risk. The file doesn't appear on any penalties list, doesn't confuse Googlebot, and doesn't violate any structured-data or content guidelines.
What Google did not say:
llms.txt and uses its contents for rankingAll of those would be overclaims from the available guidance. "Completely fine" means "we won't penalize you," not "we'll reward you."
Here's the exact framing that matters. Google permits the file. Google does not use it (at least not as a confirmed ranking or indexing input).
A useful parallel: Google also permits you to publish a JSON file called /humans.txt crediting your dev team. That's "completely fine" too. Doesn't mean Google reads it. The permission is about harm avoidance, not about feature adoption.
llms.txt sits in that category for Google Search specifically. The file was invented for a different set of readers — AI assistants, coding agents, and LLM-powered tools — and that's who's actually fetching it. Google clarified that shipping one won't hurt you in Search. It didn't say it would help.
This matters because some coverage of the Google statement implied an endorsement. It isn't one. It's a clearance.
The file was proposed by Jeremy Howard (Answer.AI, fast.ai) in September 2024 with a specific problem in mind: AI assistants spend expensive context tokens parsing noisy HTML when a site owner could just hand them a clean markdown index. The spec is at llmstxt.org and it's deliberately minimal.
The clients that actually fetch the file today:
llms.txt when you paste a URL or ask it what a site coversllms-full.txt when available, uses llms.txt for navigationllms-full.txt to pre-warm context when you point them at a library or APINone of those are Google. All of them are growing. For many developer-focused or content-heavy sites, the Cursor + Windsurf fetch is already the most valuable use case — a coding agent loading your entire API reference in one request instead of scraping 40 individual HTML pages.
For a deeper look at what each of these AI surfaces does with your content, the llms.txt guide walks through the full picture, including how to structure the file for maximum agent readability.
A few things this guidance does not support, even though they circulate:
It won't boost your Google rankings. There's no confirmed signal path from llms.txt to PageRank, Core Web Vitals, or any other ranking factor. Shipping one does not accelerate indexing, improve crawl budget, or change your position for any keyword.
It's not robots.txt. Robots.txt is a crawl-control file that every web crawler respects as a protocol standard. llms.txt is a voluntary convention. AI clients that support it will read it; those that don't will ignore it. There's no enforcement mechanism.
"Completely fine" is not an endorsement. Google didn't say "ship this and benefit from it in Search." The statement was about risk, not reward.
Having one doesn't guarantee AI citations. The file helps AI clients navigate your site more efficiently, but citation decisions are editorial — they depend on your content quality, how well-structured your pages are, and whether the AI's training data or retrieval pipeline points to you. That's a separate problem from file format, and one that requires content work, not just a text file.
The Google clearance resolves the only real reason to hesitate. The question that remains is whether the file is working for you in the channels that actually read it — and that's a log problem, not a format problem.
Most sites that ship llms.txt have no idea whether any AI bot fetches it. They set it up and assume something is happening. The honest picture is more nuanced: a significant share of llms.txt files get zero AI bot requests, which doesn't mean the file was a mistake — it means you need to measure.
What instrumentation gives you:
llms-full.txt (a strong signal of developer adoption)Without that data, you're guessing. The file might be working great or sitting idle. The answer is in your server logs, and most analytics platforms don't surface it by default because they filter bot traffic out.
If you want to know whether your llms.txt is in the working minority, the Agent-Ready Grader gives you a quick read on file health, and Crawlytics' bot analytics show you per-bot fetch activity so you're measuring the actual audience, not assuming one.
Google said something useful and narrow: shipping llms.txt is safe. That clears the way for the sites that were sitting on the fence out of penalty anxiety. It doesn't create a new SEO lever, and reading it as an endorsement sets up false expectations.
The file's value is real — it's just not where most people look for it. The value is in AI assistants reading your content cleanly instead of scraping it messily, in coding agents loading your API reference efficiently, and in you having a measured baseline of which AI clients are actually touching which pages. None of that depends on Google Search. Google just confirmed it won't get in the way.
Ship it because the AI audience is real and growing. Instrument it because "I think bots are reading it" is not a strategy. The aeo-vs-seo-vs-geo distinction — which AI surfaces exist beyond Google and how each retrieves content — is worth reading if you're building an AI visibility plan from scratch: AEO vs SEO vs GEO covers the full map.
Written by Crawlytics Team. Crawlytics tracks AI bots, generates llms.txt, and powers WebMCP commerce, all from one snippet on any stack. See how it works →
This page is part of Crawlytics.app. View all pages: llms.txt · llms-full.txt