Crawlytics vs Ahrefs Bot Analytics: a Cloudflare-only beta logs AI-bot hits in the Ahrefs suite; Crawlytics tracks bots on any stack plus llms.txt from $29.99.
Quick answer
This is a narrow head-to-head on AI-bot log analytics — Crawlytics versus the Ahrefs Bot Analytics beta, not Ahrefs Brand Radar (that is prompt-sampling share-of-voice, a different category Crawlytics deliberately does not play in). Ahrefs Bot Analytics (as of June 2026) is a Cloudflare-only beta that shows which AI bots hit your site, viewable inside the Ahrefs suite (base plan ~$129+/mo). Crawlytics tracks the same AI-bot traffic from your own access logs on any stack, sorts bots into three intent tiers, returns valuable/parasite/watch ROI verdicts, and then goes well past tracking: it generates and serves llms.txt, serves clean HTML to AI fetchers, and runs a WebMCP commerce layer — none of which the Ahrefs suite ships. Pick Ahrefs Bot Analytics if you already pay for Ahrefs and run Cloudflare and just want a quick free bot view in one login. Pick Crawlytics if you want bot tracking on any host plus the tools to actually fix AI-readiness, standalone at $29.99/mo with no suite required.
First, which Ahrefs product? This trips people up, so I'll be blunt. Ahrefs ships two distinct AI features and they are not the same thing. Brand Radar is a prompt-sampling share-of-voice tool — it estimates how often AI models mention your brand. Bot Analytics is a separate beta that tracks AI-bot crawl hits on your own site. Crawlytics does not do share-of-voice, so comparing it with Brand Radar would be a category mismatch. If the share-of-voice question is what brought you here, read do you need Ahrefs or Semrush for AI visibility instead — that page handles Brand Radar and the SoV debate directly.
This post is the fair fight: Crawlytics versus Ahrefs Bot Analytics, head to head, on the one job they both do — telling you which AI bots are hitting your site. I'll cover what each does, where they genuinely overlap, where Ahrefs wins, where Crawlytics wins, and the real cost of each. Quick disclosure: I work on Crawlytics, so weigh the wins accordingly. I've kept the Ahrefs facts to what the company itself publishes (ahrefs.com), and hedged anything beta-dependent with "as of June 2026."
Ahrefs Bot Analytics is a beta feature, launched inside the Ahrefs suite, that surfaces AI-bot crawl activity against your verified site. As of June 2026, the picture looks like this:
That is a genuinely convenient combination for an existing Ahrefs-plus-Cloudflare shop. Worth naming the boundary clearly, though: Ahrefs has been publicly skeptical of llms.txt on its own blog, and the suite does not generate one, does not serve clean HTML to bots, and has no agent-commerce layer. Bot Analytics is a viewing feature, not a fix-it feature.
Crawlytics is a standalone AI-agent-readiness platform, and AI-bot tracking is the front door, not the whole house. The bot tracking itself reads from your own access logs, so it is not tied to any one CDN:
llms.txt and llms-full.txt from your sitemap and serves them at the stable URLs bots look for. Because it both writes the file and reads the logs, it can show your llms.txt Coverage Gap — pages you declared that no bot ever fetched.The throughline: Ahrefs Bot Analytics tells you the bots showed up. Crawlytics tells you that too, then tiers them by intent, tells you which ones pay off, and hands you the publishing tools — log-grounded LLM tracking on one side, llms.txt and clean-HTML serving on the other — to do something about it.
The overlap is exactly one thing: counting AI-bot hits on your site.
Both tools answer "is GPTBot crawling me, is ClaudeBot, is PerplexityBot, and roughly how much." If that single question is the whole of what you need, either tool gets you an answer. Ahrefs reads it from Cloudflare; Crawlytics reads it from your logs. Same headline number, two different pipes.
Everything else diverges. Ahrefs stops at the count. Crawlytics treats the count as the input to intent tiering, ROI scoring, and the publishing layer. So the overlap is real but thin — it is the foundation feature, not the building.
Three honest wins, all of them about already being an Ahrefs customer:
If both of those are already true, the Bot Analytics beta costs you nothing extra. You are paying the ~$129+/mo Ahrefs bill anyway for backlinks and rank tracking, Cloudflare is already in front of your site, and the AI-bot view rides along for free. That is a legitimately good deal for a quick look, and I would not pretend otherwise.
No new tool to evaluate, no new vendor to onboard, no second dashboard to remember. For a team that already runs its whole SEO practice inside Ahrefs, having AI-bot hits show up in the same place as everything else has real workflow value. Fewer tabs is a feature.
If your only question right now is "have the AI crawlers found us at all," the beta answers it without you building anything. For a site early in its AI-readiness journey that happens to be on Cloudflare and already in Ahrefs, that is a perfectly reasonable place to start before you decide whether the deeper analysis is worth a dedicated tool.
The wins here are the gap between "a bot view bolted onto an SEO suite" and "a tool built for this one job":
This is the structural one. Ahrefs Bot Analytics is Cloudflare-only as of June 2026. If you are on Vercel, Netlify, Fastly, a generic VPS, or any non-Cloudflare CDN, it has nothing to show you. Crawlytics reads your access logs directly, so the stack you happen to run is irrelevant. For the large slice of the web not behind Cloudflare, that is the difference between data and a blank screen.
A raw count treats a training-data harvester and a live ChatGPT user-fetch as the same event. They are not. Crawlytics sorts bots into three intent tiers and surfaces ROI verdicts (valuable / parasite / watch), so you can see which crawlers actually correlate with downstream value and which are just burning your server. A hit count cannot tell you that.
Tracking is the start. Crawlytics also generates and serves llms.txt, serves clean AI-optimized HTML to fetchers, and (on Commerce) runs the WebMCP layer for agent transactions. The Ahrefs suite does none of these — and has publicly dismissed llms.txt rather than build it. If your goal is to improve how AI consumes your site rather than just watch it happen, this is the whole ballgame.
Crawlytics is $29.99/mo standalone for the Visibility tier. You do not need a $129+/mo SEO suite wrapped around it, and you do not need Cloudflare. If AI-bot tracking and AI-readiness are the jobs you are buying for, you pay only for those jobs.
This is where "free beta" needs an asterisk. Ahrefs Bot Analytics is free during beta, but you reach it from inside the Ahrefs suite, and Ahrefs plans start around $129/mo (as of June 2026). So the true cost to access Bot Analytics is the price of an Ahrefs subscription — unless you are already paying it for other reasons. Plus the Cloudflare requirement, which is a cost in flexibility even when Cloudflare's own plan is free.
Crawlytics prices the other way around — pay for the AI job, nothing else:
The honest framing: if you are already paying the Ahrefs bill and running Cloudflare, marginal cost of the bot view is zero and convenience is high. If you are not already an Ahrefs-plus-Cloudflare shop, standing up the whole suite just to see AI-bot hits is a poor trade against a $29.99/mo standalone tool that works anywhere and does more.
Two clean cases:
Get Ahrefs Bot Analytics if you already pay for Ahrefs, your site already runs on Cloudflare, and you want a quick AI-bot view in a login you already use. As a free first signal under those conditions, it is genuinely fine — start there and see if you need more.
Get Crawlytics if any of those conditions is missing, or if you want more than a hit count. Not on Cloudflare? Crawlytics works anyway. Not an Ahrefs customer? You skip the $129+/mo suite. Want intent tiering, ROI verdicts, llms.txt, clean-HTML serving, or WebMCP commerce? Only Crawlytics ships those. If you are deciding between dedicated AI-bot tooling and a suite side-feature in general, the broader best AI-bot tracking tools roundup lays out the field, and Crawlytics vs Google Analytics covers why GA itself goes blind on this traffic.
These are not really competitors fighting over the same buyer. One is a free convenience inside a suite you may already own; the other is a purpose-built, any-stack platform that tracks bots and fixes readiness. Pick by where you already live and how far past "are the bots here" you need to go.
Written by Crawlytics Team. Crawlytics tracks AI bots, generates llms.txt, and powers WebMCP commerce, all from one snippet on any stack. See how it works →
Ahrefs Bot Analytics is a Cloudflare-only beta that shows AI-bot hits inside the Ahrefs suite; Crawlytics is a standalone AI-readiness platform that tracks bots on any stack and also generates llms.txt, serves clean HTML to bots, and runs WebMCP commerce. As of June 2026.
The Bot Analytics feature itself is in free beta as of June 2026, but you reach it inside the Ahrefs suite, which starts around $129/mo, and it only reports data if your site runs on Cloudflare. So it is free-with-an-asterisk, not free standalone.
No. You can track AI bots from your own server logs without any Ahrefs subscription. Crawlytics does exactly this on any hosting stack, starting at $29.99/mo, and it adds intent tiering, ROI verdicts, and llms.txt that a log grep alone will not give you.
Not as of June 2026 — the beta sources its AI-bot data from Cloudflare, so if your site is on Vercel, Netlify, Fastly, a generic VPS, or another CDN, it has nothing to report. Crawlytics reads from your own access logs and works regardless of your CDN.
Bot Analytics tracks AI-bot crawl hits on your site (a log-analytics feature). Brand Radar is a separate share-of-voice product that samples AI-model prompts to estimate how often your brand is mentioned. They answer different questions; this post is about Bot Analytics, not Brand Radar.
This page is part of Crawlytics.app. View all pages: llms.txt · llms-full.txt