25+ AI crawlers are reading the web — can they read you?
Make your site AI-agent ready.
AI bots crawl your site every day. You can't see them.
They try to read your pages. Most can't.
Agents arrive ready to buy. They can't.
Crawlytics is one tag that fixes all three.
no charge until day 8 · cancel anytime · live in ~5 min
Watch the 60-second explainer// why it matters
An AI agent is a customer in a hurry.
When ChatGPT or Google's agent lands on your site, it arrives with a job: book the appointment, buy the product, request the quote. It either finishes the task or leaves for a competitor whose site let it. No browsing, no second visit.
Crawlytics shows which agents already visit, fixes whatever stops them finishing, and hands them direct actions — so the fastest path to a sale is yours. What is the agentic web? →
// from snippet to AI-ready
Three commands. Under five minutes.
No code changes to your site, no DNS, no reverse proxy.
Point to your site
Provide your URL and sitemap. Crawlytics crawls your pages and converts them to clean markdown — llms.txt, llms-full.txt, and per-page .md.
Drop in one tag
Paste the snippet, middleware, or edge function. Works on Cloudflare, Vercel, Netlify, WordPress, Express, nginx — no DNS, no reverse proxy.
Watch who is reading
Every LLM request is logged with bot, company, timestamp, and page. Full analytics dashboard in real time.
// one tag · one URL · three readers
Same URL. Three readers. one snippet routes it.
routes by User-Agent ↓
no reverse proxy · no DNS · one <script> does the routing
// three layers, one tag
Detect. Serve. Sell.
Track every AI bot that visits. Serve them a version of your site they can actually read. Hand agents ready-made actions — search, book, buy — that you control.
See every AI that touches your site
25+ bot signatures across 19 providers, classified the moment they hit.
Serve bots the format they prefer
Auto-generated llms.txt + per-page markdown from your sitemap. Bots get AI-Optimized HTML; humans still get your HTML.
Let agents transact with your site
One tag exposes agent-callable tools. Every payment attributed to the agent.
▲ 44% · 5 providers · Stripe, Paddle…
// everything else in the one tag
AI attribution
Catch the ChatGPT & Perplexity clicks Google calls “direct.”
Analytics dashboard
Real-time trends, projections, date-range comparisons.
Multi-site
Every property you own in one workspace.
Security
Per-tool gates, signed webhooks, no PII stored.
Agent-Ready Grader
Scan your site’s AI-readiness in seconds — free.
// new · the action layer
Let agents actually do things on your site.
One <script> tag and the AI agents in Gemini, Claude, ChatGPT, and Comet can search your catalog, start checkout, book appointments, and submit quotes — every conversion attributed to the agent that drove it. No CMS plugin.
<script src="https://crawlytics.app/snippet/loader.js" data-site="<YOUR_SITE_ID>" async></script>
Agents take action
An in-page confirmation modal gates every write tool. User approves → your backend executes.
Conversion attribution
5 payment providers (Stripe, Paddle, Lemon Squeezy, PayPal, Square) + a lead path. Every sale tied to the agent that drove it.
Works on Cloudflare · Vercel · Netlify · WordPress · Express · Next.js · nginx · Apache — plus log import for any HTML site.
// two tiers. pick. ship.
Simple pricing.
Monthly or annual (20% off · save up to $120/yr). 1 site included on Solo, bundle up to 25.
Visibility
detect · serveSee and feed the agents.
or $287.90/yr — save $72
- ✓Real-time AI bot tracking (25+ signatures)
- ✓AI referral attribution (ChatGPT, Claude, Perplexity)
- ✓llms.txt, per-page markdown, readiness audit
Commerce
RecommendedVisibility + WebMCP — let agents transact.
or $479.90/yr — save $120 · no charge until day 8
- ✓Everything in Visibility
- ✓WebMCP snippet — agent-callable tools
- ✓Conversion attribution (Stripe, Paddle, PayPal…)
Cancel anytime · No surprise charges · Credit card required on trial (we never auto-bill before day 8)
Multi-site? Growth (5 sites) $99.98/mo · Scale (25 sites) $199.96/mo
20+
LLM bots tracked
3
Tracking backends
<1ms
Tracking overhead
Be the site that AI agents pick.
One tag, three layers — track every AI bot, generate llms.txt automatically, and let agents transact via WebMCP. Live in 90 seconds on any stack.
7 days free · No charge until day 8 · Cancel anytime
// faq
Frequently asked.
llms.txt is an open standard (llmstxt.org) for telling AI systems what a website contains — a plain-text file at /llms.txt with a curated index of your pages and one-line descriptions. AI bots fetch it instead of trying to parse your JavaScript-heavy HTML. Crawlytics generates it (plus llms-full.txt and per-page markdown) from your sitemap automatically.
Probably. GPTBot is OpenAI's crawler for ChatGPT — it visits public websites a few times per week looking for content to train and answer with. Most sites don't know it's happening because GPTBot doesn't show up in Google Analytics. Crawlytics tracks every GPTBot hit (plus ClaudeBot, PerplexityBot, and 22 others) and shows you which pages they're reading.
Most analytics tools log ChatGPT clicks as "direct traffic" because ChatGPT's mobile and in-app browsers strip the Referer header. Crawlytics solves this by injecting per-LLM UTM tags (utm_source=chatgpt, utm_medium=ai_referral) into the AI-Optimized HTML bots fetch — so when ChatGPT cites your page and a user taps the link, attribution survives even with no Referer.
Three usual reasons: (1) ChatGPT can't read your site because it's JavaScript-heavy and GPTBot doesn't execute JS — fix with llms.txt + per-page markdown; (2) GPTBot is being blocked by your CDN bot rules or robots.txt — check both; (3) Your content isn't cited because it lacks the topical depth or structured signals AI assistants prefer. Crawlytics handles all three layers.
WebMCP (Web Model Context Protocol) is the draft web spec that lets web pages expose tools to in-browser AI agents via navigator.modelContext. Currently supported in Chrome 146+ Canary, with agents in Gemini, Claude, ChatGPT, and Comet able to invoke registered tools. Crawlytics ships a one-tag loader so you don't have to write browser-API code yourself.
No reverse proxy, no DNS changes. Add your site to the dashboard, get a unique tracking ID, then paste the snippet for your stack. Events stream in real time. Works on Cloudflare, Vercel, Netlify, WordPress, Express, Next.js, nginx, Apache — plus a log-import path for any HTML site. Live in under 5 minutes.