What is llms.txt — and does your site actually need one?

llms.txt is a proposal (from Jeremy Howard, of fast.ai) for a plain-markdown file at yoursite.com/llms.txt that tells AI systems what your site is and where the important pages are — a sitemap written for language models instead of crawlers. Curated links with one-line descriptions, not XML.

What it's for

LLMs work with limited context. When an AI system lands on your site — a live assistant fetch answering a user's question, or an agent researching vendors — it can't read 400 pages. An llms.txt hands it the short version: here's what this product does, here's pricing, here's the docs, here's the API. You choose what gets read first, instead of leaving it to luck and internal linking.

The honest part

Adoption is real but young. Thousands of sites publish one (we do: visitortype.ai/llms.txt), and several AI dev tools consume the format — but no major AI search engine has committed to fetching it, and you shouldn't expect a traffic jump the week you add one. The case for doing it anyway:

1. It costs ten minutes. A markdown file. No build step, no risk.
2. The downside is zero — unlike robots.txt changes, it can't accidentally block anything.
3. Assistant fetches are growing. Live, per-user page fetches (ChatGPT-User, Claude-User, Perplexity-User) are the fastest-growing slice of AI traffic, and those are exactly the visits a curated map helps.

How to write one that helps

• Start with one H1 (your name) and a one-paragraph summary — the elevator pitch an AI should repeat about you.
• Group links under H2s (## Docs, ## Pricing, ## Guides), one line of context per link.
• Link pages that answer questions (pricing, comparisons, how-tos) — not your homepage carousel.
• Keep it current. A stale map is worse than none; an AI confidently citing your 2024 pricing is a support ticket.

Where VisitorType fits

Publishing the file is step one; knowing whether AI systems come at all is step two. VisitorType classifies every visit — crawler, AI search bot, assistant fetch, agentic browser — so you can see whether the audience llms.txt is written for is actually showing up, and which pages they read instead of the ones you'd choose.

See who's reading you before optimizing for them: create a free account — the free tier is enough to find out.