llms.txt explained: what it is and how to make one

llms.txt is a Markdown text file you place at the root of your website (/llms.txt) so that language models and AI agents get a concise, single-location overview of what the site is about and where its important content lives. The format was proposed by Jeremy Howard, co-founder of Answer.AI, in September 2024 on llmstxt.org. It's not an official web standard but an open proposal: Google Search doesn't use it, yet it has become common practice for developer documentation.

What is llms.txt?

Websites are built for people. On a typical HTML page the actual information is wrapped in navigation, cookie banners, scripts and decoration. A language model has to turn all that back into clean text, which is imprecise and wastes context window. The idea behind llms.txt is simple: give the machine a short, edited entry point that tells it what's worth reading.

The proposal got its second version in August 2026. According to the v2 changelog, thousands of sites publish an llms.txt file, documentation platforms generate one automatically, and coding agents use them reliably. It's live on the developer docs of Anthropic, OpenAI, Cloudflare, Vercel and Stripe, among others, and Google itself publishes one for the Gemini API docs. Mintlify, GitBook and Wix can generate one, and on WordPress so can Yoast SEO and AIOSEO.

Do you need an llms.txt for your website?

Honest answer: it depends on what you expect from it.

• If you have an API, a SaaS product or developer docs: yes, it's worth it. This is where it's used most: coding agents use it to find your reference pages and examples.
• If you're a service business, a webshop or an institution: it's not required, but it's cheap and harmless. An AI agent that's already on your site finds your pricing, shipping terms or contact details faster. And if someone pastes the file into a chat, the model gets the essentials in your own words.
• If you expect Google traffic or AI Overview citations from it: don't. More on that below.

We run one on our own site because it's a few hours of work, it breaks nothing, and AI agents browsing websites is no longer a prediction but everyday reality. But it doesn't replace solid technical foundations, which we covered in our post on technical SEO and AI discoverability.

What does a good llms.txt look like?

According to the specification, the file is Markdown, but it follows a precise order so it can also be processed with classic tools (parsers, regular expressions):

1. An H1 with the name of the project or site. This is the only required element.
2. A blockquote (a line starting with >) with a short summary containing the key information needed to understand the rest of the file.
3. Any amount of free text (paragraphs, lists), without headings: more detail, and how to interpret the linked files.
4. H2 sections with link lists. Each list item is a required [name](url) link, optionally followed by a colon and a short note.

By convention, an H2 section named “Optional” marks secondary links that an agent can skip when it needs a shorter context. Since v2 it no longer carries mechanical meaning, but it's still a useful signal.

The other half of the proposal is that important pages should offer a clean Markdown version at the same URL with a .md suffix (page.html.md or page.md; for URLs without a file name, index.html.md or index.md). v2 recommends standard link relations for this: rel="alternate" type="text/markdown" points to the Markdown version, and rel="describedby" points to the llms.txt that covers the page. An llms.txt can also live in a subpath (e.g. /docs/llms.txt), in which case it covers the pages beneath it.

You'll often see a file called llms-full.txt too: it concatenates the entire documentation into a single Markdown file. It was developed jointly by Mintlify and Anthropic, but the current text on llmstxt.org doesn't define it. It's useful for documentation and pointless for a twenty-page company website.

A complete example file

A template for a webshop (with made-up details, as a starting point):

# Example Webshop

> Online store for handmade kitchenware. We ship across the EU from our
> Budapest warehouse, with customer support in Hungarian and English.

- Prices include VAT, in HUF; domestic shipping is free over 20,000 HUF.
- In-stock items are dispatched within 1–2 business days.
- Markdown versions of product pages are available by appending .md to the URL.

## Shopping

- [Shipping](https://example.com/shipping.md): rates, delivery times, pickup points
- [Returns](https://example.com/returns.md): how the 14-day withdrawal works
- [Payment methods](https://example.com/payment.md): card, bank transfer, cash on delivery

## Products

- [Categories](https://example.com/categories.md): the full range by category
- [Materials guide](https://example.com/materials.md): which material is best for what

## Company

- [About us](https://example.com/about.md): company details, team, contact
- [Terms](https://example.com/terms.md): general terms and conditions

## Optional

- [Blog](https://example.com/blog.md): recipes and usage tips
- [Press kit](https://example.com/press.md)

Our live file

On flowproject.hu, our llms.txt is live. A shortened, real excerpt:

# FlowProject Kft.

> Independent digital studio in Budapest. Two disciplines: software
> development, and marketing — a marketing agency with strong developer
> and AI expertise. [...]

## About

- **Founded:** 2019
- **Location:** Budapest, Hungary
- **Languages served:** Hungarian (primary), English

## Optional

- [Sitemap](https://flowproject.hu/sitemap.xml)
- [Blog RSS](https://flowproject.hu/blog/rss.xml)
- [Robots policy](https://flowproject.hu/robots.txt)

Ours isn't textbook either: the services section uses prose and H3 subheadings instead of link lists, and the blog links point to HTML pages rather than .md versions. That's a common compromise on business sites: a language model reads it fine, but a strict parser only recognizes [name](url) lines as links.

llms.txt vs. robots.txt vs. sitemap.xml

All three live at the root, but they do very different jobs:

• robots.txt: access control. It tells bots what they may and may not fetch. Well-behaved crawlers read it first. It's the only one of the three that allows or blocks anything.
• sitemap.xml: a complete inventory for search engines. Every indexable URL, in XML, with modification dates. It's neither curated nor a summary, and on a larger site it won't come close to fitting in a model's context.
• llms.txt: a curated guide for language models. Not every page, just the important ones, with human-readable descriptions and external links too. Per the spec, it's mainly used when an agent is helping someone in the moment, not for model training.

The key difference: llms.txt neither grants nor removes access. If your robots.txt blocks AI crawlers, llms.txt won't change that. In our robots.txt we explicitly allow GPTBot, ClaudeBot and PerplexityBot, as well as the Google-Extended token, which controls how Google's AI models may use your content.

Does Google take it into account?

No, and Google has said so in writing. Search Central's AI optimization guide says you don't need to create new machine-readable files, AI text files or Markdown to appear in Google Search (including its generative AI features), because Google Search doesn't use them. The same page also says you're free to create an llms.txt for other services: it will neither harm nor help you in Google, because Google Search ignores it.

John Mueller of Google's Search Relations team has been skeptical more than once. In April 2025 he compared it to the keywords meta tag: it only shows what a site owner claims about their site. In June 2025 he wrote on Bluesky: “no AI system currently uses llms.txt”. In June 2026, on the Search Off the Record podcast, he argued that a self-reported file can't be used to rank sites against each other, but that such a guide may help someone who is already on the site.

Meanwhile, another Google product sends a different message: Chrome Lighthouse now has an llms.txt audit in its “agentic browsing” category. It only flags the file if fetching it returns a server error. If there's no file (404), the audit is marked “not applicable” and it doesn't affect the score.

The other big providers publish llms.txt files for their own docs, but to our knowledge, as of October 2026, none of the major LLM providers has officially announced that its crawler or chatbot uses other sites' llms.txt files. So the sober summary is: forget it as a ranking factor, but it makes sense as something you serve to agents and developer tools.

How to create one, step by step

1. List what an agent needs to know about you. The five to ten most common questions: what you do, for whom, at what price, where, how to reach you, and on what terms.
2. H1 and summary. The company or product name, followed by a one-to-three sentence blockquote that stands on its own.
3. Key facts, briefly. A few factual lists or paragraphs, no headings, no marketing copy. Numbers, lead times, terms.
4. H2 sections with link lists. Group your important pages and give every link a short, informative description.
5. An Optional section at the end for secondary links: blog, press, archive.
6. Offer Markdown versions if you can. If your pages have .md versions, link to those rather than heavy, JavaScript-laden HTML.
7. Put it at the root as /llms.txt, as plain text, UTF-8 encoded, returning a 200 status. On WordPress, Yoast SEO or AIOSEO can generate it; on a custom site, make it part of the build so it doesn't go stale.
8. Test it. As the spec suggests, give an AI assistant nothing but your llms.txt and ask it questions about your site. Wherever it's wrong or unsure, the file is missing something.
9. Maintain it. Price changes, new services, retired pages: the file has to keep up. An outdated llms.txt is worse than none.

Common mistakes

• Treating it as a ranking trick. Google has stated that it ignores it. If someone promises you rankings from it, ask what they're basing that on.
• Confusing it with robots.txt. llms.txt blocks nothing and allows nothing. Access is still controlled by robots.txt.
• Saying something different from your site. This is exactly Mueller's objection: diverging content is effectively cloaking aimed at models. llms.txt should summarize your site, not present an alternative reality.
• Stuffing it with marketing copy. “Market-leading innovative solutions” carries zero information for a model. It needs facts.
• Dumping the whole site into it. The goal is a guide that fits in context. The details belong behind the links.
• Bare URLs without descriptions. The [name](url): note format helps an agent decide which link is worth opening.
• Forgetting to update it. Last year's prices, discontinued services, links that return 404.
• A server that errors. If /llms.txt returns a 500, Lighthouse flags it. Better to have no file than one that throws errors.

Frequently asked questions

Does llms.txt hurt or help Google rankings?

Neither. According to Google's official guidance, Google Search ignores llms.txt, so it neither helps nor harms your visibility.

Where do I upload llms.txt?

To the root of your domain, so it's available at https://yourdomain.com/llms.txt. The v2 specification also allows subpaths (e.g. /docs/llms.txt), which then cover the pages beneath them.

Which language should I write it in?

The language your audience searches in. Language models handle both well. For our bilingual site we wrote it in English and listed the URLs for both languages.

Do I also need an llms-full.txt?

For developer documentation it's useful, because the whole corpus can be pasted into an AI tool as one file. For a smaller company site, a well-described llms.txt is enough.

Do ChatGPT or Claude use llms.txt?

None of the major LLM providers has officially announced that its crawler uses other sites' llms.txt files. Agents and developer tools can open it when pointed to it, though, and anyone can paste it in as context.

How long does it take to create?

For a typical company website, a few hours if you know what matters. The real work is keeping it in sync as the site changes.

llms.txt is a small piece of a bigger picture: AI discoverability is driven by fast, clean HTML, structured data and citable content. If you're curious where your own site stands, ask us for a free SEO and AI visibility audit: we'll show you what works, what's missing and where to start.