What is llms.txt, what does it do, and do you need it?

A file called llms.txt in your site's root, listing your key pages in Markdown, was proposed as a way to tell AI tools where to look first. This guide covers what the file is, how it is written, what Google says about it, and why we keep one anyway.

What is llms.txt?

llms.txt is a proposal published by Jeremy Howard of Answer.AI on 3 September 2024; its page is now at version 2. The file is plain Markdown served at /llms.txt. The idea is that when an AI tool visits a site while answering a question, it finds a short table of contents. The author expects it to be used mostly when a user is asking for help (at inference time), not for training.

The difference from robots.txt matters: robots.txt is a rules file that says who may crawl what. llms.txt is a guide file; it blocks and permits nothing. They do not replace each other. We cover bot access in our robots.txt guide. A robots.txt line named Content-Signal also manages no access. It only states how content may be used: What is Content-Signal?.

How do you write llms.txt?

According to the proposal, the file has this order:

1. A first-level heading (#) naming the site or project. This is the only required part.
2. A short summary in a blockquote (>).
3. Optional: extra notes in any Markdown except headings.
4. Optional: second-level headings (##), each followed by a list of links. Each line is a [name](url) link, optionally followed by : and a short note.

The proposal also defines an "Optional" section: links there are secondary and can be skipped when a tool needs shorter context.

A small example:

# Example Shop

> Example Shop sells handmade ceramics online. Prices in TRY, shipping within Turkey.

## Products
- [Cup sets](https://example.com/cups): All sets with prices and sizes
- [Plates](https://example.com/plates): Diameter and material

## Help
- [Shipping and returns](https://example.com/returns): Times and terms

## Optional
- [About us](https://example.com/about)

The proposal also suggests serving a clean Markdown version of pages at the same address with .md added. That is not required.

What does Google say about llms.txt?

In its guide to optimizing for generative AI search, Google writes that Google Search does not use these files. The guide says you do not need new machine-readable files, AI text files or special markup to appear in Google Search, that it is fine to maintain such files for other services, and that doing so will "neither harm nor help" your visibility or rankings in Google Search. The guide also notes the terms AEO and GEO, and calls optimizing for generative AI search "still SEO".

So you do not need llms.txt to appear in Google. What you need is for the page to be indexed and eligible to be shown in Search with a snippet.

Do OpenAI, Anthropic and Perplexity use it?

Company documents: could not be verified. The OpenAI, Anthropic and Perplexity bot documents we read talk about robots.txt; none says llms.txt is read. So we make no claim like "you need it for ChatGPT" or "Claude reads it". If you see such a claim, ask for the source document.

Measurement: we know of one public server log. Evil Martians, a software agency, recorded agent requests to its own site for two months and published the result on 21 July 2026. In that log /llms.txt was fetched about 660 times and /llms-full.txt about 110 times directly. Of those roughly 770 requests, 37 came from named AI assistants (Perplexity, ChatGPT, Claude, GPTBot and OAI-SearchBot together). The rest came from search crawlers, SEO tools and scripts. In the same log, the agent ChatGPT uses to open a page during a conversation (ChatGPT-User) made 196,973 requests, and 272 of them were for Markdown versions.

This is the log of a single site. Another site may see different numbers, and we have not measured our own site's logs yet. We read the post on 18 October 2026.

Why do we keep one anyway?

Three small reasons:

• It takes little time to write and is easy to maintain.
• Some AI and developer tools may use it when summarizing a site; we could not verify which ones, and in the log above named AI assistants rarely requested it.
• Writing it forces you to decide which ten pages matter most. That decision is valuable on its own.

We keep our own llms.txt and say in the file that Google does not use it. Since we cannot measure its effect, we promise no result.

How do you check whether your site has one?

Open yourdomain.com/llms.txt in a browser. If a plain-text file loads, you have one. Our site scanner, Ruescan, is not open yet (the early-access list is open on the home page). When it opens, it will look for llms.txt and report on AI bot permissions in robots.txt, the sitemap and structured data in the same report.

Frequently asked questions