llms.txt
A markdown file at a site's root that gives a language model a short, curated map of the site instead of leaving it to infer structure from rendered HTML.
What it is
llms.txt is a plain markdown file served at /llms.txt: the site's name, a one-line summary, then the pages worth reading, each with a link and a short description. HTML is written for browsers, so a model reading it must guess which content matters.
What it gives the knowledge layer
It gives an agent engine a small, stable entry point and an account, written by the site itself, of the knowledge it exposes. An engine meeting an unfamiliar domain can read one file and learn which pages exist, instead of crawling the site and reconstructing its shape.
Where the spec lives
The convention is documented at llmstxt.org, which states the file's purpose, its placement at the site root, and a suggested structure: an H1 with the project name, a blockquote summary, then markdown links with brief descriptions. This site publishes its own example at /llms.txt.
What it does not solve
It is a convention, not a standard with a validator: nothing checks that a file is well formed and nothing enforces that it is current. It is also not a substitute for the pages themselves — the index says where knowledge lives, not whether it is any good.
FAQ
It is a convention, not a standard: no standards body owns it, there is no conformance clause, and no validator will tell you whether a file is correct.
No. A sitemap enumerates URLs for crawlers and robots.txt states access rules; llms.txt is a curated summary of what the site contains and which parts matter.
It gives an agent engine a small, stable entry point that names the knowledge a site exposes and links to the pages themselves, making the site's own account the starting point.
Nothing enforces freshness, so a file accurate at publication can drift. It does not make the underlying pages correct, and it does not replace them.
Sources
- llmstxt.org — the convention's own description: purpose, placement at the site root, and suggested structure.