llms.txt is a text file at the root of a website that summarizes, in Markdown, what the site is and which pages matter, designed for language models and AI agents to read in one go. Its long version, llms-full.txt, includes the full content of the pages.
How it differs from robots.txt and the sitemap
- robots.txt tells crawlers what they may and may not visit.
- sitemap.xml lists the URLs and their change date for search engines.
- llms.txt explains the site in plain language: what it is, who it is for, what it solves and where everything is.
How useful it really is
Google has said it does not need files “for AI” and that its generative features use the same indexed pages as Search. So llms.txt is not a ranking lever; it is documentation for agents: it costs nothing to maintain and, for an agent or an assistant that lands on the site, it is the fastest way to understand it.
The ōrbita one: orbitasolutions.org/llms.txt and llms-full.txt. More about bots and agents in /bot (in Spanish).
Also known as: llms-full.txt, file for language models.