What is llms.txt and does your website need one?

llms.txt is a plain Markdown file at yoursite.com/llms.txt. In a few hundred words, it tells AI agents who you are and which pages hold the facts they need. It doesn't control crawling. It doesn't improve rankings. Google states that Google Search ignores it. For most B2B companies it's a one-hour task with a small, uncertain payoff. My take: write one, keep it accurate, and spend the real effort elsewhere.

What is llms.txt?

Jeremy Howard, co-founder of fast.ai and Answer.AI, published the proposal on 3 September 2024 at llmstxt.org. The idea is simple. Web pages are built for people. They wrap information in navigation, scripts and cookie banners, and turning that back into clean text is imprecise. An AI agent with a limited context window does better with a short, expert summary in one predictable place.

In August 2026 the proposal moved to version 2. If you implement it properly, these changes matter:

• Scope by path. A file can sit at the root or in a subfolder such as /docs/llms.txt. It covers the pages under its path. Agents should use the most specific file.
• Markdown copies of pages. Pages agents might need should have a clean Markdown version at the same URL, either page.html.md or page.md.
• Discovery through link relations. A page can point to its Markdown version with rel="alternate" plus type="text/markdown" and to its covering llms.txt with rel="describedby", as an HTML link element or an HTTP Link header.
• No special tooling. Version 1 described a tool that expanded the file into one big context block. Version 2 expects agents to read the file and follow only the links they need.

The spec itself says llms.txt sees the heaviest use in software documentation, where coding agents follow it to find API references. It also names business sites and personal CV sites as good fits.

What does an llms.txt file look like?

It's Markdown in a fixed order, so a language model and a simple parser can both read it:

1. An H1 with the name of the site or company. It's the only required element.
2. A blockquote with a short summary that holds the key facts.
3. Optional paragraphs or lists with more detail, no headings.
4. Optional H2 sections, each with a list of Markdown links and a short note after each link.
5. By convention, an H2 called Optional for secondary links an agent can skip.

Here's a shortened version of the file on this site. It's a typical shape for a small B2B advisory:

# Thomas Pohle

> B2B marketing advisor based in Zurich. Thomas gets marketing moving with the client's team: workshops, projects and advisory mandates. Legal entity: Pohle Advisory GmbH, Hedingen, Switzerland.

- Audience: founders and CEOs of DACH B2B companies with 1 to 200 people.
- Experience: 10+ years in B2B marketing, in-house at Link11, Beekeeper and smino.

## Formats

- [Momentum Workshop](https://thomas-pohle.com/#how): one day, positioning, ideal customer and the first channel.
- [Momentum Project](https://thomas-pohle.com/#how): four to twelve weeks, one campaign or launch.

## Pages

- [Home](https://thomas-pohle.com/): problem, formats, about, contact.
- [Imprint](https://thomas-pohle.com/imprint.html): legal entity and address.

## Optional

- [TEISENDA](https://teisenda.com/): the AI-native marketing operating system Thomas co-founded.

You'll also see files called llms-full.txt that dump a whole site into one file. The spec on llmstxt.org doesn't define them.

Which AI systems actually read llms.txt?

Most articles get vague here. So here's what the providers themselves say, checked in October 2026.

Google is the only one with a clear public position. Its guide to optimizing for generative AI features lists llms.txt under things you can ignore. You don't need new machine-readable files, AI text files or Markdown to appear in Google Search. Creating one is fine. It won't harm or help your visibility, because Google Search ignores it. The separate page on AI features adds that AI Overviews and AI Mode need no special optimizations.

OpenAI, Anthropic and Perplexity all host llms.txt files for their own developer documentation. So they find the format useful for documentation and coding agents. That doesn't mean their search crawlers read your file or use it to pick citations. None of their crawler pages says so.

The Chrome Lighthouse check is narrow. It flags a server error when Lighthouse requests the file. A missing file returns a 404 and counts as not applicable, because the file is, in Google's words, optional at the moment.

The agent case is what stays plausible. Ask an AI assistant with browsing, or a coding agent, to look into a company, and it may fetch /llms.txt because it's a known convention. That's a real benefit. It's also small and hard to measure.

Does llms.txt improve SEO or AI visibility?

No primary source shows that it does. Google says it doesn't for Search. The other providers say nothing. Independent studies exist, but they're vendor log analyses of small site samples. I wouldn't base a decision on them.

What actually decides whether AI systems find and quote you is documented and boring. Crawlers need access through robots.txt and your CDN. Pages need to be indexed. Content needs to answer the question directly. Your company needs to be a clear entity with consistent data across the web. The guide to getting cited by ChatGPT, Perplexity and AI Overviews covers those levers one by one. The overview of answer engine optimization explains how AI answers pick their sources.

One point trips people up. llms.txt has no allow or disallow rules. It can't block GPTBot or invite PerplexityBot. Access is still controlled in robots.txt.

When is llms.txt worth it for a B2B company?

In B2B, it earns its hour in three situations:

• You have developer documentation or an API. That's the original use case, and coding agents do read these files. Docs platforms often generate one for you.
• Your site is heavy on scripts and light on text. If your product pages render client-side, a clean Markdown summary gives agents a reliable path to the facts.
• Your company is easy to confuse. A common name, a recent rebrand, or a legal entity that differs from the brand. A clear summary with legal name, location and what you do helps any agent that reads it.

None of these apply? It's still cheap insurance. Just don't let anyone sell it to you as an AI visibility project.

How do you write an llms.txt file this week?

1. Write the summary first. Two or three sentences in the blockquote: what the company does, for whom, where it's based, and the legal entity name.
2. Add hard facts as a short list. Audience, markets, founding year, core offers, contact. Only facts that also appear on your website.
3. Pick 5 to 15 links. Home, offers or products, pricing if public, about, case studies, contact, imprint. Group them under H2 sections such as Services and Company.
4. Write one useful note per link. Say what question the page answers. Skip the slogan.
5. Put secondary links under Optional. Blog archive, careers, sister brands.
6. Upload it as plain text. Serve it at /llms.txt with a 200 status and a text content type. Check it in the browser and with Lighthouse to rule out server errors.
7. Consider Markdown copies. If your CMS can output .md versions of key pages, link to those and add the link relations from version 2 of the spec. If it can't, link to the HTML pages. That still works.
8. Put it on your update list. Every time an offer, price or address changes, the file changes too.

What mistakes should you avoid?

• Contradicting your website. If llms.txt says 40 employees and the about page says 25, you've created exactly the inconsistency you wanted to avoid. Your structured data, profiles and llms.txt should all tell the same story. The article on connecting your website and profiles into one brand shows how.
• Stuffing it with keywords or claims. A model reading a list of superlatives learns nothing. Write it like a briefing for a smart new colleague.
• Treating it as structured data. llms.txt doesn't replace Organization or Person markup. For that, see which schema markup a B2B website needs.
• Forgetting the basics. An llms.txt on a site that blocks AI crawlers or has indexing problems helps no one. Run through the technical SEO checklist first.
• Assuming it gets read. Check your server logs for requests to /llms.txt and note which user agents fetch it. That tells you more than any blog post. The guide on measuring organic and AI search visibility explains what to look for.

So, does your website need one?

Need one? No. Nothing in Google's or the AI providers' documentation makes llms.txt a requirement, and Google says outright that it has no effect on Search. Want one? Probably, if you treat it as a one-hour hygiene task and keep it in sync with your site. The standard is young, agent use is growing, and an accurate file costs close to nothing.

Rather sort out the bigger levers first? Positioning, entity data and the content AI systems actually quote. That's what we cover in a one-day Momentum Workshop.

Frequently asked questions

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers which URLs they may fetch, and AI providers such as OpenAI, Anthropic and Perplexity document that their crawlers respect it. llms.txt has no allow or disallow rules. It's a Markdown summary with links that helps an agent find the right pages. Want to block or allow an AI crawler? You still do that in robots.txt.

Does Google use llms.txt for AI Overviews?

No. Google's Search Central guide to optimizing for generative AI features says Google Search ignores llms.txt files. You don't need new machine-readable files, AI text files or Markdown to appear in Google Search. Creating one is fine. It won't harm or help your visibility in Search. That covers AI Overviews and AI Mode.

Where do I put the llms.txt file?

At the root of your domain, so it loads at yourdomain.com/llms.txt, served as plain text with a 200 status. Version 2 of the spec also allows files in subfolders, such as /docs/llms.txt. Those cover only the pages under that path. If several files apply to a page, agents should use the most specific one.

Do I need an llms-full.txt as well?

The spec on llmstxt.org doesn't define llms-full.txt. Some documentation tools generate one with the full text of a site in a single file. For a typical B2B marketing site it adds little. Version 2 of the spec prefers a short llms.txt that links to clean Markdown copies of individual pages, so agents fetch only what they need.

How can I tell whether AI agents read my llms.txt?

Check your server or CDN logs for requests to /llms.txt and group them by user agent. That shows which bots and tools actually fetch it. JavaScript-based analytics won't see these requests, because crawlers and agents usually don't run your tracking scripts. Check monthly and compare with requests to robots.txt.

Sources

1. The /llms.txt file, v2 (llmstxt.org)
2. Changes from v1 to v2 (llmstxt.org)
3. Google's guide to optimizing for generative AI features on Google Search
4. AI features and your website (Google Search Central)
5. Lighthouse: llms.txt audit (Chrome for Developers)
6. Overview of OpenAI crawlers
7. Does Anthropic crawl data from the web, and how can site owners block the crawler?
8. Perplexity crawlers
9. llms.txt of thomas-pohle.com