Meta-Scan

AI-Friendly AEO & GEO & SEO Analyzer

Blog
ONLINE
Support
© 2026 Meta-Scan. All rights reserved.
BlogAboutPrivacy PolicyTerms of Service
SYSTEM ONLINE
Home/Blog

What Is llms.txt? How to Make One, What to Expect

Yuuuk94·Published 2026-11-12

Summary

llms.txt is a Markdown file at your site root (/llms.txt) that tells AI models what the site is and where its important pages are. It was proposed in September 2024 by Jeremy Howard of Answer.AI.

It takes ten minutes to write and costs nothing. But it isn't an official standard yet, and few major AI search providers say they use it for ranking or citation. Make one with accurate expectations.

01Why the file exists

Web pages carry a lot of material unrelated to the content: navigation, ads, scripts. A model has a limited context window when it answers, so reading a whole site as raw HTML is wasteful.

The idea behind llms.txt is simple: alongside the HTML meant for people, offer a short summary and a list of key links in Markdown that is easy for AI to read. It is especially useful for well-structured sites like developer documentation.

02Format: a fixed order

The format proposed at llmstxt.org follows the order below. Only the H1 on the first line is required, but the rest is what makes the file useful.

  1. # Site name (H1, required)
  2. > One- or two-sentence summary (blockquote)
  3. Optional paragraphs with more detail about the site
  4. ## Section name headings, each followed by a - [Link title](URL): description list
  5. An ## Optional section for links that can be skipped when context is short
# META_SCAN.BIZ

> Free SEO/AEO/GEO checker. Enter one URL to check meta tags,
> robots.txt, sitemap, structured data, AI signals and Lighthouse.

## Tool

- [META_SCAN.BIZ](https://meta-scan.biz/en): checker home

## Blog

- [How to check robots.txt](https://meta-scan.biz/en/blog/robots-txt): syntax and common mistakes

## Optional

- [Terms](https://meta-scan.biz/en/terms)

03Common mistakes

MistakeProblemFix
/llms.txt returns the home page HTMLLooks like the file exists but it doesn't (soft 404)Make sure it responds with real text/plain content
Listing every pageThe summary stops being a summaryKeep it to the 10–30 key pages
Links without descriptionsThe model can't tell which page to readAdd a one-line : description to each
Stale contentPoints to pages that no longer existGenerate it at build time, like a sitemap

Real example

META_SCAN itself used to return the home page with a 200 at /llms.txt. The file didn't exist and the router caught every path. Open the URL directly and check that the response content-type is text/plain.

04How to add one in common setups

The simplest option is a static file. If your site keeps adding pages, though, a hand-maintained file goes stale quickly, so generating it at build time like a sitemap is better.

SetupHow
Static site or plain hostingPut llms.txt in your static folder (often public/) so it is served from the root.
Next.js (App Router)Create a GET handler in app/llms.txt/route.ts that responds with text/plain, building the link list from your content.
WordPressUse a plugin that generates llms.txt, or upload the file to the site root.

Whatever you choose, open the URL after deploying, confirm the content-type header is text/plain, and check that the Markdown shows as-is.

05What to expect from it

To be candid, there is little evidence yet that llms.txt improves AI search visibility. Google says no special AI text files or markup are needed to appear in AI features such as AI Overviews.

There are still reasons to make one. Developer tools and agents that read site documentation increasingly use the file, and it costs almost nothing. Writing it also forces you to decide what your site's key pages are, which helps SEO directly.

What matters more for AI search: not blocking AI crawlers in robots.txt, having the main text in the HTML without JavaScript, answer-first paragraphs and FAQs, and structured data. META_SCAN's AI signals card checks these together.

FAQ

Q. Without llms.txt, will I be left out of AI search?
No. Most AI search products crawl ordinary web pages to build answers. llms.txt is an optional extra signal, and there is no evidence of a penalty for not having one.
Q. How is it different from robots.txt?
robots.txt is a set of rules that allow or block crawlers. llms.txt is a document that summarizes your site for AI. llms.txt can't block anything.
Q. What is llms-full.txt?
A variant that inlines the full text of key documents into one file instead of linking to them. It is mostly used by documentation-heavy developer sites. For a typical service site, llms.txt alone is enough.

Related reading

  • How to Check robots.txt and 5 Common Mistakes
  • SEO vs AEO vs GEO: What Each One Means
  • How to Get Your Site Cited by ChatGPT and AI Search

References

  • llmstxt.org: The /llms.txt file (original proposal)
  • Google Search Central: AI features and your website
  • OpenAI: Overview of OpenAI Crawlers

Author

Yuuuk94

Frontend developer who grew a product's MAU by 350% using SEO and Lighthouse as the yardstick, then turned that checklist into META_SCAN.

More about me →

How does your site do?

Enter one URL to run the free SEO/AEO/GEO checklist, including everything covered in this post.

Check for free