llms.txt
A proposed convention for pointing AI systems toward concise, useful site resources.
Definition
llms.txt is a proposed Markdown file at /llms.txt that gives LLMs and agents a curated overview of important site content and links.
Why It Matters
It can help AI tools or users quickly locate clean documentation, but it is not a universally adopted ranking standard.
How AI Uses It
When supported or manually supplied, it acts as a compact map to high-value Markdown or HTML resources.
Commerce Example
A DTC brand lists product guides, policy pages, ingredient glossary, fit guide, and canonical facts page.
Copy/Paste Prompts
Replace the bracketed placeholders and run these prompts against your priority product lines, categories, or brand pages.
Draft an llms.txt file for this commerce site using only the most useful AI shopping resources: [SITE MAP].Audit this llms.txt for stale links, vague descriptions, and missing buyer-intent sections: [FILE].Optimization Checklist
- Use an H1 title.
- Add a short summary.
- Group links by intent.
- Describe each URL.
- Keep the file current.
Common Data Gaps
| Gap | Why AI Struggles | Fix |
|---|---|---|
| File lists every page | Signal is diluted. | Curate only high-signal resources. |
| No page descriptions | Agents cannot choose the right URL. | Add one-line purpose notes. |
| Stale links after CMS changes | The map breaks. | Regenerate from sitemap or content inventory. |
Downloadable-Style Artifacts
Copy this structure into a spreadsheet, Notion page, or internal ticket.
llms.txt operating worksheet
| Primary audit question | Use an H1 title. |
|---|---|
| Highest-risk gap | File lists every page |
| First fix to ship | Curate only high-signal resources. |
| Success metric | llms.txt fetches |
| Retest cadence | Monthly or after material catalog changes |
Title: Improve llms.txt readiness for [PRODUCT / CATEGORY]
Observed issue:
[WHAT THE AI ANSWER MISSED OR MISSTATED]
Most likely data gap:
File lists every page
Recommended fix:
Curate only high-signal resources.
Affected prompt:
[PASTE PROMPT]
Owner:
[TEAM OR PERSON]
Acceptance criteria:
- Use an H1 title.
- Add a short summary.
- Track: llms.txt fetches
- Prompt test has been re-run after publicationCommon Mistakes
- Calling it a replacement for SEO.
- Using it for access control.
- Stuffing promotional claims.
- Listing low-quality pages.
What To Measure
- llms.txt fetches
- Linked page crawl rate
- Broken links
- AI answer improvement in tests
Strategic Takeaway
Treat llms.txt as a helpful map, not a magic visibility switch.