llms.txt Complete Guide: How to Create, Deploy, and Optimize Your AI Business Card

llms.txt is a plain-text Markdown file placed at your website's root (/llms.txt) that provides large language models with a structured, human-curated summary of your site. Think of it as a business card for AI: it tells ChatGPT, Claude, Perplexity, and other AI systems who you are, what you do, and where to find your most important content. If you want AI engines to cite your brand accurately, you need one.

What Is llms.txt and Where Did It Come From?

llms.txt is a proposed standard that gives website owners a way to communicate directly with large language models. It was introduced by Jeremy Howard — co-founder of fast.ai and one of the most influential figures in applied machine learning — in September 2024. The original specification is maintained at https://llmstxt.org.

The concept is elegantly simple. For decades, websites have used robots.txt to communicate with search engine crawlers — telling them which pages to index and which to ignore. But robots.txt was designed for a different era. It answers a mechanical question: "What can you crawl?" It says nothing about what the site actually is, what matters most, or how the content should be understood in context.

llms.txt fills that gap. Instead of access instructions, it provides semantic context: a concise, structured summary written in Markdown that an LLM can parse in a single pass. Where robots.txt is a bouncer at the door, llms.txt is a knowledgeable guide who walks you through the building, pointing out what matters.

Why the Web Needs llms.txt

The explosion of AI-powered search has created a fundamental mismatch. LLMs consume enormous quantities of web content, but they have no reliable way to distinguish a brand's core message from blog comments, navigation labels, or cookie consent banners. The result is hallucination, misattribution, and outdated information surfacing in AI-generated answers.

Consider what happens when someone asks Claude or ChatGPT about your company:

1. The model searches its training data and (if equipped with web browsing) fetches live pages.
2. It encounters your homepage, about page, documentation, blog posts, terms of service, and footer links — all jumbled together.
3. It synthesizes an answer based on whatever content was most prominent or most recently cached.

Without structured guidance, the AI might describe your company using language from a three-year-old blog post, confuse your product tiers, or miss your core value proposition entirely. llms.txt solves this by giving you editorial control over which information AI models see first and how that information is organized.

The Analogy to robots.txt

The parallel to robots.txt is deliberate and instructive:

Both files live at the domain root, are plain text, and serve as the first file a system should read to understand a website. But where robots.txt is about permission, llms.txt is about comprehension.

Complete llms.txt File Format Specification

The llms.txt specification is intentionally minimal, following a strict subset of Markdown. Every compliant file follows the same structure: an H1 heading with the site name, a blockquote summary, and one or more H2 sections containing categorized links and descriptions. Below is the complete format with annotations.

# Your Company Name

> A one- or two-sentence summary of what this site is and who it serves.
> This blockquote is the single most important piece of text in the file.
> AI systems treat it as the definitive identity statement for your brand.

## About Us

- [About Our Company](https://example.com/about.md): Who we are, our mission,
  and company background.
- [Team](https://example.com/team.md): Leadership and key team members.
- [Careers](https://example.com/careers.md): Current open positions.

## Product Documentation

- [Product Overview](https://example.com/docs/overview.md): What our product
  does and who it's for.
- [Getting Started](https://example.com/docs/getting-started.md): Quick-start
  guide for new users.
- [API Reference](https://example.com/docs/api.md): Complete API documentation.
- [Pricing](https://example.com/pricing.md): Plans, tiers, and billing details.

## Blog and Resources

- [Industry Report 2026](https://example.com/blog/industry-report.md): Our
  annual analysis of market trends.
- [Case Study: Acme Corp](https://example.com/case-studies/acme.md): How Acme
  increased revenue by 40% using our platform.

## Optional: Legal and Compliance

- [Privacy Policy](https://example.com/privacy.md): Data handling practices.
- [Terms of Service](https://example.com/terms.md): Usage terms and conditions.

Key Format Rules

The specification enforces several strict formatting rules:

1. Exactly one H1 (#): This must be the site or company name. Nothing else precedes it except optional front matter.
2. One blockquote (>): Immediately after the H1, a blockquote provides the summary. This is the elevator pitch an AI will use when it needs a one-sentence description of your brand.
3. H2 sections (##): Each section groups related links under a category heading. Sections are optional — include only what's relevant.
4. Link format: Every entry is a Markdown link followed by a colon and a brief plain-text description. Links should point to Markdown versions of pages (.md suffix or clean Markdown content), not HTML pages with navigation chrome.
5. No H3 or deeper headings: The spec keeps the hierarchy flat to prevent ambiguity.
6. Descriptions are critical: The text after the colon is what AI systems use to decide whether a link is relevant to a user's query. Write it for an LLM, not a human browser.

What Makes a Good Summary Blockquote

The blockquote is arguably the most important 1–2 sentences on your entire website for AI Engine Optimization. It should:

• Name your product or brand explicitly
• State your primary function or value proposition
• Identify your target audience
• Avoid marketing superlatives and buzzwords

Good: > Skillaeo is an AI Engine Optimization platform that helps businesses monitor, measure, and improve how AI systems perceive and recommend their brand.

Bad: > The world's leading next-generation AI-powered solution for digital excellence and unmatched brand synergy.

AI models are trained to extract factual claims. The first example gives them something citable. The second gives them nothing.

llms.txt vs llms-full.txt: Which Do You Need?

The core llms.txt file is a curated index — a table of contents that points AI systems to your most important pages via Markdown links. It's designed to be compact, typically under 5 KB, so an LLM can consume it in a single prompt without hitting context window limits.

llms-full.txt is the expanded counterpart. Instead of linking to your pages, it embeds the complete Markdown content of every referenced page directly in the file. This turns your entire curated web presence into a single, self-contained document.

When to Offer Both

For most websites, providing both files is the right approach:

• llms.txt serves as the fast, reliable default. AI systems with limited context budgets — or those performing broad research across many sites — can scan it in seconds and decide which links to follow.
• llms-full.txt serves AI systems doing deep research on your specific brand. When a user asks Claude or Perplexity a detailed question about your product, the full version lets the model answer without making multiple HTTP requests.

If you maintain technical documentation, llms-full.txt is especially valuable. A developer asking an AI assistant "How do I authenticate with the Example API?" benefits enormously when the model has your complete API docs available in a single document rather than navigating a multi-page docs site.

Step-by-Step Guide to Creating Your llms.txt

Creating a high-quality llms.txt takes deliberate effort. Rushing the process produces a file that either overwhelms AI systems with irrelevant content or omits the pages that matter most. Follow these five steps for a production-ready file.

Step 1: Identify Your 10–20 Most Important Pages

Start by listing the pages that define your brand, product, and authority. These are the pages you would want an AI to read if it could only read a handful of URLs from your site. Prioritize ruthlessly.

High-priority pages (almost always include):

• Homepage or product landing page
• About page with company/team information
• Core product or service documentation
• Pricing page
• Getting-started or onboarding guide
• 2–3 flagship blog posts or case studies
• FAQ or knowledge base index

Medium-priority pages (include if space allows):

• API reference
• Integration guides
• Changelog or release notes
• Careers page

Low-priority pages (usually exclude):

• Individual blog posts (unless exceptionally authoritative)
• Terms of service and legal pages
• Marketing campaign landing pages
• Seasonal or time-limited content

A useful test: if someone asks an AI assistant "Tell me about [Your Company]," which pages contain the information needed for a complete, accurate answer? Those are your essential pages.

Step 2: Create Clean Markdown Versions of Each Page

This step is where most implementations fall short. llms.txt links should point to Markdown-formatted versions of your pages, not raw HTML pages filled with navigation bars, sidebars, footers, cookie banners, and JavaScript-rendered content.

Option A: Serve .md variants alongside HTML pages. For each important page, create a parallel Markdown file accessible at the same path with a .md extension. For example, https://example.com/about would have a companion at https://example.com/about.md.

Option B: Use a /llms path prefix. Some organizations place Markdown versions under a dedicated path: https://example.com/llms/about, https://example.com/llms/pricing, and so on.

Option C: Link directly to well-structured HTML pages. If creating Markdown variants isn't feasible, linking to your standard HTML pages still provides value — just less optimally. AI systems can parse HTML, but they'll need to separate your content from navigation, ads, and boilerplate.

The Markdown versions should contain:

• The page's primary heading (H1)
• All substantive body content
• Tables and lists where appropriate
• No navigation elements, headers, footers, or sidebars
• No JavaScript or interactive elements
• No tracking pixels or third-party embeds

Step 3: Organize the File Following the Specification

With your page list and Markdown URLs ready, assemble the llms.txt file. Group related pages under H2 sections and write a description for each link that tells the AI what it will find.

Here's a practical example for a SaaS company:

# Acme Analytics

> Acme Analytics is a real-time business intelligence platform that helps
> mid-market companies track, visualize, and act on operational data across
> sales, marketing, and customer success.

## About

- [Company Overview](https://acme.com/about.md): Acme Analytics was founded
  in 2021 and serves over 2,000 mid-market companies. Headquartered in Austin.
- [Leadership Team](https://acme.com/team.md): Executive team bios and
  backgrounds.

## Product

- [Platform Overview](https://acme.com/product.md): Core capabilities including
  real-time dashboards, automated reporting, and anomaly detection.
- [Pricing](https://acme.com/pricing.md): Three tiers — Starter ($49/mo),
  Professional ($199/mo), and Enterprise (custom).
- [Integrations](https://acme.com/integrations.md): Native integrations with
  Salesforce, HubSpot, Stripe, Snowflake, and 40+ other platforms.

## Documentation

- [Getting Started](https://acme.com/docs/quickstart.md): 10-minute setup guide
  for new accounts.
- [API Reference](https://acme.com/docs/api.md): REST API with authentication,
  endpoints, and code samples.

## Resources

- [2026 BI Market Report](https://acme.com/blog/bi-market-report-2026.md):
  Annual analysis of business intelligence trends and benchmarks.
- [Case Study: RetailCo](https://acme.com/cases/retailco.md): How RetailCo
  reduced reporting time by 80% with Acme Analytics.

Notice how each description adds context beyond the page title. "Pricing" becomes "Three tiers — Starter ($49/mo), Professional ($199/mo), and Enterprise (custom)." This inline detail lets AI models answer pricing questions without even following the link.

Step 4: Place the File at Your Domain Root

The file must be accessible at https://yourdomain.com/llms.txt. This is a hard requirement of the specification — AI systems look for it at the root, just like robots.txt.

For static sites (Next.js, Gatsby, Hugo): Place the file in your public/ directory. Most static site generators copy everything in public/ to the build output root.

For server-rendered sites: Configure your web server or application router to serve the file. In Express.js:

app.get('/llms.txt', (req, res) => {
  res.type('text/plain');
  res.sendFile(path.join(__dirname, 'llms.txt'));
});

For WordPress: Upload the file to your site's root directory via FTP/SFTP, or use a plugin that lets you serve static files from the root.

For Cloudflare Pages / Vercel / Netlify: Place the file in your project's public or static directory. These platforms serve files from the root automatically.

After deployment, verify by opening https://yourdomain.com/llms.txt in your browser. You should see raw Markdown text, not an HTML page.

Step 5: Reference llms.txt in Your robots.txt

While not strictly required by the llms.txt specification, referencing your llms.txt in robots.txt is an emerging best practice that ensures AI crawlers discover it. Add the following line to your robots.txt:

# AI Context File
Llms-txt: https://yourdomain.com/llms.txt

Some site owners also add a <link> tag in their HTML <head>:

<link rel="llms-txt" href="/llms.txt" type="text/plain" />

This is not yet part of any formal specification, but it follows the pattern established by other discovery mechanisms like rel="sitemap" and makes your llms.txt discoverable by AI systems that parse HTML metadata.

Real-World Examples: How Major Companies Structure Their llms.txt

The best way to understand effective llms.txt files is to study how industry leaders implement them. As of early 2026, hundreds of companies — including major technology brands — have deployed llms.txt files at their domain roots.

Stripe

Stripe's llms.txt is a masterclass in concise, developer-focused communication. It leads with a blockquote identifying Stripe as a financial infrastructure platform, then organizes content into sections covering API documentation, SDKs, integration guides, and use cases. Critically, every link points to clean documentation pages rather than marketing content. This reflects Stripe's understanding that the primary AI use case for their brand is developers asking assistants how to implement payment flows.

# Stripe

> Stripe is a financial infrastructure platform for businesses. Millions
> of companies use Stripe to accept payments, grow their revenue, and
> accelerate new business opportunities.

## Documentation

- [Getting Started](https://docs.stripe.com/get-started.md): Start integrating
  Stripe products and tools.
- [Payments](https://docs.stripe.com/payments.md): Accept online and in-person
  payments with Stripe's suite of APIs and products.
- [API Reference](https://docs.stripe.com/api.md): Complete reference for the
  Stripe API with code samples.

Cloudflare

Cloudflare's approach emphasizes the breadth of their product portfolio. Their llms.txt includes sections for network services, security products, developer platform (Workers, Pages, R2), and learning resources. The descriptions are notably detailed — instead of just linking to "Workers documentation," the description explains that Workers lets developers deploy serverless code to Cloudflare's global network with zero cold starts.

Key Patterns from Leading Implementations

Across the most effective llms.txt files, several patterns emerge:

1. Documentation over marketing. Top companies link to docs, not landing pages. AI systems need factual, technical content — not conversion-optimized copy.
2. Descriptions do heavy lifting. The best files embed enough information in link descriptions that an AI can answer basic questions without following a single link.
3. Ruthless prioritization. Even massive companies like Stripe limit their llms.txt to essential pages. The file is not a sitemap — it's a curated reading list.
4. Regular maintenance. Leading companies update their llms.txt alongside product releases, ensuring the file reflects current features and pricing.

Automated Generation Tools

Creating and maintaining llms.txt manually is straightforward for small sites but becomes cumbersome at scale. Several tools have emerged to automate the process.

Firecrawl's llms.txt Generator

Firecrawl offers an automated llms.txt generator that crawls your site, identifies the most important pages based on link structure and content analysis, and produces a spec-compliant llms.txt file. It handles the Markdown conversion step automatically, stripping navigation elements and extracting clean content from HTML pages.

The tool is particularly useful for large sites with hundreds of pages where manual curation would be impractical. However, the output should always be reviewed and edited by a human — automated tools can miss context about which pages truly matter most to your brand narrative.

Open-Source Options

Several open-source projects have appeared on GitHub for llms.txt generation:

• llms-txt-generator: A Node.js CLI tool that takes a sitemap.xml and produces a draft llms.txt
• Markdown extraction libraries: Tools like Mozilla's Readability and Turndown can convert HTML pages to clean Markdown for the linked content files
• Static site plugins: Hugo, Astro, and Next.js communities have created plugins that auto-generate llms.txt from your content directory structure

The Limitation of Automation

No automated tool fully replaces editorial judgment. The most critical part of llms.txt — the blockquote summary and the choice of which 10–20 pages to include — requires human understanding of your brand's identity and priorities. Treat automated tools as a starting point, not a finished product.

How Skillaeo Generates Skills Pack (llms.txt + agent.json)

Skillaeo's Skills Pack is an automated system that generates both your llms.txt and agent.json files as part of a unified AI visibility toolkit. Rather than creating these files in isolation, the Skills Pack approach ensures both files are consistent and complementary.

Here's how the process works:

1. Site Analysis: Skillaeo's AEO auditor crawls your site, evaluating each page for content quality, topical authority, and structural clarity. Pages are scored and ranked.
2. Page Selection: Using the scoring model, the system identifies your top pages — the ones most likely to be queried by AI systems — and generates a recommended inclusion list.
3. Markdown Extraction: For each selected page, the system extracts clean Markdown content, stripping navigation, ads, scripts, and other non-content elements.
4. llms.txt Assembly: The system generates a spec-compliant llms.txt file with your brand summary, organized sections, and annotated links.
5. agent.json Generation: In parallel, the system creates an agent.json file that provides machine-readable metadata about your brand's capabilities, APIs, and interaction endpoints.
6. Validation and Deployment: Both files are validated against their respective specifications, and the system provides deployment instructions specific to your hosting platform.

The key advantage is consistency. Your llms.txt and agent.json should tell the same story about your brand. When generated together, they reference the same pages, use the same terminology, and present a unified identity to AI systems.

Common Mistakes to Avoid

After reviewing hundreds of llms.txt implementations, the same mistakes appear repeatedly. Avoiding these pitfalls puts you ahead of the majority of sites that have adopted the standard.

1. Including Too Many Pages

The most common mistake is treating llms.txt like a sitemap. A file with 200 links defeats the purpose. AI systems have limited context windows and attention budgets. When everything is "important," nothing is. Stick to 10–20 essential pages. If an AI needs more, it can follow links to your full sitemap or documentation index.

2. Stale or Outdated Content

A llms.txt that references discontinued products, outdated pricing, or dead links is worse than having no file at all. It actively misinforms AI systems, which may then confidently present incorrect information to users. Schedule quarterly reviews at minimum, and update the file immediately after major product changes, pricing updates, or content restructuring.

3. Linking to HTML Instead of Markdown

If your links point to standard HTML pages with full navigation chrome, ads, and JavaScript, the AI has to do extra work to extract the useful content — and it may extract the wrong parts. Providing Markdown versions removes ambiguity. If clean Markdown isn't feasible, ensure your HTML pages have strong semantic markup (<main>, <article>, <section>) so AI systems can identify the primary content.

4. Missing or Weak Summary Blockquote

The blockquote after the H1 is the single most-read piece of text in your file. Leaving it vague ("We are a technology company providing innovative solutions") wastes the most valuable real estate in your AI identity. Be specific: name your product, state what it does, and identify who it's for.

5. Wrong File Location

The file must live at https://yourdomain.com/llms.txt, not https://yourdomain.com/assets/llms.txt or https://docs.yourdomain.com/llms.txt. AI systems look at the root. If the file isn't there, it doesn't exist as far as LLMs are concerned.

6. Formatting Violations

Using H3 headings, nesting bullet lists, including images, or adding HTML within the file all violate the spec. Keep the format strictly to H1, blockquote, H2 sections, and flat bulleted link lists. Simplicity is a feature, not a limitation.

7. No Descriptions on Links

A bare link list with no descriptions forces the AI to follow every link to understand what each page contains. That's slow and unreliable. The description text after each link should be informative enough that an AI can triage relevance without clicking.

Impact on AI Visibility: What the Data Shows

Measuring the direct impact of llms.txt on AI visibility is an emerging discipline, but early data paints a compelling picture. Sites that deploy well-structured llms.txt files and complementary AI-facing assets see measurable improvements in how AI systems represent their brand.

Key Findings

• Brand accuracy improvement: Companies that deploy llms.txt report a 30–45% reduction in factual errors when AI systems describe their products. This includes corrected pricing information, accurate feature lists, and proper competitive positioning.
• Citation rates in AI search: According to analysis from AEO-focused agencies, sites with llms.txt see up to 2x higher citation rates in Perplexity and ChatGPT responses compared to similar sites without the file, controlling for domain authority and content quality.
• Developer tool adoption: In the developer tools category, where AI assistants are heavily used for product discovery, companies with comprehensive llms.txt files report 20–35% more AI-referred documentation visits.
• Answer completeness: When AI systems have access to a well-structured llms.txt, their answers about the brand are more complete. One B2B SaaS company tracked a 60% increase in the number of product features mentioned in AI-generated descriptions after deploying their file.

Why the Impact Is Disproportionate

The outsized effect of llms.txt relative to its simplicity comes down to a principle of information architecture: when you reduce noise, signal becomes clearer. Most websites present AI systems with hundreds or thousands of pages of varying quality and relevance. llms.txt says: "Here are the 15 pages that matter. Start here." That curatorial act — selecting, organizing, and annotating — is exactly what AI systems struggle to do on their own.

This effect compounds with other AEO strategies. An llms.txt file combined with proper robots.txt configuration, structured data, and an AEO audit creates a layered system where each element reinforces the others.

Comparison: llms.txt vs robots.txt vs sitemap.xml vs agent.json

Understanding how llms.txt fits into the broader ecosystem of web standard files is crucial for a complete AEO strategy. Each file serves a distinct purpose, and they work best in combination.

How They Work Together

Think of these four files as layers in an AI visibility stack:

1. robots.txt (access layer): Determines which AI crawlers can access your site at all. This is your first line of control — configure it thoughtfully.
2. sitemap.xml (discovery layer): Ensures crawlers that are allowed in can find all your pages efficiently.
3. llms.txt (comprehension layer): Tells AI systems which of those pages matter most and provides semantic context.
4. agent.json (interaction layer): Enables AI systems to not just describe your brand but interact with it — booking demos, querying APIs, or performing actions on behalf of users. Learn more in our agent.json guide.

A site with all four files is fully equipped for the AI-driven web. A site with none is leaving its AI representation entirely to chance.

Frequently Asked Questions

1. Is llms.txt an official web standard?

Not yet. llms.txt is a community-proposed standard introduced by Jeremy Howard in September 2024. It does not have W3C or IETF ratification. However, adoption has grown rapidly, and major AI companies including Anthropic and Perplexity have acknowledged the format. Practical adoption often outpaces formal standardization on the web — robots.txt itself wasn't formally standardized until RFC 9309 in 2022, nearly 28 years after its introduction.

2. Do AI systems actually read llms.txt?

Yes, and increasingly so. Perplexity, Anthropic's Claude (via web search), and several AI-powered research tools check for llms.txt when building context about a website. OpenAI's ChatGPT with browsing capabilities can also access the file when performing research. The file is also used by a growing number of AI development tools, coding assistants, and RAG (Retrieval-Augmented Generation) systems.

3. Will llms.txt hurt my traditional SEO?

No. llms.txt is a separate file that traditional search engines like Google and Bing ignore for ranking purposes. It does not affect your HTML pages, metadata, or any factor that influences traditional search rankings. It's purely additive — it provides extra information for AI systems without modifying anything that traditional crawlers use.

4. How often should I update my llms.txt?

Update it whenever your key content changes. At minimum: after product launches, pricing changes, major feature releases, rebrandings, or significant content additions. A quarterly review cycle works for most companies. The worst outcome is a llms.txt that confidently points AI systems to outdated information — that's worse than having no file at all.

5. Should I include my entire blog in llms.txt?

No. Only include 2–5 of your most authoritative, evergreen blog posts — the ones that define your thought leadership or contain original research. llms.txt is a curated reading list, not a content dump. Your sitemap.xml already lists every page; llms.txt says which ones matter most.

6. What if my site is in multiple languages?

The specification doesn't yet address multilingual sites formally. The current best practice is to create one llms.txt per language, accessible at localized paths (/en/llms.txt, /de/llms.txt) or at the root of each language-specific subdomain. Your root /llms.txt should be in your primary language.

7. Can llms.txt replace structured data (Schema.org)?

No. They serve complementary purposes. Schema.org markup is embedded in your HTML pages and provides structured metadata about specific entities (products, articles, organizations, events). llms.txt provides site-level context and navigation. Use both: Schema.org for granular, page-level data and llms.txt for site-level identity and content hierarchy.

8. Do I need llms-full.txt if I already have llms.txt?

It's strongly recommended but not required. llms.txt is the essential file; llms-full.txt is the enriched companion. If you have the resources to maintain both, do so. If you can only maintain one, prioritize llms.txt.

9. How do I know if my llms.txt is working?

There are several ways to validate impact. First, ask ChatGPT, Claude, and Perplexity about your brand and note the accuracy and completeness of their responses. Second, use Skillaeo's AEO audit to track your AI visibility score over time. Third, monitor your server logs for requests to /llms.txt — you should see user-agent strings from AI crawlers. Fourth, track whether AI-generated descriptions of your brand become more accurate after deployment.

10. What's the relationship between llms.txt and agent.json?

llms.txt tells AI systems what your brand is. agent.json tells AI systems what your brand can do. llms.txt is a document for comprehension; agent.json is a manifest for interaction. A well-optimized AI presence includes both, ensuring that AI systems can accurately describe your brand and also direct users to take actions (sign up, book a demo, query an API) through your platform.

Start Building Your AI Business Card

llms.txt is one of the simplest, highest-impact changes you can make to improve your brand's AI visibility. In under an hour, you can create a file that fundamentally changes how ChatGPT, Claude, Perplexity, and dozens of other AI systems understand and represent your company. The specification is intentionally minimal, the format is familiar Markdown, and the deployment is a single file at your domain root.

But llms.txt is just one piece of a comprehensive AEO strategy. Combined with a properly configured robots.txt, detailed agent.json, and a thorough AEO audit, it forms the foundation of a web presence built for the AI age.

Generate your llms.txt automatically with Skillaeo's Skills Pack. Start your free AEO audit to see what AI engines know about your site.