AI Search Visibility · AuditJet Glossary
llms.txt
A proposed standard file placed at a website's root (like robots.txt) that provides structured guidance to AI language models about the site's content.
The llms.txt file helps AI language models understand a website's content structure, key pages, and preferred summaries for AI indexing. It's similar to robots.txt but oriented toward AI consumption rather than traditional crawlers. The format uses Markdown to describe the site's purpose and link to key content. AuditJet checks for llms.txt presence and validity as part of its AI Visibility audit.
Related terms
A website's ability to be discovered, crawled, and recommended by AI-powered search engines including ChatGPT, Perplexity, Claude, and Gemini.
OpenAI's web crawler used to collect training data and power ChatGPT's web search capabilities.
A text file at a website's root that instructs search engine crawlers which pages or sections of a site should not be crawled.
Monitor llms.txt continuously
AuditJet tracks Core Web Vitals on a schedule with revenue impact alerts.
Start Free