LLMs.txt Curator

LLMs.txt Curator helps site owners generate and curate a high-quality llms.txt file for AI assistants and retrieval systems, ensuring only relevant, well-described content is exposed to large language models.

It generates and maintains llms.txt and llms-full.txt, the emerging standard for telling AI systems (ChatGPT, Claude, Perplexity, Gemini, and others) which pages on your site matter most and what they contain.

Unlike auto-generators that dump every URL into a flat file, LLMs.txt Curator takes a curation-first approach. You choose the pages, organise them into sections, fill descriptions, override titles for AI, validate quality, and see exactly which AI bots are reading your file, all from a single interface.

What llms.txt is, and what it isn’t (June 2026)

Being straight with you matters more than overselling. Google Search ignores llms.txt: it will not help or harm your Google rankings, and Google confirmed in June 2026 that it is fine to maintain the file for other systems that do read it. So this plugin is not an “AI ranking” tool.

Where the file genuinely earns its place: on-site navigation for agents already on your site, the surfaces that actually read it (such as Perplexity and coding agents), and passing Chrome’s Agentic Browsing audit. The honest, useful question is not “does this rank me” but “which AI bots are actually reading my file”, and this plugin answers it from your own server logs.

What makes this different

Most llms.txt plugins treat the file as a static output. LLMs.txt Curator treats it as a living asset:

• Quality Score: every generated file shows your coverage percentage, which pages have descriptions, and which still need attention.
• Description Suggestions: one click fills all missing descriptions from a five-step fallback chain (schema → SEO meta → excerpt → Open Graph → page content). Never overwrites what you’ve set manually.
• Change Detection: a banner appears in your admin when curated pages have been updated since the last generation, with a Regenerate Now button. Your llms.txt stays current without manual checking.
• Per-Page Title Override: set a different title for each page specifically for AI consumption, without touching your on-site SEO.
• Safety Mode: blocks generation if validation errors exist, preventing a broken file from going live. Includes an on-demand validator with plain-English results.
• AI Crawler Analytics: see which bots visited in the last 7 days, with a visual bar chart. OpenAI, Anthropic, Perplexity, Google: know who is actually reading your file.

Core features

• Drag-and-drop curation: reorder sections and pages visually; each section becomes an ## heading per the spec
• llms-full.txt generation: the companion file with full Markdown content for each curated page
• Five SEO plugin integrations: Rank Math, Yoast SEO, All in One SEO, SEOPress, The SEO Framework
• Schema-aware descriptions: uses structured data before falling back through the description chain
• Scheduled regeneration: choose Instant (on publish), Daily, Weekly, or Manual
• WooCommerce support: SKU, price, stock, categories, dimensions; respects product visibility
• WordPress Multisite: network-activate across all sites; each site manages its own independent llms.txt; Network Admin overview with per-site and bulk regenerate
• Pre-built templates: Business, E-commerce, SaaS, Blog, Local Business
• Live preview: see your exact llms.txt output before saving
• Import / export: move your configuration between sites as JSON
• WP-CLI: wp llms-txt regenerate, wp llms-txt status, wp llms-txt crawler-log
• REST API: POST /wp-json/llms-txt/v1/regenerate, GET /wp-json/llms-txt/v1/status
• Atomic file writes: temp file → rename; no half-written files served to bots
• ETag/304 caching: proper HTTP headers for CDN revalidation
• Subdirectory / Bedrock support: correctly finds the site root on non-standard installs
• Robots.txt reference: automatically adds a spec-compliant comment

Description Suggestions in detail

When pages lack descriptions, AI systems get less context. The suggestion engine fills the gap automatically:

1. Schema markup (Rank Math, custom _schema_json)
2. SEO plugin meta description
3. WordPress excerpt
4. Open Graph description (_og_description / og_description)
5. First 160 characters of post content

Already-set descriptions are never touched. Pages that can’t be filled automatically are listed for manual review.

Quality Score

In the admin, the Quality Score shows how many of your listed pages have descriptions, and which ones still need one.

Every generated llms.txt ends with a short coverage report containing counts only:

Description coverage: 94%
Pages included: 48
Pages with descriptions: 45
Pages missing descriptions: 3

The list of pages that still need a description is never published in the file. You’ll find it in the admin, under Suggest Descriptions and Validate.

AI Crawler Analytics

Track 12 known bots: GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, Bytespider, CCBot, Cohere, DeepSeek, Amazonbot.

The 7-day analytics card shows a visual bar chart of recent activity. All-time totals are kept separately. IP addresses are anonymised before storage: last octet zeroed for IPv4, last 80 bits for IPv6. No data leaves your server.

Safety Mode

Before generation, the validator checks:

• Pages missing or unpublished
• Duplicate URLs across sections
• Noindex pages (left out of both files by default)
• Items whose content type is not ticked
• Canonical mismatches
• Password-protected pages (always left out)
• Thin content (< 100 words, no meta, no excerpt)
• File size > 50 KB
• Too many pages (> 80)

When Safety Mode is on, errors block generation and the results are shown immediately. Warnings are surfaced but don’t block. Everything is explained in plain English.

Scheduled regeneration

• Instant: regenerates ~30 seconds after any page is published or trashed (default)
• Daily: once per day, via WP-Cron recurring event
• Weekly: once per week
• Manual: only when you click Save & Generate

How to get started

1. Install and activate. The plugin auto-scans your content and creates initial sections.
2. Curate: drag, drop, add, remove. Aim for 20-60 pages that best represent your site.
3. Click Generate Missing Descriptions to fill gaps automatically.
4. Fix any warnings shown in the Safety Mode card.
5. Set the update mode that suits your workflow.
6. Save & Generate. Your files are live at /llms.txt and /llms-full.txt.
7. Enable the AI Crawler Log to see which bots start visiting.

WooCommerce integration

When WooCommerce is active, the plugin automatically includes SKU, price, and stock status in llms.txt descriptions, and full product details in llms-full.txt. Products with “hidden” visibility are excluded, and you can optionally exclude out-of-stock products.

Developer hooks

• llmscu_capability filter: override the required capability (default: manage_options)
• llmscu_post_limit filter: scanner post limit per type (default: 500)
• llmscu_full_word_limit filter: per-page word cap in llms-full.txt (default: unlimited)
• llmscu_full_max_bytes filter: size ceiling for llms-full.txt in bytes (default: 5 MB, minimum 256 KB)
• llmscu_max_sections / llmscu_max_entries filters: limits for saved or imported settings (defaults: 200 sections, 5,000 entries)
• llmscu_backup_dir filter: where replaced files are backed up (default: uploads/llms-txt-curator-backups; point it outside the web root if your host allows)
• llmscu_regenerated action: fires after each successful regeneration with the content string