Description
AI Content Signals allows you to easily implement the Content Signals Policy in your WordPress site’s robots.txt file. This gives you more control over how AI crawlers and large language models (LLMs) can use your content.
What are Content Signals?
Content Signals is an extension to the robots.txt standard created by Cloudflare that lets you specify three types of permissions for AI crawlers:
- search – Allow or deny search indexing and traditional search results
- ai-input – Allow or deny using your content for real-time AI responses (RAG, AI Overviews)
- ai-train – Allow or deny using your content for training AI models
Key Features
- Easy-to-use settings page in WordPress admin
- Set global defaults for all crawlers
- Configure specific settings for individual AI bots (GPTBot, ClaudeBot, PerplexityBot, etc.)
- Add custom bot User-Agents
- Supports both physical and virtual robots.txt files
- Option to create physical robots.txt with basic WordPress rules
- Preview generated Content Signals before applying
- Optional legal text with EU Directive reference
- Works with existing robots.txt from SEO plugins
- Automatic sitemap detection and inclusion
Supported Bots
The plugin includes predefined settings for major AI crawlers:
- OpenAI GPTBot and ChatGPT-User
- Anthropic ClaudeBot and Claude-Web
- Perplexity Bot
- Google Extended (Bard/Gemini)
- Common Crawl Bot
- Meta/Facebook Bot
- And many more…
Important Notice
Content Signals is a declarative standard – it expresses your preferences but does not technically enforce them. AI companies are not legally required to respect these signals, though the plugin includes legal text referencing EU copyright directives.
This plugin works best when combined with other protection measures like traditional robots.txt rules and server-level bot management.
Screenshots
Main settings page with global Content Signals configuration 
Bot-specific settings for granular control 
Custom bots section for adding new AI crawlers 
Live preview of generated robots.txt content
Installation
- Upload the plugin files to
/wp-content/plugins/ai-content-signals/, or install through the WordPress plugins screen - Activate the plugin through the ‘Plugins’ screen in WordPress
- Go to Settings > AI Content Signals to configure your preferences
- Save settings to apply Content Signals to your robots.txt
FAQ
-
What happens if I have a physical robots.txt file?
-
The plugin will detect your physical robots.txt file and append Content Signals to the end of it. Make sure the file is writable (permissions 644 recommended).
-
Can the plugin create a robots.txt file if I don’t have one?
-
Yes! Since version 1.0.0, you can enable the option “Create physical robots.txt file if it doesn’t exist” in the settings. This will create a basic robots.txt with standard WordPress rules plus your Content Signals.
-
What rules are included in the automatically created robots.txt?
-
The plugin creates a basic robots.txt with these rules:
* Disallow /wp-admin/ (except admin-ajax.php)
* Disallow /wp-includes/
* Disallow /wp-content/plugins/
* Disallow /wp-content/themes/
* Disallow /wp-content/cache/
* Disallow readme.html and license.txt
* Automatically includes your WordPress sitemap if available -
What if I use an SEO plugin that manages robots.txt?
-
No problem! The plugin uses WordPress’s robots_txt filter, so it will add Content Signals to the virtual robots.txt generated by your SEO plugin without conflicts.
-
Will AI crawlers actually respect these signals?
-
Content Signals is a proposed standard and not all AI companies have committed to respecting it. It’s a way to express your preferences, but not a guarantee of enforcement. However, major AI companies like OpenAI, Anthropic, and Google have shown interest in respecting such signals.
-
Can I use this with Cloudflare?
-
Yes! If you use Cloudflare’s managed robots.txt, you can still use this plugin. The settings will apply to your WordPress-generated robots.txt.
-
What are the default settings?
-
By default, all signals are set to “yes” (allowed) for all user-agents. The option to create a physical robots.txt is disabled by default. You can change any of these settings in the settings page.
-
How do I remove Content Signals completely?
-
Simply uninstall the plugin. The uninstall process will remove all settings and clean up your physical robots.txt if applicable.
-
Do I need to manually edit my robots.txt file?
-
No! The plugin handles everything automatically. Just configure your preferences in the settings page and save.
Reviews
Contributors & Developers
“AI Content Signals” is open source software. The following people have contributed to this plugin.
Contributors
“AI Content Signals” has been translated into 1 locale. Thank you to the translators for their contributions.
Translate “AI Content Signals” into your language.
Interested in development?
Browse the code, check out the SVN repository, or subscribe to the development log by RSS.
Changelog
1.0.1
- Tested up to WordPress 6.9
1.0.0
- Initial release
- Global Content Signals settings
- Bot-specific configuration for 17 predefined AI crawlers
- Custom bot support
- Physical and virtual robots.txt handling
- Option to create physical robots.txt with basic WordPress rules
- Automatic sitemap detection and inclusion
- Legal text option with EU Directive reference
- Live preview of robots.txt output
- Status notifications for file permissions