AI Robots.txt Checker

Check if popular AI crawlers and user agents are allowed to access your website according to your robots.txt file.

Website Domain

AI User Agents We Check

This tool checks if the following AI crawlers and user agents are allowed to access your website:

GPTBot - OpenAI (ChatGPT)

ChatGPT-User - ChatGPT browsing

Google-Extended - Google Bard/Gemini

ClaudeBot - Anthropic Claude

PerplexityBot - Perplexity AI

CCBot - Common Crawl

anthropic-ai - Anthropic

cohere-ai - Cohere

💡 Why this matters: Blocking AI crawlers can reduce your visibility in AI-powered search engines like ChatGPT, Perplexity, and Gemini. Consider allowing these bots if you want better AI visibility.