ClaudeBot
Anthropic's training crawler for Claude. Allowing it lets your content inform future Claude models.
| Operator | Anthropic |
|---|---|
| Powers | Training data for Anthropic's Claude models |
| Purpose | Model training |
| User-agent token | ClaudeBot |
| Respects robots.txt | Yes |
ClaudeBot collects publicly available content used in training Anthropic's Claude family of models. It is the Claude equivalent of OpenAI's GPTBot.
ClaudeBot honors robots.txt directives. Disallowing the ClaudeBot user-agent removes your pages from Anthropic's crawl going forward.
Full user-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; [email protected])
Allow ClaudeBot
Helps Claude represent your domain and brand accurately from its training knowledge.
User-agent: ClaudeBot Allow: /
Block ClaudeBot
You want to keep your content out of Claude's training data.
User-agent: ClaudeBot Disallow: /
Can ClaudeBot read your page right now?
Test any URL and see exactly what AI crawlers receive.