Claude-Web / anthropic-ai
Anthropic's user-facing and legacy tokens. Add these alongside ClaudeBot for complete Anthropic coverage.
| Operator | Anthropic |
|---|---|
| Powers | Live fetches and legacy Anthropic crawling |
| Purpose | User-triggered fetch |
| User-agent token | Claude-Web |
| Respects robots.txt | Yes |
Claude-Web is associated with live retrieval when Claude accesses a page in response to a user, and anthropic-ai is an older token Anthropic has used. Listing both ensures rules apply across Anthropic's current and historical agents.
If your goal is to control everything Anthropic, set rules for ClaudeBot, Claude-Web, and anthropic-ai together — they cover training, live access, and legacy crawling respectively.
Allow Claude-Web / anthropic-ai
Lets Claude fetch your page when a user asks about it directly.
User-agent: Claude-Web Allow: /
Block Claude-Web / anthropic-ai
Blocks Anthropic's non-training access in addition to ClaudeBot.
User-agent: Claude-Web Disallow: /
Can Claude-Web / anthropic-ai read your page right now?
Test any URL and see exactly what AI crawlers receive.