Meta-ExternalAgent
Meta's crawler for training Meta AI and the Llama models. Honors robots.txt rules for its token.
| Operator | Meta |
|---|---|
| Powers | Meta AI and Llama model training |
| Purpose | Model training |
| User-agent token | meta-externalagent |
| Respects robots.txt | Yes |
Meta-ExternalAgent is the user-agent Meta uses to collect web data for training its AI products, including the Llama model family and Meta AI assistant features across its apps.
Use the meta-externalagent token in robots.txt to allow or disallow this crawling.
Allow Meta-ExternalAgent
Your content can inform Meta AI and Llama, which reach a very large user base.
User-agent: meta-externalagent Allow: /
Block Meta-ExternalAgent
Keep your content out of Meta's AI training pipeline.
User-agent: meta-externalagent Disallow: /
Can Meta-ExternalAgent read your page right now?
Test any URL and see exactly what AI crawlers receive.