Meta-ExternalFetcher
Meta's user-triggered fetcher - retrieves a specific page when a Meta AI user asks about it. Sibling to the Meta-ExternalAgent training crawler.
Reference reviewed by Sona
| Operator | Meta |
|---|---|
| Powers | Meta AI user-triggered link fetches |
| Purpose | User-triggered fetch |
| User-agent token | meta-externalfetcher |
| Respects robots.txt | Mostly |
Meta-ExternalFetcher fires when a user of Meta AI (across Facebook, Instagram, WhatsApp, and meta.ai) triggers a fetch of a specific link. It is the user-triggered sibling of Meta-ExternalAgent, which handles bulk training crawls.
Because these fetches are performed on behalf of a user request, Meta documents that Meta-ExternalFetcher may bypass robots.txt rules. To enforce a block, filter the user-agent at the server or firewall level.
Full user-agent string
meta-externalfetcher/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)
Allow Meta-ExternalFetcher
Users asking Meta AI about your page get answers grounded in your actual content.
User-agent: meta-externalfetcher Allow: /
Block Meta-ExternalFetcher
You want to prevent all programmatic access from Meta AI, including user-initiated fetches.
User-agent: meta-externalfetcher Disallow: /
Can Meta-ExternalFetcher read your page right now?
Test any URL and see exactly what AI crawlers receive.