Google-CloudVertexBot
Google Cloud's crawler for Vertex AI - fetches site content when Vertex AI customers build agents that ingest it.
Reference reviewed by Sona
| Operator | |
|---|---|
| Powers | Vertex AI Agents site ingestion |
| Purpose | User-triggered fetch |
| User-agent token | Google-CloudVertexBot |
| Respects robots.txt | Yes |
Google-CloudVertexBot crawls sites at the request of Google Cloud customers building Vertex AI Agents and search applications. It is triggered by customer configuration rather than by Google's own indexing pipeline.
The bot is documented among Google's crawlers and respects robots.txt rules set for its token; blocking it does not affect Google Search or Googlebot.
Full user-agent string
Mozilla/5.0 (compatible; Google-CloudVertexBot/1.0; +https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers)
Allow Google-CloudVertexBot
Businesses building AI agents on Google Cloud can ground those agents in your content - relevant if partners or customers build on Vertex.
User-agent: Google-CloudVertexBot Allow: /
Block Google-CloudVertexBot
You don't want third-party Vertex AI applications ingesting your site.
User-agent: Google-CloudVertexBot Disallow: /
Can Google-CloudVertexBot read your page right now?
Test any URL and see exactly what AI crawlers receive.