OAI-SearchBot
OpenAI's search crawler. Controls whether your pages can surface and be cited in ChatGPT's search features.
Reference reviewed by Sona
| Operator | OpenAI |
|---|---|
| Powers | ChatGPT search results and citations |
| Purpose | Search & answers |
| User-agent token | OAI-SearchBot |
| Respects robots.txt | Yes |
OAI-SearchBot exists to build an index, not a training set. OpenAI's own description is that it is used to "surface websites in search results in ChatGPT's search features" - so this token, not GPTBot, is the one that decides whether ChatGPT can show your page as a linked source.
For most publishers this makes OAI-SearchBot the single most consequential OpenAI token to get right. Citations in ChatGPT are a referral channel with a live click attached; training exposure is not. The two are independently controlled, which means the widely recommended posture - disallow GPTBot, allow OAI-SearchBot - is a coherent policy rather than a contradiction.
The crawler respects robots.txt and reads its own group. Note the shape of its user-agent: it leads with a full Chrome-like browser string and appends the OAI-SearchBot/1.4 identifier at the end. Naive log filters that only look at the first token of the user-agent will misfile this traffic as ordinary desktop Chrome.
Being crawled is necessary but not sufficient for citation. OAI-SearchBot has to be able to reach the page, get a 200, and find the substance in the initial HTML response - so server-side rendered content, honest status codes, and no interstitial gates are what turn an allow rule into an actual citation.
How OAI-SearchBot behaves
- Sends a Chrome-prefixed user-agent with the OAI-SearchBot identifier appended at the end - easy to misclassify as human desktop traffic in log analysis.
- Crawls for index freshness, so expect repeat visits to pages that change rather than a single sweep.
- Reads only the OAI-SearchBot robots.txt group, independently of GPTBot's group.
- Content must be present in the server-rendered HTML to be reliably indexed for citation.
Full user-agent string
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Allow OAI-SearchBot
You want your pages eligible to appear as cited, clickable sources when ChatGPT answers a relevant query - the OpenAI surface that actually sends traffic.
User-agent: OAI-SearchBot Allow: /
Block OAI-SearchBot
You don't want your pages surfaced inside ChatGPT's answer interface at all, accepting the loss of referral clicks that come with citations.
User-agent: OAI-SearchBot Disallow: /
How to verify OAI-SearchBot
Published IP range feed
OpenAI publishes a dedicated feed at openai.com/searchbot.json - separate from the GPTBot ranges, so verify against this one specifically rather than assuming a shared OpenAI pool. Because the user-agent embeds a realistic Chrome string, spoofing is unusually easy and unusually easy to miss; the IP check is what separates real indexing traffic from scrapers borrowing the identity.
Check an IP against this botCommonly confused with OAI-SearchBot
Reverse of the common assumption: GPTBot governs training, OAI-SearchBot governs whether ChatGPT can cite and link you. Blocking the wrong one costs you traffic instead of protecting content.
OAI-SearchBot builds the index ahead of time; ChatGPT-User fetches a page live during one user's conversation. Both must be allowed for the full search-plus-browse path to work.
OAI-AdsBot reviews pages submitted as ChatGPT ads for safety. It has nothing to do with organic citation eligibility, which is OAI-SearchBot's job.
OAI-SearchBot FAQs
Which OpenAI bot decides whether ChatGPT can cite my page?
OAI-SearchBot. It builds the index behind ChatGPT's search features. GPTBot governs training only, so blocking GPTBot has no effect on your eligibility to be cited and linked.
Can I opt out of OpenAI training but still get cited in ChatGPT?
Yes, and it is a common policy. Disallow GPTBot and allow OAI-SearchBot. The two tokens are read independently, so the combination does exactly what it looks like.
Why does OAI-SearchBot look like Chrome in my logs?
Its user-agent begins with a full Chrome desktop string and appends OAI-SearchBot/1.4 at the end. If your log parsing only reads the leading token, this traffic gets bucketed as ordinary human visits.
I allow OAI-SearchBot but I am still not cited. Why?
An allow rule only grants access. The crawler still has to receive a 200 and find real content in the server-rendered HTML. Client-side-only rendering, consent walls, and soft 404s all defeat an otherwise correct robots.txt.
Can OAI-SearchBot read your page right now?
Test any URL and see exactly what AI crawlers receive.