OAI-SearchBot

OAI-SearchBot is OpenAI's separate crawler that indexes web content specifically for ChatGPT Search, building the retrieval pool that feature cites from, independent of the GPTBot crawler used for model-training data collection. Blocking OAI-SearchBot removes a page from ChatGPT Search's citation pool without affecting training opt-outs.

Why it matters

For any brand actively pursuing ChatGPT Search visibility, confirming OAI-SearchBot has unblocked access is a more direct and consequential check than confirming GPTBot access, since OAI-SearchBot is the crawler that actually feeds citations rather than training data.

OpenAI's crawler documentation states that sites which opt out of OAI-SearchBot will not be shown in ChatGPT search answers, although they can still appear as navigational links, and that robots.txt changes take about 24 hours to apply. ChatGPT-User, which fetches pages for user requests, is not used to decide what appears in search.

Its activity is growing fast: in a Botify analysis of seven billion log events, OAI-SearchBot requests rose 3.5 times in the eight months after GPT-5 launched in August 2025. Rate-limiting or blocking it to save server load can cost ChatGPT citations, as our article on AI crawler rate limits explains.

Related reading

AI crawler traffic tripled after GPT-5. Rate-limiting the wrong bot can quietly cost you citations

Related terms

GPTBot

GPTBot is OpenAI's crawler for collecting web content used to train its models, operating separately from the crawlers OpenAI uses to power ChatGPT Search citations. A site can block GPTBot specifically to opt out of having its content used for model training, without affecting whether ChatGPT Search can still retrieve and cite that same content.

ChatGPT Search

ChatGPT Search is OpenAI's web-grounded answer mode inside ChatGPT, which retrieves current web pages before generating a response and displays a source panel alongside the answer. It launched as a standalone feature in late 2024 and uses a separate crawler, OAI-SearchBot, to build its citation index apart from GPTBot's model-training crawl.

ChatGPT-User

ChatGPT-User is the user-agent OpenAI applies when ChatGPT fetches a specific page live, in real time, at a user's explicit request within a conversation, distinct from the background indexing done by GPTBot or OAI-SearchBot. It represents an on-demand fetch rather than a systematic crawl.

robots.txt (AI directives)

robots.txt is the long-standing standard crawler-control file at a site's root, now extended with AI-specific user-agent rules that separately allow or disallow crawlers like GPTBot, ClaudeBot, and PerplexityBot. A single robots.txt file can permit standard search crawling while blocking one or more AI crawlers, or the reverse.

Optimize your presence across Generative AI and Answer Engines.

Track your AI visibility · Get a demo