GPTBot vs OAI-SearchBot: Should You Block Either?

"Should I block GPTBot?" is usually the wrong question wearing the right robots.txt syntax. OpenAI operates three main agents with three different jobs, and the one everyone names — GPTBot — is the one whose blocking has the least visible consequence. This page separates the three decisions, quotes what OpenAI actually documents about each, and shows the choices we made across our own production builds and why.

What is the difference between GPTBot, OAI-SearchBot, and ChatGPT-User?

They are three independently documented agents: a training crawler, a search-index crawler, and a user-triggered fetcher. OpenAI's bot documentation defines each with its own robots.txt token and its own purpose, and describes them as operating independently.

Per OpenAI's crawler documentation, current as of August 2026.

The core sentences from that documentation are worth reading verbatim, because they settle most forum arguments. On GPTBot: disallowing it "indicates a site's content should not be used in training generative AI foundation models." On OAI-SearchBot: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers."

Does blocking GPTBot hurt your visibility in ChatGPT?

Per OpenAI's documentation, no — GPTBot and search operate separately, and ChatGPT search visibility rides on OAI-SearchBot. That is the platform-documented answer, and it is the one to act on.

There is a subtler, unproven layer worth naming honestly. Kevin Indig's State of AI Search Optimization 2026 found that 24% of ChatGPT answers involve no web fetch: the model answers from training memory. What a model learned about your brand or topic during training can surface in those answers, and training-data presence is the only channel into them. Whether allowing GPTBot measurably improves how model-memory answers treat you is not established by any study on our source list — treat it as plausible but unproven, and weight it accordingly. It argues for caution before blocking, not for expecting anything from allowing.

What blocking GPTBot definitely does not do is retroactive: it cannot remove content a model already trained on. The robots.txt line changes what future crawls collect, nothing more.

What happens if you block OAI-SearchBot?

You disappear from ChatGPT search answers — this is the one OpenAI decision with an explicit, documented visibility cost. The retrieval pipeline behind ChatGPT search draws on the index OAI-SearchBot builds; opting out of the crawler is opting out of the index, and OpenAI's docs state the consequence without hedging.

For most sites that want AI-referred visitors, blocking OAI-SearchBot while leaving GPTBot open is exactly backwards, yet we see it in audits: an operator pastes a block-all-AI list from a blog post, then wonders why ChatGPT never cites them. How retrieval and citation actually connect is laid out in how ChatGPT selects cited sources, and the index-building role of each company's search crawler is in the complete AI crawler list.

Can you block ChatGPT-User at all?

Not with robots.txt alone — OpenAI states that because these fetches are user-initiated, "robots.txt rules may not apply." ChatGPT-User retrieves a page when a person asks ChatGPT to visit or act on it, and OpenAI treats the human request as the authorizing event.

Reliable blocking therefore means infrastructure, not directives: server rules or a WAF matched against the IP ranges OpenAI publishes in its bot documentation. Before building that, ask what the block buys you. A user-triggered fetch is a human actively interested in your page — for most business models that is the traffic, not the threat. The cases with a real argument (paywalled content, scraping-sensitive data) are covered in the robots.txt decision framework.

What do we allow on our own sites?

All OpenAI agents, on all three production builds — allowed deliberately, reviewed periodically [our data]. Our businesses are authority and lead-generation sites: value arrives when an engine can retrieve, cite, and send a visitor, so the search crawler and the user fetcher are working for us, and we accept training use as the price of the ecosystem we want visibility in.

We hold that position knowing the extraction math. Cloudflare's 2025 analysis put OpenAI's crawl-to-refer ratio at roughly 1,091 pages fetched per referral sent in July 2025 — down 10.4% from January but still steep. Crawling is not reciprocity; we allow it because retrieval candidacy is what our model monetizes, not because the ratio flatters anyone.

Argue it against our interest: if your revenue is pageviews or licensed content, the same math points the other way, and blocking GPTBot while keeping OAI-SearchBot open is a coherent, defensible robots.txt — you withhold the training subsidy without leaving ChatGPT search. The per-business-model table is in the robots.txt decision framework.

How do you write the actual robots.txt rules?

Name each token in its own group and make each line a decision you can defend. The three coherent postures:

# Allow everything (our posture — authority/lead-gen)
# No OpenAI-specific rules needed; absence of a Disallow is consent.

# Block training, keep search (publisher posture)
User-agent: GPTBot
Disallow: /

# Block all OpenAI crawling (walled-garden posture)
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow: /

Remember what the file is: a published request that compliant crawlers honor, not an access control. OpenAI documents GPTBot and OAI-SearchBot as respecting it, while ChatGPT-User may fetch regardless — and enforcement beyond that lives at the CDN or WAF layer. The wider practice of structuring a site engines can retrieve and quote is the subject of our generative engine optimization guide.