AI bots in your server logs: real OAI-SearchBot or a fake?

"Did OAI-SearchBot visit my site?" is answered in your server log. But the name in the log is not enough on its own. This guide shows how to check, with an IP list, whether a line really belongs to the bot.

Why the name is not enough

A bot identifies itself with the User-Agent header. That header is plain text, and any program can send any name. So seeing OAI-SearchBot in a log does not show the visit came from OpenAI. We do not base this on a sentence in the docs; it is how HTTP works. It is also why the companies publish IP lists.

What each company publishes

OpenAI lists its bots and an IP list for each in its docs (read 16 October 2026):

For OAI-SearchBot, OpenAI recommends "allowing requests from our published IP ranges". The page describes no other verification method (such as reverse DNS), so we use only the IP list.

The others:

• Perplexity: separate lists for PerplexityBot and Perplexity-User (perplexity.com/perplexitybot.json, perplexity.com/perplexity-user.json). Its docs advise combining user-agent matching with IP verification.
• Anthropic: IP addresses for ClaudeBot, Claude-User and Claude-SearchBot are at claude.com/crawling/bots.json. Anthropic does not recommend blocking by IP: it says this may not give a lasting opt-out and can stop the bots from reading your robots.txt. Use the list to recognise visits in your log, not to block.
• Google: for Googlebot there are two routes: a reverse DNS lookup followed by a forward lookup (the domain must be googlebot.com, google.com or googleusercontent.com), or IP range files such as common-crawlers.json. This guide focuses on AI bots, so we skip the detail.

Step by step

1. Pull the lines. Search your access log for the bot name: grep OAI-SearchBot access.log.
2. Download the IP list. openai.com/searchbot.json holds a prefixes array; each entry has an ipv4Prefix (for example 13.66.216.176/28). The file also has a creationTime field; when we read it on 16 October 2026 it said 2026-01-02. Lists can change, so download them regularly.
3. Compare each IP with the ranges. If the IP in the line's first field falls inside one of the ranges, it is verified.
4. Read the result. Lines outside the list are "unverified". That is not enough proof of a fake (your copy of the list may be old, or a proxy may have forwarded the request), but do not count them as OpenAI either.

A short script

This Python script uses only the standard library and scans a whole log at once. We ran it on a sample log and a sample list; check the output on your own log as well. It assumes the IP is the first field (the standard Apache and Nginx format). Behind a CDN, check which field holds the real client IP.

import ipaddress, json, sys
from collections import Counter

def yukle(yol):
    veri = json.load(open(yol))
    return [ipaddress.ip_network(p.get("ipv4Prefix") or p.get("ipv6Prefix"))
            for p in veri["prefixes"]]

ag = yukle(sys.argv[1])   # indirdiginiz IP listesi
bot = sys.argv[2]         # ornek: OAI-SearchBot
sayac = Counter()
for satir in open(sys.argv[3]):   # erisim gunlugu
    if bot not in satir:
        continue
    ip = ipaddress.ip_address(satir.split()[0])
    sayac["dogrulandi" if any(ip in n for n in ag) else "dogrulanamadi"] += 1
print(dict(sayac))

Run it with python3 dogrula.py searchbot.json OAI-SearchBot access.log. It prints how many lines were verified and how many were not.

Limits of this method

• A visit is not a citation. If OAI-SearchBot crawls your page, that does not mean ChatGPT will show you in an answer. OpenAI's docs say the bot honors robots.txt for search and that sites that opt out are left out of search answers; they do not say crawling guarantees a mention.
• ChatGPT-User is different. OpenAI says that because these actions are user-initiated, robots.txt rules may not apply. See our robots.txt guide.
• No log, no answer. On shared hosting or some CDN plans you may not get raw access logs. Then you cannot tell which bot visited; accept that you cannot verify it.
• We give no frequency figures. We found no official value for how often these bots visit.

What Ruescan shows, and what the log shows

Ruescan shows whether your robots.txt allows AI bots. That is a permission check. Only your server log tells you whether a bot actually came. Read them together: permission plus verified visits in the log means all is well; permission but no visits means you should look for the cause (a CDN block, IPs missing from the list, missing logs).

FAQ

Did OAI-SearchBot visit my site, and how do I tell?

Find the lines containing OAI-SearchBot in your access log, then compare their IPs with searchbot.json. Matches came from OpenAI.

What if the user agent is right but the IP is not in the list?

Treat it as unverified. Re-download the list first; if the IP is still missing, do not treat the request as OpenAI's bot. Whether to block it is your call.

Does this work for GPTBot too?

Yes, same logic: use gptbot.json. For ChatGPT-User, use chatgpt-user.json. Download the list and compare the log line's IP with its prefixes. ChatGPT-User is a special case: OpenAI says robots.txt rules may not apply to actions a user starts. An IP that is not in the list stays unverified.

Why doesn't Anthropic recommend blocking by IP?

Anthropic says an IP block may not give a lasting opt-out and can stop the bots reading your robots.txt. Use robots.txt to opt out.

How often do the IP lists change?

Not verified. The creationTime field shows when a list was last generated; we found no documentation on how often it changes. Have your script re-download the list on every run.