Is your page blank without JavaScript? A 2-minute curl test

If the main text of your page is in the first HTML the server sends, a tool that does not run JavaScript can read it. If it is not, the page may look empty to that tool. You can test this yourself with curl, from your own computer, in two minutes.

Do AI crawlers run JavaScript?

The official documents do not answer this question, so no blanket "they do" or "they don't" can be written.

We read these pages on 20 October 2026. Not finding a sentence does not mean the bots do not run JavaScript; it only means the companies do not say.

There is one third-party source. Vercel measured traffic on its own network and, in a study published on 17 December 2024, wrote that the crawlers of OpenAI, Anthropic, Meta, ByteDance and Perplexity did not execute JavaScript. The same study says Google's Gemini infrastructure and Apple's crawler do. That is one company's measurement on its own network, and almost two years old; we could not confirm it still holds. So the rule stays simple: do not leave your main text to JavaScript.

How do you run the 2-minute curl test?

curl is a command-line tool that requests an address without a browser and prints the answer exactly as received. That is the same thing a tool that does not run JavaScript would see. The examples below use two made-up test pages called "Example Store": in one the text is in the HTML, in the other JavaScript writes it.

1. Pick a long, distinctive sentence on your page. For example: "Orders of 500 TL and over ship free."

2. Search for part of that sentence in the first HTML:

curl -s https://example.com/ | grep -c 'ship free'

grep -c prints the number of matching lines. 1 or more means the sentence is in the first HTML. 0 means it is not.

On the test page with the text in the HTML (we really ran it on a local server on 20 October 2026), the output was:

1

On the test page where JavaScript writes the text (same run), the output was:

0

3. Check the title and the main heading:

curl -s https://example.com/ | grep -o '<title>[^<]*</title>'
curl -s https://example.com/ | grep -o '<h1[^>]*>[^<]*</h1>'

The page with the text in HTML returned both lines. The JavaScript page returned the title (the title was in the HTML) but not the main heading.

4. Check the status code:

curl -s -o /dev/null -w '%{http_code}\n' https://example.com/

You expect 200. If you see 403 or 429, a security layer is cutting the request. That is a separate problem and has to be solved before the text matters (is a CDN such as Cloudflare blocking AI bots?).

Run these commands on your own site only.

How do you read the result?

A note: this test imitates a tool that does not run JavaScript. It does not imitate Google. Google runs JavaScript, but its own guide says server-side rendering is good for speed and for other bots.

Does pretending to be a bot help?

You can give curl a bot identity (a user agent). With the string OpenAI's documentation lists for OAI-SearchBot:

curl -s -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot' https://example.com/ | grep -c 'ship free'

This only tests rules that look at the identity. The identity is plain text and anyone can write it. Real bots come from documented IP addresses and a firewall can decide on those; a request from your own computer does not imitate that. To see real visits you need your server log: AI bots in your server log.

Can you also turn JavaScript off in the browser?

Yes, but it is a different test. In Chrome, with DevTools open, you can open the command menu and run "Disable JavaScript" (Chrome's page is from 2019, so menu names may have changed). If the page is blank after a reload, the text depends on JavaScript. The difference: the browser test looks at the page as shown; the curl test looks at the first answer the server gives. The second is closer to what a bot-like tool sees.

Which pages should you test?

Not all of them. In this order: the homepage, your best-earning product or service page, your three most important guides, the pricing page. Pick one sentence on each and search for it.

Ruescan is Rueseo's site scanner. In our measurement of 30 Turkish homepages, "missing" meant we did not find the item in the first HTML, so a schema or heading added later by JavaScript may not have shown up (details and limits of the measurement). The same limit applies to your own check.

Frequently asked questions