GPTBot, ClaudeBot, and PerplexityBot do not render JavaScript. They request a URL, read whatever HTML comes back in that first response, and move on. No browser engine, no script execution, no waiting around for your React app to hydrate. If your homepage ships an empty div and fills it in with JavaScript after the page loads, every one of those crawlers sees the empty div. Your actual content, the text a human visitor reads a half second later, simply is not there as far as the bot is concerned.
This single fact explains a surprising number of "why doesn't ChatGPT know about us" conversations I have with clients. Their SEO looks fine. Googlebot has indexed the site for years. But ask ChatGPT about the brand and you get a shrug, or worse, outdated or wrong information pulled from a directory listing instead of the actual site. The content is there. A crawler that cannot run JavaScript just never saw it.
What "rendering" actually means
When your browser loads a page, two separate things happen. First, it downloads the raw HTML document your server sends back. Second, if that HTML includes script tags pointing to a JavaScript bundle, the browser downloads and runs that bundle, which can then write new content into the page, fetch data from an API, and build out everything the user eventually sees. That second step is rendering in the client side sense, and it can add, replace, or restructure most of what is visible on the page.
A crawler that only performs step one gets a snapshot of the page as it existed before any JavaScript executed. For a traditional server rendered site built with plain HTML templates, that snapshot already contains the full content, so nothing is lost. For a single page application built with React, Vue, Angular, or a similar framework using client side rendering, that snapshot can be close to empty. A single div with an id, a few script tags, and nothing else.
Google solved part of this problem for its own search index years ago. Googlebot runs a two wave indexing process: it crawls the raw HTML first, queues the page, then comes back later with a headless Chromium instance to render the JavaScript and index what it finds. That second wave can take anywhere from a few seconds to several days depending on Google's rendering queue. AI crawlers have no equivalent second wave. There is one fetch, and it is not a rendering fetch.
Googlebot eventually renders JavaScript through a delayed second indexing pass. GPTBot, ClaudeBot, and PerplexityBot do not have a second pass. Whatever your server sends on the first request is the entire opportunity these crawlers get to read your content.
What each major AI crawler actually does
Not every AI crawler behaves identically, and the differences matter when you are deciding where to spend engineering time. Here is what is documented or widely observed about the crawlers that matter most for AI visibility. For a deeper breakdown of each bot's purpose, user agent string, and robots.txt directives, see our guide to GPTBot, ClaudeBot, and PerplexityBot.
| Crawler | Operator | Renders JavaScript | Used for |
|---|---|---|---|
| GPTBot | OpenAI | No | Training data collection for future models |
| ClaudeBot | Anthropic | No | Training data collection |
| PerplexityBot | Perplexity | No | Real time answer generation and citation |
| Google-Extended | No (separate from Googlebot's rendering pass) | Gemini and AI Overviews training data | |
| Googlebot | Yes, on a delayed second wave | Core Search index, which can feed AI Overviews | |
| Bingbot | Microsoft | Yes, on supported pages | Bing Search index, which feeds Copilot |
Notice the pattern. The crawlers built specifically to gather content for large language model training or for live retrieval augmented generation (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) skip rendering entirely. The crawlers built for traditional search indexing (Googlebot, Bingbot) have invested in rendering infrastructure because their entire product depends on indexing the modern, JavaScript heavy web. AI crawlers were built faster, leaner, and later, and nobody has added a rendering layer to them yet. That could change, but there is no public roadmap suggesting it will soon.
This creates an odd asymmetry. A page that only renders through client side JavaScript might still show up correctly in Google AI Overviews, because Google's own rendering pipeline can feed that index. The same page can be functionally invisible to ChatGPT and Claude, because the crawlers behind those products never see past the empty shell.
How to check what a crawler actually sees on your site
You do not need an enterprise SEO platform to answer this question. Two simple checks will tell you almost everything.
Disable JavaScript in your browser
Open your browser's developer tools, find the setting to disable JavaScript, and reload your most important page. What remains on screen is close to what a non rendering crawler receives. If your headline, body content, and key claims are still visible, you are in reasonable shape. If you are looking at a blank page or a loading spinner frozen in place, that is exactly what GPTBot and ClaudeBot see too.
Fetch the raw HTML directly
Run a plain HTTP request against your URL and read the response body without letting anything execute it. A simple command line fetch works:
curl -A "GPTBot" https://yoursite.com/your-page
Read through the output. Search for the sentences and headings you expect to find. If your main content is missing from that raw response and only a container div or a handful of script tags remain, your content is not reaching AI crawlers no matter how good it reads in a browser.
Use the Rich Results Test as a secondary check
Google's Rich Results Test tool renders your page and shows you the rendered HTML it generated, which is useful for understanding what a rendering crawler eventually sees. But remember that result represents Googlebot's rendered view, not GPTBot's raw view. Use it to confirm your rendering works at all, then go back to the curl or JavaScript disabled test to understand the non rendering case.
Two tests, five minutes total: load your page with JavaScript off, then curl the raw URL and read the response. Whatever text appears in both is safely visible to every AI crawler. Whatever only appears with JavaScript enabled is invisible to GPTBot, ClaudeBot, and PerplexityBot today.
Why this matters more than it used to
Ten years ago, whether a crawler rendered JavaScript was mostly an SEO nuance that affected indexing speed. Today it decides whether your brand exists inside the training data and retrieval systems that power how millions of people get answers. When someone asks ChatGPT to recommend a vendor in your category, the model is drawing on what it learned during training and, increasingly, on what retrieval systems fetch live from the web. A brand that is invisible to GPTBot and PerplexityBot is simply not part of either process. Not ranked lower. Not deprioritized. Absent.
This is a different failure mode than a typical SEO problem. A page that ranks on page three of Google is still indexed, still findable, still improvable with better content or links. A page that a non rendering crawler cannot read at all is not in the pool of candidates to begin with. There is no ranking signal to improve, because there is no content for the crawler to evaluate.
The sites most at risk are the ones built as single page applications with client side routing and client side data fetching: marketing sites rebuilt on a modern JavaScript framework for the development experience, with content hydrated entirely in the browser. If your developer stack prioritizes interactivity and your content team prioritizes words on the page, this gap often opens without anyone noticing, because the site looks completely normal to every human who visits it.
The fixes, ranked by how durable they are
Server side rendering (most durable)
Server side rendering generates the full HTML for a page on your server before it ever reaches the crawler or the browser. Frameworks like Next.js, Nuxt, SvelteKit, and Remix support this natively. The crawler gets complete content on the first request, the same request every other visitor makes. There is no special casing, no second code path, and no risk of the rendered version drifting from what a real visitor sees, because they are the same response.
If you are building a new site or planning a rebuild, server side rendering should be the default assumption, not an optimization bolted on later. Our guide to AI agent website readiness covers the broader set of technical decisions that affect whether AI systems, crawlers included, and increasingly AI browsing agents acting on a user's behalf, can actually use your site.
Static site generation (durable, with caveats)
Static site generation pre builds every page into plain HTML files at deploy time. This produces the same crawler friendly result as server side rendering for content that does not change per request, which covers most marketing pages, blog posts, and service pages. The caveat is content that is personalized or frequently updated, which does not fit a build time generation model well. For the bulk of a marketing site, static generation is often the simplest path to full crawler visibility.
Bot-specific prerendering (stopgap, not a fix)
Bot-specific prerendering detects known bot user agents and serves them a separately generated, fully rendered HTML snapshot, while regular browsers still get the standard client side rendered application. This can solve the immediate visibility problem without a framework migration, but it introduces a second rendering pipeline you now have to maintain and keep in sync. Any drift between what the snapshot shows and what your live application actually contains becomes a cloaking risk, even if unintentional. Treat bot-specific prerendering as a bridge while you plan a move to server side rendering, not as a permanent architecture.
Hybrid rendering (practical middle ground)
Many modern frameworks let you choose rendering strategy per route. Render your marketing pages, blog, and service pages on the server or at build time, where crawler visibility matters most, while keeping client side rendering for authenticated dashboards or interactive tools where no crawler needs to read the content anyway. This targets the fix at the pages where AI citation actually matters instead of rewriting an entire application.
What this does not fix
Rendering is only one layer of the problem. A fully server rendered page with excellent content can still be invisible to AI crawlers for other reasons entirely. Your robots.txt might be blocking GPTBot or ClaudeBot outright, which no amount of rendering will overcome. Your content might lack the schema markup that helps a model understand what it is reading and who is responsible for it, a gap we cover in depth in our guide to schema markup for AEO. And rendering says nothing about whether your content is actually worth citing once a crawler can read it. Fixing rendering gets your content in front of the crawler. It does not make the content good.
Think of rendering as the floor, not the strategy. It determines whether you are even in the conversation. Everything else, from entity signals to content depth to structured data, determines whether you win it.
A quick audit you can run this week
- Pick your five most important pages: homepage, your core service or product pages, and your highest traffic blog post.
- For each one, disable JavaScript and reload. Note whether your primary content, headline, and body text survive.
- For each one, run
curl -A "ClaudeBot" https://yoursite.com/pageand search the output for a sentence you know is on the page. - Check your robots.txt for explicit disallow rules against GPTBot, ClaudeBot, PerplexityBot, or Google-Extended. A rendering fix does nothing if the crawler is blocked before it even tries to fetch the page.
- If content is missing in the no JavaScript test, flag the page for a server side rendering or static generation fix, prioritized by how much AI citation traffic that page's topic could realistically earn.
Most sites that fail this audit fail on two or three pages, not the whole site. A marketing site built on a modern framework but statically generated at build time will usually pass cleanly. The failures concentrate on pages built as true single page applications with heavy client side data fetching, which is often the homepage or a product catalog page, exactly the pages you most need an AI crawler to read.
Why this gap exists in the first place
It helps to understand why GPTBot, ClaudeBot, and PerplexityBot were built this way rather than assuming it is an oversight someone will patch soon. Rendering JavaScript at crawl scale is expensive. Googlebot's rendering pipeline runs on infrastructure built over more than a decade, tuned specifically for the scale and diversity of the modern web, and it still introduces a delay measured in hours or days between the first crawl and the rendered index update. Running a browser engine for every page fetch multiplies compute cost by a wide margin compared to a plain HTTP request.
For a company training a language model, the priority is breadth and freshness of raw text, not pixel perfect rendering of every site's interactive layer. A crawler built to collect training data at the scale of billions of pages is optimized for speed and simplicity. Adding a full rendering engine to that pipeline is a real infrastructure investment, and so far, none of the major AI labs have made it standard for their primary crawlers. For retrieval augmented generation crawlers like PerplexityBot, which fetch pages in near real time to answer live user queries, rendering would add latency directly into the user facing response time, which works against the product's core promise of fast answers.
None of this means the situation is permanent. Crawler behavior has changed before, and it will likely change again as AI companies compete harder on answer quality and source coverage. But planning your site architecture around a rendering capability that does not exist yet is a bet against your own visibility today. Build for the crawlers as they behave now, and treat any future rendering support as a bonus, not a dependency.
The business cost of an invisible site
It is worth being concrete about what is actually lost when a site is invisible to non rendering crawlers, because the abstract version of this problem is easy to shrug off. Picture a potential customer who does not search Google first anymore. They open ChatGPT and ask which company in your category they should use, or they ask Perplexity to compare the top three options and explain the tradeoffs. If your site never made it into the training data or the retrieval index behind that answer, you are not one of the options being weighed. You are not even a footnote. The buyer makes a decision using a shortlist that was assembled without you in it.
This matters most in categories where buyers increasingly skip the traditional search and compare step entirely and go straight to an AI assistant for a recommendation. The businesses that show up in that first AI generated shortlist get a meaningful head start, because the buyer's mental model of "who exists in this space" is now anchored by whatever the model named first. A business that is technically excellent, well reviewed, and well positioned but invisible to GPTBot and PerplexityBot loses that shortlist position to a competitor whose website happens to render without JavaScript, even if that competitor's actual offering is weaker.
The frustrating part is that this is rarely a deliberate choice. Nobody sits down and decides to block AI crawlers from reading their content. It happens as a side effect of a framework decision made for reasons that had nothing to do with AI visibility: faster perceived page transitions, a preferred developer experience, a component library that assumes client side rendering by default. The business cost shows up months or years later, disconnected from the original technical decision, which is exactly why it goes unnoticed for so long.
Framework specific notes
The right fix depends heavily on what you are already running. A few patterns come up often enough to call out directly.
React and Create React App
A plain Create React App deployment is close to the worst case for AI crawler visibility. The initial HTML is a near empty shell with a single root div, and everything else is injected by JavaScript after the bundle loads. If your marketing site runs on an unmodified Create React App build, treat this as a priority fix. Migrating to Next.js, which supports server side rendering and static generation on top of the same React component model, is usually the most direct path, since it preserves your existing components while fixing the delivery mechanism.
Vue and Nuxt
Plain Vue single page applications face the same issue as plain React. Nuxt, the server side rendering and static generation framework built for Vue, solves it the same way Next.js solves it for React. If your team already knows Vue, moving your marketing pages to Nuxt's static generation mode is usually a contained project rather than a full rewrite.
Angular
Angular Universal provides server side rendering for Angular applications and has existed for years, so if your site runs on Angular and still ships a client side only bundle, the fix may be enabling a capability that is already available in your framework rather than adopting a new one.
WordPress, Webflow, and traditional CMS platforms
If your site runs on WordPress, Webflow, or a similar CMS without a heavy client side JavaScript layer, you are very likely already fine. These platforms generate complete HTML on the server by default. The rendering problem described in this article mainly affects sites built as JavaScript driven single page applications. Worth verifying with the no JavaScript test regardless, since page builders and certain plugins occasionally inject critical content through client side scripts in ways that are easy to miss.
What to tell your development team
If you are the marketing or AEO lead raising this issue with engineering, frame the ask specifically. "Make the site AI friendly" is too vague to act on. Instead, ask for this: confirm that the primary content of our top pages, meaning headlines, body text, and key claims, is present in the raw HTML response before any JavaScript executes. If it is not, scope the cost of moving those pages to server side rendering or static generation, and give me a before and after comparison of the raw HTML so I can verify the fix once it ships.
That framing turns a vague concern into a testable requirement, and it gives your developers a clear definition of done. It also means you do not have to take anyone's word for it. Run the curl test yourself before and after the change, and you will see the difference directly in the response body.
Expect some pushback if the fix requires a framework level change rather than a quick patch. A move from client side rendering to server side rendering touches build tooling, deployment pipelines, and sometimes the hosting provider itself, so it is a legitimate project, not a configuration tweak. Ask for a rough estimate of effort alongside the technical confirmation, and weigh that against the queries you stand to gain citations for. For a brand competing in a category where buyers are already asking AI assistants for recommendations, the math usually favors fixing it sooner rather than later.
What good looks like once it is fixed
After a rendering fix ships, the test is the same one you ran before: disable JavaScript, reload the page, and read what is there. A page that passes this test cleanly will show the full headline, the complete body copy, every internal link, and any FAQ content exactly as a logged out visitor would see it, with no placeholder text, no loading state frozen mid render, and no missing sections. Run the curl command again with a GPTBot or ClaudeBot user agent and confirm the response matches. If both checks come back clean, you have closed the gap this article is about, and any remaining visibility problem belongs to a different layer, whether that is crawler access in robots.txt, missing schema, or the strength of the content itself.
How we approach this for clients
When we run a technical AEO audit, checking crawler rendering is one of the first items on the list, right alongside robots.txt access and schema validation. We pull the raw HTML response the way GPTBot would see it, compare it against the fully rendered page a visitor sees, and flag any gap between the two. Where a framework migration is not realistic in the short term, we scope a bot-specific prerendering layer as an interim fix while planning the longer term move to server side rendering or static generation. The goal is always the same: make sure the content you have already written is the content AI crawlers actually receive.
This kind of audit usually surfaces one of two outcomes. Either the site is already rendering fine and the visibility problem lives somewhere else, in robots.txt rules, missing schema, or weak entity signals, or the site has a rendering gap that nobody noticed because every human visitor sees a perfectly normal page. Both outcomes are good news in their own way. A rendering gap is one of the more mechanical problems to fix, because it does not require rewriting your content or rebuilding your entity presence. It requires making sure the words you already wrote actually reach the crawler reading your site.