Technical SEO / Rendering / AI search
Can Google and AI crawlers actually see your JavaScript?
Your page looks perfect in Chrome. That tells you almost nothing. Three different pathways reach your URL, they render differently, and only one of them is Googlebot. Here is how to find the gap, prove it, and hand a developer something they can fix.
By Manikandan Nagappan · Enterprise SEO, AEO & GEO
Every few months someone tells me their site is fine because it is server-rendered. Then we open the response and the product description is not in it. The framework label is not the evidence. The response body is. This guide is the sequence I use to find out which parts of a page survive the trip, and which parts only exist in a browser.
What is JavaScript SEO, and what is it really asking?
JavaScript SEO is the practice of making sure the content, links and indexing signals that matter are available to the systems that need them, rather than only to a browser running your application code. The work is mostly diagnosis: compare what your server returns with what the browser builds, and decide which differences actually cost you something.
The working method
Don’t ask whether crawlers can run JavaScript. Ask which pathway reached this page, what that pathway received, and whether anything you care about was missing from it.
That reframing matters because the honest answer to “can they run JavaScript?” is now it depends which one, on which pass. Google’s search crawler renders. Several AI systems reach your page twice — once through an index that rendered, once through a direct fetch that did not — and give you different answers depending on which path supplied the content.
Three artefacts
The raw HTTP response, the rendered DOM, and what a named consumer actually came back with.
Render Gap Ladder
Six tests, cheapest first. Each one answers a different question, and none of them substitutes for another.
A ticket, not an opinion
Affected template, the exact difference, the evidence, and acceptance criteria an engineer can retest.
Which lane actually reached your page?
Crawlers get lumped together as “bots”, which hides the only distinction that changes your decisions. Operators run separate agents for separate jobs, and the jobs render differently. OpenAI, Anthropic and Perplexity each document a three-way split, and Cloudflare’s bot reference categorises them the same way: a training crawler, a search-index crawler, and an assistant fetching a page because a person asked for it.
The SEOAnalyst.me model
One response. Three lanes. Different renderers.
Why it matters: the same assistant can describe your page accurately from its index and return nothing from a pasted link. Both observations are real. They are measuring different lanes.
| Consumer | Lane | What to validate |
|---|---|---|
| Googlebot + Web Rendering Service | Index | Rendered HTML in URL Inspection, loaded resources, indexing state. Rendering is documented. |
| Bingbot | Index, and grounding for Copilot | Initial HTML, Bing URL inspection, index status, AI Performance citations. |
| OAI-SearchBot · Claude-SearchBot · PerplexityBot | Index for AI answers | Raw response completeness, robots rules, verified requests in logs. Don’t assume rendering. |
| ChatGPT-User · Claude-User · Perplexity-User | Live fetch, user-triggered | What a pasted URL actually returns. Test it directly rather than inferring it. |
| GPTBot · ClaudeBot · CCBot | Training | Nothing, for visibility purposes. Decide the policy separately and stop conflating it with citations. |
Do AI crawlers execute JavaScript?
Mostly not, in the lanes that fetch your page directly — and the useful version of that statement comes with dates attached, because this is a moving target and nobody should be quoting a two-year-old test as a permanent property of the web.
Two dated studies carry most of the weight here. They agree on the direction and disagree usefully on the detail, which is exactly why both are worth reading rather than one being summarised into a slogan.
Neither study licenses the claim that a given provider will never render. Treat both as snapshots of tested systems on tested dates.
- December 2024 · Network-level study
Vercel and MERJ: fetched, not executed
Working from real crawler traffic rather than a lab page, the analysis found no evidence of JavaScript execution by the major AI crawlers. The detail that makes it convincing: these crawlers did request JavaScript files — around 11.5% of ChatGPT’s crawler requests and around 24% of Claude’s — and still did not run them.
- October 2025 · Controlled page test
searchVIU: one system rendered on a live fetch
A test page carried the same price embedded eight different ways, and five AI systems were asked to fetch it. A price that existed only in JSON-LD was extracted by none of them. Gemini was the only system to return the JavaScript-rendered price on a direct live fetch. Google AI Mode and Perplexity returned it too — but after indexing, which is the index lane doing the work, not the fetcher.
- Today
What neither study proves
Not that a vendor is incapable of rendering. Not that your specific page behaves the way a test page did. Not that content a crawler can reach will be used, ranked or cited. Access is the floor, not the outcome.
The slogan
“AI crawlers don’t run JavaScript, so CSR kills AI visibility.”Too absolute. It ignores the index lane entirely, and it has been falsified for at least one system on a live fetch.
The defensible version
“Content that only exists after JavaScript runs is reliably available in one lane and unreliable in the others.”Which is still a strong reason to serve it in HTML — and one you can actually defend in a meeting.
The five-minute test you can run right now
No tooling, no licence, no developer. Pick one important public page and one unusual sentence from its main content. Then find out whether that sentence exists before JavaScript runs.
- Copy a distinctive sentence from the visible main content — a product description, a service benefit, something no other page would have. Avoid navigation and footer text.
- Open the response HTML. In Chrome or Edge, press Ctrl + U, or prefix the address with
view-source:. Search inside it with Ctrl + F. This is what your server sent. - Open the rendered DOM. Press F12 (or Ctrl + Shift + I) and search the Elements panel for the same sentence. This is what the browser built.
- Turn JavaScript off and reload. In DevTools press Ctrl + Shift + P, type Disable JavaScript, run it, then hard-reload with Ctrl + Shift + R. Whatever disappears is JavaScript-dependent.
- Repeat for the signals, not just the copy: the H1, the canonical tag, the robots meta tag, your primary navigation links, and any JSON-LD.
- Confirm from outside the browser. In Command Prompt or PowerShell, run
curl.exe -sSL https://example.com/page— in PowerShell, plaincurlis an alias forInvoke-WebRequestand behaves differently, so callcurl.exeexplicitly.
It tells you the content is JavaScript-dependent. It does not tell you whether Google indexed it, whether an assistant could retrieve it, or whether it mattered. The next four tests are what turn the clue into a finding.
See the difference: raw response versus rendered page
Switch the architecture below to watch the same page change shape. This is a teaching simulation drawn on the page — it does not fetch a live site or emulate any real crawler.
What the server returned
<div id="app"></div> <script src="/assets/app.js"></script>
What the browser built
Trail Running Shoe
Built for mixed terrain, with a breathable upper, a grippy outsole and a size guide.
View details →What you’d conclude: the response is an empty shell. Everything a reader values is inserted by client JavaScript, so only a consumer that executes it will ever see the product.
How Google turns JavaScript into an indexed page
Google describes three phases: crawling, rendering and indexing. A URL is fetched, parsed for links, queued for rendering in a headless Chromium kept close to current, and the rendered HTML is what gets indexed. Rendering is deferred rather than instant, which is why the raw response and the rendered DOM can both end up influencing how a page is treated.
Four rules worth knowing exactly
Google’s JavaScript documentation picked up several clarifications over the past year, and each one closes an argument that used to run and run.
Non-200 may never be rendered
Pages returning 200 are queued for rendering whether or not they use JavaScript. If the status is not 200 — a 404, for instance — rendering might be skipped entirely.
You cannot script your way out of it
When Google meets a noindex tag it may skip rendering and JavaScript execution altogether. Removing that tag with client-side JavaScript therefore may not work. If a page should be indexed, don’t ship noindex in the original code.
HTML first, and only once
You can inject a canonical with JavaScript, but you should not use it to change the canonical to a different URL from the one in the original HTML. Either put it in the HTML, or leave it out of the HTML and set it with JavaScript — never both, and never conflicting.
Fingerprint your filenames
Google’s renderer caches aggressively and may ignore your caching headers, which can leave it running an outdated bundle. Content-based filenames such as main.2bb85551.js sidestep the problem.
Two more that catch people out: links are only discoverable as <a> elements with an href, so client-side routing should use the History API rather than hash fragments; and with web components, Google flattens shadow and light DOM, so only what ends up visible in the rendered HTML can be indexed. The JavaScript SEO basics documentation is the authority on all of this.
SSR, CSR, SSG and ISR: what changes for SEO?
Rendering strategy is an engineering trade-off, not a ranking lever. What it determines is whether the content exists in the response — everything downstream still has to be earned.
| Approach | What the response contains | Good fit | Where it goes wrong |
|---|---|---|---|
| CSRClient-side rendering | Usually an app shell; content arrives in the browser | Dashboards and signed-in applications | JavaScript-only copy and links, slow or failing APIs, nothing for the live-fetch lane |
| SSRServer-side rendering | Page-specific HTML from the server | Service pages, dynamic product pages | Latency, cache misconfiguration, hydration mismatches, client-only islands nobody audited |
| SSGStatic site generation | Prebuilt HTML | Guides, documentation, evergreen content | Build times, and stale pages when content changes between builds |
| ISR / hybridIncremental or mixed | Pre-rendered HTML refreshed on platform rules | Large catalogues and frequently updated sites | Invalidation logic, freshness, edge-cache correctness |
And hydration?
Hydration attaches interactivity to server-rendered markup, and it is often the right compromise. It is also where I find the nastiest bugs, because the response looks perfect and the final DOM does not. A failed API call during hydration can blank a section that was present in the HTML a moment earlier. Inspect both states, not just the one that supports your argument.
Serving a pre-rendered version to bots while users get client-side rendering is described by Google as a workaround rather than a long-term solution, with server-side rendering, static rendering or hydration recommended instead. It still appears in audit templates. It should not appear in new builds.
The Render Gap Ladder: six tests, cheapest first
Each rung answers a question the one below it cannot. Work upward and stop when you have enough evidence to act — most problems are settled by rung two or three. Your ticks are saved in this browser only.
The Screaming Frog raw-versus-rendered parity audit
This is rung four, done properly. The SEO Spider crawls in text-only mode by default, which is the raw HTML; switching to JavaScript rendering makes it crawl both versions and report the differences. The exact paths below are from the current tutorial, and menu labels do shift between releases, so treat the official tutorial as the authority if yours looks different.
Set it up
Configuration > Spider > Rendering→ set Rendering to JavaScript.Configuration > Spider > Extraction→ tick Store HTML and Store Rendered HTML.- Confirm images, CSS and JavaScript are checked under
Configuration > Spider, and tick external links if resources sit on another domain. - The default rendering viewport is Googlebot Smartphone; change the agent under
Configuration > HTTP Header > User-Agentif you need to.
Read the result
- The JavaScript tab carries 15 filters covering the common dependencies and faults.
- Lower pane → View Source → Show Differences for a raw-versus-rendered diff, and switch the HTML filter to Visible Content to isolate the text.
- Lower Outlinks tab → link origin filter Rendered HTML shows links that only exist after execution.
- Rendered Page tab gives you the screenshot and the blocked resources together.
The filters that earn their keep
Four of the fifteen do most of the work in a real audit, because they catch signal problems rather than word-count differences.
| Filter | What it is telling you | Why it matters |
|---|---|---|
| Noindex Only in Original HTML | The raw HTML carries a noindex that the rendered HTML does not. | The highest-severity finding on this list. Googlebot may skip rendering when it sees that tag, so the JavaScript that removes it never runs. |
| Canonical Mismatch | Raw and rendered HTML declare different canonicals. | Conflicting canonicals produce unpredictable consolidation. Fix the raw one. |
| Canonical Only in Rendered HTML | No canonical until JavaScript runs. | Works, but relies on the render. Put it in the response or the HTTP header instead. |
| Contains JavaScript Content / Links | Body text or hyperlinks that appear only after execution. | The core render gap. Triage by template and commercial value, not by percentage. |
| Pages with Blocked Resources | Scripts, CSS or images disallowed in robots.txt. | Blocked resources break rendering for everyone. Export the offenders via Bulk Export > Response Codes > Blocked Resource Inlinks. |
Interactive tool
How much of this page needs JavaScript?
Drop in the word counts from your own crawl. This is a triage aid for deciding where to look first — it is not a Google metric and there is no safe threshold.
Formula: (rendered − original) ÷ rendered. Differences can also come from menus, footers, personalisation or extraction settings, so always confirm which content moved before you call it a problem.
Twelve rendering failures, and what each one needs
Filter to the group you are investigating. Each entry gives the observable symptom, the test that confirms it and the direction a developer should take — in that order, because a fix proposed before the test is just a guess with a Jira ticket.
01 · Main copy only exists after JavaScript
Symptom: the H1 and body text are visible in the browser and absent from the response.
Test: unique-sentence search in view-source versus the Elements panel, then the Search Console live render.
Fix: server-render or statically generate the important public copy, and make sure hydration preserves it.
02 · Important internal links need a click handler
Symptom: navigation built from <div onclick>, href="#" or script-only routing.
Test: inspect the actual href attributes, and compare the crawler’s link export with the rendered outlinks.
Fix: real <a href="/path"> elements pointing at real URLs. Layer the JavaScript behaviour on top.
03 · Infinite scroll with no crawlable pagination
Symptom: items load as you scroll and no distinct paginated URLs exist.
Test: disable JavaScript, crawl the links, then request the pagination routes directly.
Fix: expose linked paginated URLs and keep infinite scroll as the enhancement.
04 · The canonical changes after rendering
Symptom: the raw canonical disagrees with the rendered one, or several appear.
Test: Screaming Frog’s Canonical Mismatch and Canonical Only in Rendered HTML filters.
Fix: one correct canonical in the server HTML, and nothing client-side that rewrites it.
05 · JavaScript tries to remove a noindex
Symptom: the response says noindex and client code strips it on load.
Test: read the robots directive in the initial response and check the indexing state in Search Console.
Fix: remove it from the source. Google may skip rendering when it sees noindex, so the script never gets its chance.
06 · Titles, descriptions or JSON-LD diverge between versions
Symptom: metadata arrives late, duplicates, or describes the wrong item.
Test: the Page Title / Meta Description “Updated by JavaScript” filters, plus a structured data test on the rendered output.
Fix: emit stable critical metadata on the server; keep any client injection consistent and testable.
07 · Scripts, APIs or CSS fail for the crawler but not for you
Symptom: blank regions, console errors, blocked resources or failed API calls in the rendered view.
Test: Search Console’s page resources list, the browser Network panel, and your server logs for the bot’s follow-up requests.
Fix: unblock what rendering needs, make the APIs resilient, and give critical content a fallback.
08 · Hydration removes correct server-rendered content
Symptom: the HTML is complete and the final DOM is emptier than the response.
Test: compare the initial markup with the post-hydration DOM and read the hydration warnings.
Fix: align server and client state, handle API errors without destroying the markup, and avoid wholesale replacement of core content.
09 · Content needs an interaction or a session to appear
Symptom: critical copy requires a click, a scroll, a stored cookie or a signed-in session.
Test: a clean browser profile with no cookies, JavaScript disabled, then a live Search Console render.
Fix: public indexable information should not depend on user state. Google does not click, hover or scroll.
10 · SPA URLs are not independently addressable
Symptom: hash fragments or client-only state stand in for distinct pages.
Test: open the deep links directly in a clean browser and see what the server returns.
Fix: stable URLs with History API routing and a meaningful server response for each one.
11 · A missing product returns HTTP 200
Symptom: the page shows an error message and the response says everything is fine.
Test: check the status with a header request, the Network panel, and Search Console’s soft 404 reporting.
Fix: return a real 404, or redirect client-side to a URL that does. Adding noindex with JavaScript is the documented fallback when a real status code is impractical.
12 · Caching or edge rules hide stale and incomplete HTML
Symptom: some regions, bot sessions or cache states receive old or partial responses.
Test: compare CDN response headers, deployment hashes and locations against verified log records.
Fix: audit invalidation, cache keys and WAF rules until delivery is consistent and testable.
React, Next.js, Nuxt, Angular and headless CMS
The framework is not the diagnosis. Two sites on the same framework can behave completely differently depending on route configuration, data fetching, caching and deployment. Test the production response per route and let that settle the argument.
Check where the shell ends
Client-side rendering is often exactly right for a signed-in application. Public landing and detail routes deserve a separate delivery decision from the app they live beside.
Framework support ≠ page coverage
Inspect each route’s server output, streaming boundaries, data fetching and hydration. A framework that supports SSR does not mean every component on every route was server-rendered.
Per-deployment, not per-docs
Both offer server and static options. What matters is the HTML your deployment actually emits and whether internal links are crawlable in it.
Is content fetched before or after delivery?
If the CMS call happens client-side, your content is as reliable as that API on the day a crawler arrives. Check the caching layer and the error path too.
Which rendering approach fits the page you are working on?
Pick the page type and how often it changes. What comes back is a starting hypothesis for a conversation with engineering, not an architectural prescription — most real sites end up hybrid anyway.
Interactive tool
Start the conversation here
Before you commit, test caching, freshness, TTFB, LCP, INP, canonical consistency and crawlable URLs on a representative route.
Server-side rendering does not automatically improve Core Web Vitals
Moving rendering to the server changes where the work happens; it does not delete the work. It usually helps meaningful content appear sooner, and it can just as easily add server latency or ship a hydration bundle heavy enough to hurt responsiveness. Measure it rather than assuming it.
Compare field data by template, before and after, alongside lab traces. “We moved to SSR” is a description of a change, not evidence of an improvement.
You found the render gap. Now get a developer to care.
The strongest technical recommendation is one an engineer can reproduce in two minutes. Replace “Google can’t see this page” with the exact difference between expected and actual output, the affected template, and acceptance criteria that can be retested after deployment.
Interactive tool
Build the ticket
Fill in what you found. The output is a plain-text issue with reproduction steps and acceptance criteria.
Prefilled with an example. Replace it with real, timestamped evidence before you share it.
How do you know the fix actually worked?
Track the evidence by layer instead of compressing everything into one invented readiness score. Delivery improves first, crawling and indexing follow, and anything involving AI answers moves last and least predictably.
| Layer | Where the evidence comes from | What success looks like |
|---|---|---|
| Delivery | HTTP response and a raw-versus-rendered diff | Critical content and signals present in the response on the affected templates. |
| URL Inspection, Page indexing report, Search performance | The issue is no longer reproducible. Eligibility and traffic are tracked separately, and neither is promised. | |
| Crawl at scale | Screaming Frog parity re-crawl | Fewer high-value URLs in the JavaScript tab filters than before deployment. |
| Bot access | Verified server logs and WAF reports | Expected crawlers receive the intended resources and status codes. |
| AI visibility | Search Console’s generative AI report, Bing’s AI Performance report, repeated retrieval observations | Documented change over time — recorded as observation, never as a guarantee. |
Two first-party reports that did not exist for most of this debate
Until recently, “are we visible in AI answers?” had no first-party answer at all. That changed twice in 2026, and both reports belong in the measurement plan — neither substitutes for a rendering audit.
Generative AI performance reports
Search Console gained dedicated reports isolating impressions inside AI Overviews and AI Mode, with a separate Discover view. Announced on 3 June 2026 and rolled out to all websites worldwide on 31 August 2026. Version one reports impressions only — no clicks, no CTR — plus a toggle to opt out of generative AI features.
AI Performance in Bing Webmaster Tools
A public preview showing total citations and average cited pages, broken out by grounding query and by page, across Copilot, Bing’s AI summaries and selected partner integrations. It does not cover ChatGPT, and it reports citation frequency rather than clicks.
For the crawler side of this, pair the audit with my log file analysis playbook, and work the queries at scale using Search Console data in BigQuery. For a broader pre-release sweep, the SEO checklist for Google and AI search covers the rest of the surface.
Where this method breaks
The ladder is built to slow down confident conclusions. It has real limits, and you should know them before you lean on it in front of a client.
The studies age
The crawler findings here are from December 2024 and October 2025. Vendors change behaviour without announcements. Re-run the retrieval test rather than citing a study as a permanent fact.
No tool is the real renderer
Screaming Frog, your browser and a live Search Console test are three approximations of a pipeline you cannot observe directly. They disagree sometimes, and that disagreement is information.
Access is not outcome
Fixing delivery removes a blocker. It does not produce rankings, AI citations or revenue, and anyone promising that chain is selling something.
Built around Google and documented crawlers
This guide follows published Google guidance and dated third-party research. It does not describe undocumented retrieval pipelines, and I will not pretend to know them.
One more: a 97% render gap on a template nobody searches for is not an emergency. Triage by business value and reproducibility, not by the size of the number.
JavaScript SEO questions.
Can Google index a client-side rendered website?
Yes. Google renders JavaScript with an evergreen Chromium and indexes the rendered HTML. Individual pages still fail for ordinary reasons — blocked resources, conflicting directives, slow or failing APIs, unaddressable URLs — so test the actual rendered output rather than assuming the capability covers your implementation.
Do AI crawlers execute JavaScript in 2026?
Generally not in the lane that fetches a URL on demand. A December 2024 network-level study found the major AI crawlers requesting JavaScript files without executing them, and an October 2025 controlled test found only Gemini returning a JavaScript-rendered value on a direct live fetch. Content that reaches those systems through a search index is a different pathway and can behave differently.
Is server-side rendering always better for SEO?
No. SSR guarantees the content is in the response, but static generation achieves the same thing for suitable pages with fewer moving parts, and client-side rendering is perfectly appropriate for pages that should never be indexed. Verify the output of whatever you implement.
Does a successful Search Console live test mean the page is indexed?
No. A live test shows that Google could fetch and render the URL at that moment. Indexing is a separate decision, reported separately, and the indexed view in URL Inspection can show something quite different from the live view.
What is the difference between View Source and Inspect Element?
View Source shows the HTML your server returned. Inspect Element shows the DOM after the browser has parsed it and JavaScript has modified it. A difference between the two is a diagnostic clue about where your content lives, not a verdict on whether anything is broken.
Should I use dynamic rendering to fix this?
Not for a new build. Google describes serving a bot-specific pre-rendered version as a workaround rather than a long-term solution, and points to server-side rendering, static rendering or hydration instead. If you already run it, treat migrating off it as the project.
Will server-rendering my pages get me cited in AI answers?
It removes one reason you would not be. Access, indexing, relevance, quality, retrieval and citation are separate stages, and reliable HTML only improves the first. Anyone presenting SSR as an AI citation strategy is skipping five steps.
How should I handle structured data on JavaScript pages?
Google supports JSON-LD generated and injected with JavaScript, and says to test the result. But in a 2025 controlled test, no AI system extracted a price that existed only in JSON-LD. Mirror your structured data in visible content, and prefer stable JSON-LD in the initial HTML — it is easier to test and it survives more pathways.
What should an enterprise site fix first?
High-value indexable templates where main content, internal links or indexing directives fail reproducibly — in that order of severity, with noindex in the raw HTML at the top because it can stop rendering before anything else gets a chance. Triage by affected URLs and business importance, never by JavaScript word-count percentage.
Does blocking GPTBot remove my pages from ChatGPT?
No. GPTBot is the training crawler. Search discovery runs through a different agent, and user-triggered fetches through a third. Blocking the training crawler is a legitimate policy choice, but it is not a visibility control, and plenty of robots.txt files confuse the two.
Take the method with you
Keep the evidence, not just the screenshot.
A render-gap audit log with five worked sample rows, the six ladder tests as columns, and a decision key. Creator details travel with the file so the source stays identifiable when it is shared.
Sample rows use fictional example domains and set no benchmark.
Stuck on a rendering problem?
Send me the URL and what you have tried. I read every note.