Skip to content
pageinspection

Content that needs JavaScript: who can read it and who cannot

Googlebot runs JavaScript. That fact has been used for a decade to end the conversation about client-side rendering, and it is now the wrong fact to end it on, because the fetchers that decide whether an assistant can quote you mostly do not.

The short answer

GPTBot, ClaudeBot, PerplexityBot and CCBot do not execute JavaScript, so text that exists only after hydration does not exist for them. Googlebot renders and will usually catch up on a second pass; AI crawlers will not. The test is to fetch the URL with a plain request and count the words that come back.

CriticalAudit check · Text extractability

The split that changed the answer

For years the honest advice about client-rendered content was: Google handles it, ship it. Google does handle it, with a second rendering pass, and pages built entirely in the browser rank perfectly well.

What has changed is who else is fetching. A retrieval fetcher answering a user's question right now is optimised for latency, not completeness: it makes a request, takes the bytes, and moves on. Running a headless browser per fetch is expensive and slow, and for most of these agents there is no second pass. Whatever is in the response body is what exists.

So the same page can be entirely visible to search and nearly empty to an assistant.

What the difference actually looks like

The reason this defect survives is that it is invisible from a browser. You are looking at the rendered page, so you are looking at the version that works.

Two panels showing the same URL. With JavaScript, the page has a heading, five paragraphs and a three-item list. Without JavaScript, only a short shell remains and there is no text to extract.With JavaScriptWithout JavaScriptno text to extractSame URL. The right panel is what a fetcher that does not run scripts receives.
One URL, two fetches. Nothing about the left panel tells you the right panel exists.

The shell on the right is the giveaway. A client-rendered app typically ships a document containing a title, some link tags and an empty mount point, with every word of content arriving afterwards from an API call. To a fetcher that does not execute the bundle, the page is that shell.

Testing it in about thirty seconds

You do not need a tool for the first pass. curl makes exactly the request a non-rendering fetcher makes, so piping it through a word count answers the question.

check-render.sh
# What arrives before any script runs. Send a real
# user agent: some hosts serve a different shell to
# clients they do not recognise.
curl -s -A "Mozilla/5.0 (compatible; check)" \
  https://example.com/page \
  | tr -d '\n' \
  | sed -E 's/<script[^>]*>.*?<\/script>//g; s/<[^>]+>/ /g' \
  | tr -s ' ' \
  | wc -w

# Then look at what is actually there, not just how
# much of it: does your h1 and first paragraph appear?
curl -s https://example.com/page | grep -o '<h1[^>]*>[^<]*'

A page whose article body is two thousand words and whose raw response yields forty is client-rendered content. In the browser, the equivalent check is to disable JavaScript in devtools and reload: what remains is roughly what a non-rendering fetcher gets.

Check a template, not the homepage.

Marketing homepages are frequently static while the pages that carry your actual substance, docs, product detail, article bodies, are the rendered ones. Testing the homepage is how this gets declared fine.

Checking this on your own site

The command above answers it for one URL. Across a site, the question is which templates are affected, which is a comparison rather than a measurement.

This one needs the full crawl

Answering this means fetching the page twice, once with JavaScript and once without, then comparing what text survives. The crawl does that comparison; a single rendered snapshot has nothing to compare against.

The instant search on this site audits a single page, so rather than show you a verdict it cannot support, this guide sends you to the place the check actually runs.

Fixing it without rewriting the app

The framing that helps: you do not need to abandon client-side rendering, you need the first response to contain the content. Every fix below is a way of doing that.

Server rendering or static generation. The real answer if it is available to you. In the App Router this is the default and the problem usually comes from opting out of it: a page marked 'use client' at the top of the tree, or content fetched in an effect rather than on the server.

app/docs/[slug]/page.tsx
// The version that ships an empty shell: the fetch
// happens in the browser, after the bundle loads.
//
//   'use client'
//   export default function Page({ params }) {
//     const [doc, setDoc] = useState(null)
//     useEffect(() => {
//       fetch(`/api/docs/${params.slug}`)
//         .then(r => r.json()).then(setDoc)
//     }, [params.slug])
//     if (!doc) return <Spinner />
//     return <article>{doc.body}</article>
//   }

// The same page, resolved before the response is sent.
// The HTML that leaves the server contains the text.
export default async function Page({ params }) {
  const doc = await getDoc(params.slug)
  return (
    <article>
      <h1>{doc.title}</h1>
      {doc.body}
    </article>
  )
}

Prerendering at the edge for bots. A middle path when a rewrite is not on the table: serve a rendered snapshot to non-browser clients. It works, and it has a real cost, which is that you now maintain two paths and only one of them is exercised by your team.

A meaningful noscript body. Not a fix, and worth mentioning because people reach for it. A <noscript> block containing the article text does put the content in the response, but maintaining a parallel copy of your content is a guarantee that the two will diverge.

When this fires and the page is fine

Not every dynamic thing on a page needs to be in the first response, and the check is not asking for that.

  • Interactive widgets. A pricing calculator, a map, a chart with controls. Nobody is quoting your slider. What matters is whether the surrounding explanation is in the markup.
  • Content behind a genuine interaction. A tab panel or accordion whose content is in the HTML and merely hidden by CSS is fine. One that fetches on click is not, and the two look the same from the outside.
  • Personalised or authenticated regions. A signed-out fetcher was never going to see them, and that is the correct behaviour rather than a defect.

The line to hold is the argument of the page. If the sentences a reader came for are not in the response body, the page fails this check regardless of how much else is.

Why this outranks every other AI check

Because it is the one defect in the AI group that removes the content rather than degrading it. A missing llms.txt costs you a signpost. A weak heading outline costs you clean passage boundaries. This costs you the text, and everything else on the list is downstream of there being text.

It is also template-scoped, which is the multiplier the severity scale weighs most heavily: one rendering decision in one layout applies to every page beneath it, and the pages most likely to be affected are the deep content pages that were supposed to earn the citations.

Confirming the content is in the response

Re-run the same curl, on the same URL, and compare word counts before and after. It is a blunt measure and it is the right one: the number going from forty to two thousand is the fix landing. Then grep the response for a distinctive sentence from the middle of the page rather than trusting the count alone.

Check it again after any change to your data-fetching approach, because this is a defect that gets reintroduced by a refactor rather than by an edit. Moving one fetch into an effect to fix a loading state is enough to empty a template again, and nothing in your test suite will notice.

Questions this check raises

Do AI crawlers execute JavaScript?
The major ones do not. GPTBot, ClaudeBot, PerplexityBot and CCBot fetch the HTML and read what is in it. That is the whole reason a client-rendered page can be fully indexed by Google and completely invisible to an assistant asked about it.
How do I test what a non-rendering crawler sees?
Fetch the URL with a plain command-line request and read the response body, or disable JavaScript in your browser and reload. If the page is an empty shell with a loading spinner, that is the document every non-rendering client receives.
Do I have to rewrite my app to fix this?
Usually not. Server-rendering or statically generating the pages that matter is enough, and every major framework supports it per route. The pages that need it are the ones with content worth citing, which is rarely the whole application.