EN
Free audit

Can AI crawlers read JavaScript websites?

Updated 4 min read

Short answer

Mostly no. Crawlers used by ChatGPT, Claude and Perplexity generally read the raw HTML your server sends and don't run JavaScript, so text rendered in the browser is invisible to them. Google's systems render JavaScript, others mostly don't. Check your pages with JavaScript disabled, and fix with server-side rendering, static generation or prerendering.

Why does JavaScript matter for AI visibility?

Many modern sites send the browser an almost empty HTML file plus a bundle of JavaScript that builds the page. People see the full page, because their browser runs the scripts. A crawler that doesn't run JavaScript sees only the empty shell: a title, maybe a menu, and a <div id="app"></div>.

If your product descriptions, prices, delivery terms and company information appear only after scripts run, an AI crawler that skips JavaScript has nothing to read, nothing to quote and no reason to recommend you.

Which AI crawlers execute JavaScript?

Rendering JavaScript is expensive, and most AI crawlers skip it:

CrawlerRenders JavaScript?
Googlebot (Google Search, AI Overviews, Gemini grounding)Yes
BingbotYes, with limits
GPTBot, OAI-SearchBot, ChatGPT-UserGenerally no
ClaudeBot, Claude-SearchBot, Claude-UserGenerally no
PerplexityBotGenerally no
CCBot (Common Crawl)No

An analysis published by Vercel in late 2024, based on hundreds of millions of crawler requests, found that OpenAI's and Anthropic's crawlers download JavaScript files but don't execute them. Crawlers change, so treat this as the current pattern rather than a permanent rule, but design for the worst case: if the text is not in the HTML, assume AI can't see it.

How do you check your own site?

Three quick ways:

  1. View source. Open a key page, press Ctrl+U (or Cmd+Option+U) and search for a sentence from the visible text. If you can't find it in the source, it is rendered by JavaScript.
  2. Disable JavaScript. In Chrome DevTools, open the command menu (Ctrl+Shift+P), type "Disable JavaScript", reload the page. What remains is roughly what AI crawlers see.
  3. Fetch with a bot user agent. Run curl -A "GPTBot" https://yoursite.com/page and look at the text in the response. This also reveals firewalls that block bots.

Our free audit counts the readable words on your key pages without running scripts and flags pages that look empty.

Which pages matter most?

You don't have to fix the whole site at once. Start with pages that answer the questions customers ask AI:

  • homepage and About page (who you are, what you offer, where);
  • main category or service pages;
  • top product pages (name, key specifications, price, availability);
  • delivery, payment, returns and warranty pages;
  • contact and location pages;
  • FAQ and guides.

What are the fixes?

ApproachHow it worksGood for
Server-side rendering (SSR)The server builds the full HTML for every requestDynamic sites: stores, catalogues
Static site generation (SSG)Pages are built into HTML at deploy timeContent sites, landing pages, blogs
Incremental or hybrid renderingStatic pages refreshed on a scheduleLarge catalogues
Prerendering for botsA service serves rendered HTML to crawlersLegacy single-page apps you can't rebuild
Progressive enhancementCore text in HTML, JavaScript adds interactivityAny site

Frameworks such as Next.js, Nuxt, SvelteKit, Astro and Remix support SSR or SSG out of the box. For an existing React or Vue single-page app, a prerendering layer is the fastest fix, but make sure bots and people get the same content: serving different content to crawlers is cloaking.

What about content in tabs, accordions and sliders?

If the text is present in the HTML and only hidden with CSS, crawlers can read it. If a tab loads its content from an API only when clicked, they can't. Product specifications and reviews are frequent victims; keep them in the initial HTML.

Does this affect structured data too?

Yes. JSON-LD injected by JavaScript, for example through a tag manager, may never be seen by crawlers that don't render. Put schema markup directly in the server-rendered HTML.

Is llms-full.txt a workaround?

Partly. A clean Markdown copy of your key pages in llms-full.txt gives tools that read it a readable version of your content. But it's not a confirmed input for AI answers, and assistants that search the web read your normal pages. Fix the rendering first.

If your site is built with a JavaScript framework, the AI-Ready Pack shows exactly which pages are empty without scripts and includes the files you can publish while your developers work on rendering.