Optimising your site for AI: making your pages readable and citable by answer engines

Server rendering, robots.txt, structured data, llms.txt, citation signals and measurement: what we actually put in place so an AI can read, understand and cite your pages.

Optimising your site for AI: the chain that makes a page accessible, extractable and citable
LinkedInTwitterFacebookEmail

Optimising your site for AI means taking every page through four stages: accessible to the crawler (content served as HTML, crawler allowed), extractable (direct answer up front, tagged structure, structured data), attributable (date, author, named source, every figure tied to its origin) and measurable (server logs, an AI channel in analytics, a prompt panel). Each stage breaks independently of the others: a beautifully written page rendered in client-side JavaScript does not exist for GPTBot; a perfectly accessible page that attributes none of its figures will never be picked up as a source.

This article is the operational sequel to our guide AEO & GEO: how to be cited by ChatGPT and AI engines, which sets out the what and the why. Here we cover the how: what you put in place on Monday morning, in which order, and what is a waste of effort. Every market figure quoted is dated and attributed — which is also, incidentally, a demonstration: this article applies to itself the method it describes.

In short

  • The shift is measured, not predicted: over January–April 2026, 68.01% of Google searches in the United States ended without a single click — 680 searches out of 1,000 — while only 276 clicks reached the open web (SparkToro, Similarweb panel, June 2026; figure relayed by Search Engine Roundtable on 10 June 2026). 680 ÷ 276: roughly 2.5 zero-click searches for every click won. In France: 65.3%.
  • France has just crossed over: Google rolled out AI Overviews and AI Mode in France on 22 July 2026, after two years held up by neighbouring-rights negotiations.
  • The first breaking point is technical: the crawlers run by OpenAI and Anthropic do not execute JavaScript (Vercel × MERJ analysis, December 2024, still accurate in mid-2026). A client-rendered site is invisible to them.
  • robots.txt has become a trade-off, not a tick-box: training, answer index and user-triggered visit are three different bots at OpenAI and at Anthropic — and User-agent groups do not stack.
  • llms.txt does not deliver what it promises: 97% of published files have never received a single request (Ahrefs, 137,000 domains, May 2026 sample) and no major provider has committed to reading it.
  • What makes a page citable comes down to four signals: a visible date, a named source, an attributable figure, an identifiable author.

What is an “AI-readable” site?

An AI-readable site is one where every page can be retrieved without executing JavaScript, understood without outside context, split into self-contained answers and traced back to an identifiable source. It is not a label or a technology: it is a four-link chain, and the weakest link sets the result.

LinkThe question it answersWhat breaks it
1. AccessibleCan the crawler retrieve the content?Client-side rendering, content behind a form or a wall, crawler blocked in robots.txt
2. ExtractableCan it isolate a self-contained answer?Answer buried at the end of a section, vague headings, walls of text, information carried by an image
3. AttributableCan it say where the information comes from?Figures with no source, pages with no date, anonymous authors, unverifiable claims
4. MeasurableDo you know whether it works?Logs never read, AI traffic lost inside “Referral”, no citation tracking

The difference with classic SEO is not ideological, it is mechanical. Googlebot renders JavaScript; most AI crawlers do not. Google interprets a page in the context of a whole site; an answer engine lifts a fragment and has to be able to cite it out of context. Everything that follows flows from those two gaps.

The numbers are worth setting down properly, once. Over the first four months of 2026, 68.01% of Google searches in the United States ended without a click, against 60.45% in 2024 on a recalculated basis — the sharpest acceleration SparkToro has ever measured, using Similarweb's desktop and mobile panel. Out of 1,000 US searches, only 276 clicks reach the open web, against 360 in SparkToro's 2024 study (the widely quoted 374 figure is the European Union one, not the US one). Let us do the arithmetic in plain sight, since this is the ratio everyone repeats without ever writing it down: 68.01% of 1,000 searches is 680 searches with no click at all; against that, 276 clicks to the open web. 680 ÷ 276 = 2.46, or roughly 2.5 zero-click searches for every click a site wins. In France, the zero-click rate stands at 65.3% over the same period — one of the lowest of the six countries SparkToro studied, behind Germany (62.1%) and well below the United Kingdom (69.5%).

And the French market has only just crossed over: Google launched AI Overviews and AI Mode in France on 22 July 2026, after two years held up by the neighbouring-rights file. In other words, the French curve has not yet absorbed the shock that English-speaking markets have been taking for two years. The next six months are a window — not a given.

Are your pages accessible to AI crawlers, or hidden behind JavaScript?

If your page content is not present in the HTML returned by the server, the main AI crawlers will never see it: unlike Googlebot, they do not render JavaScript. This is by far the most common and most expensive breaking point — and it is invisible to the naked eye, since the page displays perfectly in your browser.

Client-rendered content is invisible to AI crawlers, which do not execute JavaScript

The analysis run by Vercel with MERJ, published on 17 December 2024, remains the reference measurement: across one month of traffic and close to a billion AI crawler fetches (569 million for GPTBot, 370 million for Claude, 24.4 million for PerplexityBot), GPTBot (OpenAI) and ClaudeBot (Anthropic) download JavaScript files — around 11.5% and 23.8% of their requests respectively — but do not execute them. No JavaScript execution was observed at all. The study is nearly two years old: all the more reason never to rely on a single study. The test in the next sub-section takes thirty seconds on your own pages — it is the first thing we do in every audit, before we even open the robots.txt. Gemini and Copilot are the exceptions, because they sit on indexes that do render JavaScript: Googlebot for the first, Bingbot for the second.

The direct consequence: a single-page application (React, Vue, Angular) that loads its content after paint, a tab component that only injects text on click, a client-rendered accordion FAQ, a pricing table fetched from an API — all of it is invisible to most answer engines. In the sites we audit, the culprit is almost never the main content — it is a block bolted on afterwards: a reviews widget, an accordion FAQ built by a script, a pricing module wired to an API.

The test, in thirty seconds and with no tooling:

  1. Open the page, then “View page source” (Ctrl+U / Cmd+U) — not the inspector, which shows the DOM after JavaScript has run.
  2. Search (Ctrl+F) for a specific sentence from your body copy.
  3. If it is not there, it does not exist for GPTBot or ClaudeBot.

What we put in place: server-side rendering (SSR), static generation (SSG) or pre-rendering for every route that carries content. On a modern stack this is an architecture setting, not a rebuild — Nuxt, Next and Astro all ship SSR/SSG natively. On a CMS such as Webflow or WordPress the HTML is already served by the server: the risk shifts to injected components (reviews widgets, dynamic FAQs, content loaded by third-party scripts). If your product is a business application or a custom SaaS platform, the right practice is to separate cleanly the public indexable pages (server-rendered) from the authenticated application (client-rendered) — there is no need to expose one to save the other.

The blind spot — A classic SEO audit often misses this, because most tools render JavaScript the way Googlebot does. The content therefore looks “visible”. It is visible to Google. It is not visible to ChatGPT. If your site shows several of these signals at once, the question goes beyond AEO: read the 7 signs it is time to rebuild your website.

Should you allow or block GPTBot, ClaudeBot and PerplexityBot?

The right answer is neither “block everything” nor “allow everything”, because this is not one bot: at OpenAI and at Anthropic, model training, the search index and the user-triggered visit run through three distinct agents, each controllable separately. Blocking the training bot does not stop you being cited; blocking the index bot does.

This is the distinction most robots.txt files miss — usually by copying a template found online, which cuts off the index bots in the belief that it is protecting the content from training.

User-agentPublisherWhat it doesBlocking it cuts you off from…
GPTBotOpenAICollection for model trainingTraining — not citations inside ChatGPT
OAI-SearchBotOpenAIIndex for ChatGPT web searchYour presence in ChatGPT answers
ChatGPT-UserOpenAILive visit triggered by a user(OpenAI notes that, because these visits are initiated by a human, robots.txt rules may not apply)
ClaudeBotAnthropicCollection for trainingTraining
Claude-SearchBotAnthropicClaude's search indexYour presence in Claude answers
Claude-UserAnthropicUser-triggered visit (honours robots.txt)Live look-ups by Claude users
PerplexityBotPerplexityPerplexity's indexYour presence in Perplexity answers
Google-ExtendedGoogleUse of content for Gemini applicationsGemini — not Google ranking, not AI Overviews
Applebot-ExtendedAppleUse of content for Apple's AIApple's AI
CCBotCommon CrawlPublic corpus reused by many modelsA widely distributed training corpus

The trade-off, in plain terms. Three profiles cover almost every case:

  • You want visibility (brochure site, B2B, services, e-commerce) → allow the index bots and the user visits; this is your acquisition channel for tomorrow. The training debate is secondary for you.
  • Your content is the product (media, documentation base, proprietary data, paid content) → block the training bots (GPTBot, ClaudeBot, CCBot, Google-Extended) and keep the index bots open on public pages. You stay cited, you do not feed the models.
  • You have contractual or regulatory obligations on certain content → handle it directory by directory, not site-wide.

Pitfall no. 1: User-agent groups do not stack

A bot that finds a group naming it explicitly in your robots.txt applies that group and ignores the User-agent: * group entirely. That is the RFC 9309 rule, and it is the mistake that breaks the most files: you carefully add an OAI-SearchBot group to allow it, and in doing so you open up the back office, the basket and the internal search URLs that the * group was protecting. Rules are not inherited: anything that must apply to a named bot has to be rewritten inside its own group.

Here is profile no. 2 — “my content is my product” — written out in full, with that constraint respected:

# robots.txt — “my content is my product” profile
# Goal: stay cited in answer engines, without feeding training.

# --- 1. Training bots: refused ---
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Applebot-Extended
Disallow: /

# --- 2. Answer-engine index bots: allowed ---
# ⚠ Every bot named below IGNORES the “*” group in section 3.
# Its disallow rules must therefore be REPEATED here, identically.
User-agent: OAI-SearchBot
Allow: /
Disallow: /cart/
Disallow: /my-account/
Disallow: /*?s=

User-agent: Claude-SearchBot
Allow: /
Disallow: /cart/
Disallow: /my-account/
Disallow: /*?s=

User-agent: PerplexityBot
Allow: /
Disallow: /cart/
Disallow: /my-account/
Disallow: /*?s=

# --- 3. All other bots, including Googlebot and Bingbot ---
User-agent: *
Disallow: /cart/
Disallow: /my-account/
Disallow: /*?s=

Sitemap: https://www.example.com/sitemap.xml

The redundancy between sections 2 and 3 is not clumsiness: it is the only correct way to write the file. Profile no. 1 (visibility) means removing section 1; profile no. 3 (contractual obligations) means adding the directories concerned to every group. Three checks before you push the file: an empty Disallow line allows everything, a robots.txt returning a 500 often gets the whole site treated as disallowed, and a Disallow: / left over from staging reaches production more often than anyone expects.

Two points are widely misunderstood. First, robots.txt is not security: it is a polite request, standardised (RFC 9309), with no authentication and no network blocking. Content that must stay private is protected by authentication, not by a line of text. Second, inside Google's AI features control runs through the classic search tags: noindex, nosnippet and max-snippet apply to AI Overviews and AI Mode, since, as Google's documentation puts it, “AI is built into Search”. Google-Extended has no bearing on them.

Finally, the infrastructure layer is moving: Cloudflare has blocked AI bots by default on new domains since July 2025, and tightens the setting further from 15 September 2026 — for new sites and for all free accounts: search allowed, training and agentic use blocked on ad-bearing pages. If your site sits behind a CDN or a WAF, your robots.txt may not be the final decision-maker: check your host's policy before concluding that you are accessible.

How do you structure an answer so an AI can extract it?

An extractable answer is a sentence that stays true and understandable once lifted out of the page: it names its subject, gives its answer in full, and depends on no preceding sentence. It is the writing habit with the greatest effect, and it costs nothing but a change of habit.

Anatomy of a citable page: question heading, self-contained answer sentence, table, dated source and named author

Compare:

  • “As we saw above, it mostly depends on your context, which is why it matters to think it through early.” — implicit subject, no answer, unusable out of context.
  • “Server-side rendering is essential for AI crawlers, because GPTBot and ClaudeBot do not execute JavaScript.” — self-contained, citable as it stands.

The six rules that make a page extractable:

  1. Answer up front. The answer to the heading's question fits in the first one to three sentences of the section. The reasoning comes after, never before.
  2. Write headings that are real questions. Use the wording people actually search with (“Should I…”, “How do I…”, “What is the difference between…”), as it appears in Google's “People also ask”.
  3. One section = one idea = one citable block. If a section answers two questions, split it: an engine does not lift half an answer.
  4. Ban back-references at the start of a section — “this”, “the latter”, “as seen above”. Repeat the subject. Mild repetition is the price of citability.
  5. Put comparisons in tables and procedures in numbered lists. These are the formats answer engines reproduce most faithfully.
  6. Never trust critical information to an image. A figure, a price or a step inside a visual is information lost. Text carries, images illustrate.

The calibration that works: a 40 to 60 word answer per FAQ question, one canonical definition per concept (“X is…”), and a page that covers a subject in full rather than three subjects halfway. An answer engine does not reward length, it rewards completeness on one intent.

Which structured data should you put in place — and how do you keep a coherent entity?

Structured data is not required to appear in AI answers, but it remains the cheapest way to remove any ambiguity about what your page is, who wrote it and which organisation publishes it. Let us be honest here, because plenty of articles claim the opposite: Google's official documentation is explicit — there are no additional requirements to appear in AI Overviews or AI Mode, and no special optimisation is needed. Markup is not a fast pass. It is a disambiguator.

The four types to install, in order of usefulness:

schema.org typeWhereWhat it settles
Article / BlogPostingEvery articleNature of the content, author, datePublished and dateModified, publisher, image
FAQPagePages with a real FAQEach question becomes a question/answer pair identified as such (no rich result any more — see below)
HowToPages with a procedureEach step is numbered, named and isolable (no rich result since 2023 — see below)
OrganizationWhole site (home page)Who you are: legal name, address, sameAs to your official profiles

Entity coherence is the part almost everyone neglects, and the one that matters most for GEO. A generative engine does not simply read your page: it consolidates a representation of your organisation from all of its traces. Three rules:

  1. One stable, unique @id for your Organization, referenced from every other block (publisher, worksFor) rather than redeclared on each page. An Organization copied ten times with ten name variants creates ten blurred entities.
  2. One name, one address, everywhere. METASENSE, 13 avenue Morane-Saulnier, 78140 Vélizy-Villacoublay: identical on the site, in the JSON-LD, on the Google Business Profile listing, on LinkedIn, in trade directories and in the legal notices. Any variation (shortened legal name, old address, approximate town) manufactures a duplicate entity that engines resolve badly.
  3. sameAs to your official profiles — LinkedIn, Google listing, sector profiles. This is the explicit link between your site and your external traces.

Two field notes, and they apply to the two types the market sells hardest. FAQPage and HowTo no longer produce any rich result in Google. For HowTo, removal dates from 2023 (mobile in August, desktop in September). For FAQPage, the August 2023 restriction became full deprecation on 7 May 2026: no more display, the Search Console report ended in June 2026, API support ended in August 2026. Both markups keep their value — unambiguously stating that a block is a question/answer pair or a sequence of steps, which helps an answer engine extract it — but never sell them to anyone as a SERP gain, and leave them in place: Google states that unused structured data causes no problem. We keep them on our own articles for that reason alone. And markup that contradicts the page does more harm than no markup at all: dateModified must match a real change, and the declared author must exist and be identifiable. On tooling, most CMSs handle the essentials natively or through an embed — a selection criterion we cover in our comparison Webflow or WordPress in 2026.

Should you create an llms.txt file in 2026?

No, not as a priority: llms.txt is an unadopted proposal, no major AI provider has committed to reading it, and crawlers almost never request it. Putting one in place does no harm; doing it instead of the rest is a misallocation.

What it is. A Markdown file placed at the root of the domain (/llms.txt), proposed in 2024, meant to give language models a readable map of the site: the important pages, a description, sometimes the essential content in plain text. Its promoters draw an analogy with robots.txt or sitemap.xml, but for LLMs.

What it is actually worth, with the data:

  • Nobody requests it. Across a corpus of 137,000 domains analysed by Ahrefs (May 2026 sample), 97% of published llms.txt files had never received a single request. The leading crawlers (GPTBot, ClaudeBot, PerplexityBot) do not go looking for it.
  • Google blows hot and cold, but does not use it. John Mueller has publicly compared it to the meta keywords tag — a self-declared signal nobody exploits — and Google has confirmed it does not support it. An honest caveat: Google has also added an experimental llms.txt audit to Lighthouse, which keeps the ambiguity alive. A diagnostic tool is not a commitment to read.
  • No commitment from providers. Neither OpenAI, nor Anthropic, nor Google, nor Mistral has stated that it uses the file in their answer surfaces. OpenAI's documented recommendation for bot control remains robots.txt.
  • Adoption has plateaued, with no measurable effect. The study published by SE Ranking on 20 November 2025 across nearly 300,000 domains measures an adoption rate of 10.13% and, more importantly, no correlation between the presence of the file and citations obtained in AI answers — to the point that removing the llms.txt variable from their prediction model improved its accuracy.

Our position, without hedging. If your CMS or framework generates it automatically, leave it: the marginal cost is nil and the bet is cheap. Do not spend a production day writing it by hand, and never present it as a structural AEO action — it is exactly the kind of “quick win” that keeps a team busy while client-side rendering carries on making the site unreadable. One last common confusion: llms.txt controls nothing. It does not block and it does not allow. Access control is robots.txt and your headers. The only documented use that holds up today concerns technical documentation consumed by AI-assisted development tools.

What actually makes a page citable?

A page becomes citable when an engine can answer four questions without leaving it: when is this from? where does it come from? who says so? and what is it based on? Accessibility and structure make you readable; these four signals make you citable. The distinction decides everything, because a model that has to stand behind its answer will always prefer a source it can credit.

SignalWhat we put in placeThe counter-example to eliminate
Visible freshnessPublication date and last-updated date visible on the page + a consistent dateModified“The ultimate guide” with no date at all
Named sourceInstitute or publisher + year, stated in the sentence, with the link“Studies show that…”
Attributable figureOne figure = one source = one year = one method, all in the same placeAn infographic of orphan numbers
Identifiable authorReal name, role, author page, Person in JSON-LD, LinkedIn sameAs“The editorial team”

This is not an aesthetic preference. The GEO research by Aggarwal et al. (IIT Delhi and Princeton, ACM KDD 2024), which tested nine optimisation methods across 10,000 queries, measures that adding statistics, quotations and verifiable sources gains 30 to 40% on Position-Adjusted Word Count — that is, the share of the generated answer your content occupies, weighted by position. On the subjective impression metric, the gain falls back to 15-30%. So read that result for what it is: evidence density does not get you cited by magic, it increases the surface you occupy when you are.

Two habits complete the picture. Date your figures inside the sentence itself (“68.01% over January–April 2026”) rather than in a footnote: the lifted fragment then carries its own temporal validity. And refresh the pages that carry market data — a 2024 figure in an article dated 2026 damages trust across the whole page. On fast-moving subjects, a quarterly review is the right rhythm.

The simple test — Take any paragraph from your page, copy it alone into a blank document and ask yourself: could a journalist quote this as it stands, knowing who says it and since when? If the answer is no, an answer engine will not quote it either.

How do you measure traffic and citations coming from AI?

AI traffic is hard to measure because a large share of visits arrive with no referrer, and because the citation itself — being mentioned without being clicked — produces no line in analytics. So you have to cross four sources, none of which is enough on its own.

Measuring AI visibility by crossing analytics, Search Console, server logs and a prompt panel
  1. GA4's “AI Assistant” channel. Announced by Google in the Analytics help “What's new” on 13 May 2026, this default channel natively separates traffic from conversational assistants (ChatGPT, Gemini, Copilot and a few others) from the “Referral” block, with no configuration: GA4 assigns the ai-assistant medium as soon as it recognises the referrer. Three limits to know: no backdating before that date, Perplexity stays classified as Referral, and traffic from AI Overviews is counted as Organic Search, since it comes from Google. So check your raw referrers as well (chatgpt.com, perplexity.ai, claude.ai, copilot.microsoft.com).
  2. Search Console's “generative AI features” report. Announced by Google on 3 June 2026, it isolates impressions of your pages in AI Overviews, AI Mode and Discover's generative features. The rollout, gradual at first, has been complete for every site worldwide since 31 August 2026. One limit remains as this article goes out: the report returns impressions only, no click data. Read it as an exposure indicator, not a performance report — and do not worry if it does not appear: Google only shows it above a certain data volume.
  3. Your server logs — the only truly reliable measure of reading. Count requests by user-agent (GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, PerplexityBot), track response codes and pages visited. This is where you see whether the bots come, what they read, and whether they hit errors. Verify bot authenticity against the IP ranges published by each provider: user-agent spoofing is common.
  4. A prompt panel, recorded by hand. Pick 20 to 30 real questions from your market, put them to the main answer engines every month, and note: are you cited? how are you described? who is cited in your place? It is manual, it is the most instructive, and it is also the only way to catch a wrong description of your business — which is fixed by working on entity coherence, not on content.

Two useful additions. Watch your indexing in Bing Webmaster Tools: Microsoft's index feeds Copilot and, according to several industry analyses, part of ChatGPT's web-search answers. And accept a change of unit: on these channels you track share of citation and the quality of inbound enquiries rather than session volume. Traffic from answer engines is structurally lower and markedly further along the decision path — comparing it by volume with organic SEO makes no sense.

Where do you start? The 8-step checklist

Order matters more than completeness: each step conditions the next, and the first three produce most of the result. Here is the sequence we apply to an existing site.

  1. Test the rendering. Page source + Ctrl+F on your key content, across your ten strategic pages. Content missing → absolute priority to SSR/SSG or pre-rendering.
  2. Audit your robots.txt. Separate training bots from index bots, decide according to your model, check that every named bot carries all of its rules (the * group will not apply to it), and check your CDN's policy. Never copy a template found online without reading it line by line — including the one in this article.
  3. Rewrite your section openings. One self-contained answer sentence under each heading, on the pages that matter. Nothing else at this stage.
  4. Turn headings into real questions. Align them with the questions your prospects ask and with “People also ask” in the SERP.
  5. Add the structured data. Article, FAQPage, HowTo on the relevant content; Organization with a stable @id and sameAs across the whole site. Validate with schema.org's Schema Markup Validator rather than Google's rich results test: the latter only checks types that produce a rich result and will stay silent on your Organization or on a badly chained @id.
  6. Align your entity. Name, address, description: strictly identical on the site, Google Business Profile, LinkedIn and directories. Correct the old variants.
  7. Date it, source it, sign it. Visible update date, every figure tied to its source and its year, a named author with a dedicated page.
  8. Install the measurement. AI channel in GA4, generative AI report in Search Console, user-agent counting in the logs, a panel of 20 to 30 prompts recorded monthly.

Worth remembering

  • Accessibility comes before everything. Client-rendered content is invisible to GPTBot and ClaudeBot, however carefully it is written.
  • robots.txt is a three-level trade-off — training, answer index, user visit — not a single switch.
  • Structured data is not a fast pass: Google itself says no special optimisation is required. It removes ambiguity, which is already a lot.
  • llms.txt: free if automatic, never a priority. 97% of these files receive no requests at all.
  • Citability comes down to four signals: visible freshness, named source, attributable figure, identifiable author.
  • Measure across four crossed sources, and change unit: share of citation rather than session volume.
  • And on priority order, if you keep one thing: server rendering first, robots.txt second, the answer sentence third. Nothing else produces anything until those three are settled — that is the only trade-off that counts when the budget is tight.

How METASENSE makes a site readable by AI

At METASENSE this method is not desk research: we apply it to our own pages first. In practice, on meta-sense.fr: the HTML of our articles leaves the server, without depending on a script to exist; our robots.txt is a single User-agent: * group set to Allow: /, which is profile no. 1, deliberately — we want to be cited — and incidentally the only way never to fall into the non-stacking group trap; a single Organization block carries our name and the Vélizy address, strictly identical to the one in our legal notices; every article is signed by a person, dated, and every figure carries its institute, its year and its method. We hold ourselves to what we charge our clients for — and you can check it with the same thirty seconds of Ctrl+U described above.

We will not pretend about what is not yet done: we publish no measured AEO result, because there is not enough hindsight and the measurement tools themselves are three months old. What we do know how to do is hold the join between editorial and technical work — you can rewrite all the content in the world, but if rendering is client-side, nothing comes out; you can have an immaculate technical base, but if no figure is attributed, nobody cites you. It is the same demand for a solid base that produces fast, clean deliveries: the corporate site for CPX Group (international logistics) shipped in 72 hours on a Nuxt + Vercel + headless CMS stack, or -75% production time, and the Webflow rebuild for Gourand SAS in 11 days, with +85% lift in traffic and a redesigned identity. Let us be precise about scope, because it matters: those two figures measure production velocity and a classic SEO traffic gain — they are not AEO results, and we do not present them as such.

METASENSE is a Creative Tech agency based in Vélizy-Villacoublay (13 avenue Morane-Saulnier, 78140). In practice, our work on this subject takes three forms: an AI readability audit (rendering, bots, structure, entity, measurement) that produces a prioritised list; the technical implementation on your existing site or as part of a rebuild; and the editorial framing of the content that needs to become citable. If your challenge also involves direct interaction with your visitors, read our article on the AI agent embedded in a website — the logic there is different and complementary.

Let us make your pages citable

Optimising your site for AI is not one more item in the backlog: it is a way of designing pages, from server rendering to the way a figure is attributed. The good news is that none of it comes at the human reader's expense — content that is accessible, structured, dated and sourced is simply better content.

Rather than a generic diagnosis, we look at your case: what the bots actually see on your strategic pages, what your robots.txt allows, what the answer engines say about you today, and the three actions that will shift the picture fastest.

Explore our web development & design expertise · Get your site's AI readability audited

FAQ — Optimising your site for AI

How do you optimise a site for AI?

By clearing four stages in order: make the content accessible without JavaScript (SSR, SSG or pre-rendering), allow the index bots in robots.txt, structure each section around a direct, self-contained answer, then date, source and sign the information. Measurement comes next, through logs and a prompt panel.

Do AI engines read the JavaScript on my site?

No, not for the most part. The Vercel × MERJ analysis from December 2024 shows that GPTBot (OpenAI) and ClaudeBot (Anthropic) download JavaScript files but do not execute them. Gemini and Copilot are the exceptions, because they inherit the rendering capabilities of Googlebot and Bingbot. Content injected client-side is therefore invisible to most answer engines.

Should you block GPTBot in robots.txt?

It depends on your model. GPTBot collects data for training OpenAI's foundation models: blocking it does not stop you being cited in ChatGPT, because citations run through OAI-SearchBot. If your content is your product, block training and keep the index open. If you are after visibility, allow both. Take care when writing the file: a bot named in its own User-agent group ignores the * group entirely, so all of its rules have to be rewritten for it.

What are the user-agents of AI crawlers?

The main ones are GPTBot, OAI-SearchBot and ChatGPT-User (OpenAI), ClaudeBot, Claude-SearchBot and Claude-User (Anthropic), PerplexityBot (Perplexity), Google-Extended (Gemini applications), Applebot-Extended (Apple) and CCBot (Common Crawl). Training, answer index and user visit are distinct agents, controllable separately.

Is the llms.txt file actually any use?

Very little so far. Across 137,000 domains analysed by Ahrefs, 97% of published llms.txt files had never received a request, no major AI provider has committed to reading it, and Google has confirmed it does not support it. Leave it if your CMS generates it automatically; never prioritise it over server rendering and structure.

Which structured data do you need to be cited by AI?

None is mandatory: Google states that no special optimisation is required to appear in AI Overviews or AI Mode. In practice, install Article or BlogPosting, FAQPage, HowTo where there are steps, and Organization with a stable @id and sameAs links. They remove ambiguity about the author, the date and the publisher.

What is entity coherence, and why is it decisive?

Entity coherence means presenting exactly the same name, the same address and the same description everywhere: site, JSON-LD, Google Business Profile listing, LinkedIn, directories. Generative engines consolidate a representation of your organisation from all of its traces; every variant creates a blurred duplicate that dilutes your citations.

How do you know whether ChatGPT can read your page?

Open the page source (Ctrl+U), not the inspector, and search for a specific sentence from your content. If it is not there, it is injected by JavaScript and stays invisible to OpenAI's crawlers. Then check that OAI-SearchBot is not blocked in your robots.txt or by your CDN.

How do you measure traffic from ChatGPT or Perplexity?

By crossing four sources: GA4's “AI Assistant” channel (since 13 May 2026, not backdated, with Perplexity still in Referral), Search Console's “generative AI features” report (announced on 3 June 2026, with no click data to date), server logs by user-agent, and a panel of 20 to 30 prompts recorded each month.

Is AI traffic visible in Google Analytics?

Only partly. Since May 2026, GA4 isolates an “AI Assistant” channel, but a large share of visits arrive with no referrer, traffic from AI Overviews is counted as Organic Search, and a citation without a click produces no line in analytics. Server logs and a manual prompt record complete the measurement.

Are AI Overviews available in France?

Yes. Google rolled out AI Overviews and AI Mode in France on 22 July 2026, after two years held up by the neighbouring-rights file. The French market is therefore only beginning to absorb the zero-click effect that English-speaking markets have been taking for two years, which leaves a window of advantage to sites preparing now.

How long does it take to see results?

Expect several weeks to a few months. Accessibility fixes (server rendering, robots.txt) produce the fastest effect, from the bots' next pass. Structure and structured data follow at recrawl pace. Entity coherence and authority build over several months.

Should you drop SEO in favour of AEO?

No. SEO remains the base: a page that is unindexable, slow or thin is equally so for answer engines. Optimising for AI sits on top — crawler accessibility, extractability, attribution — without replacing indexability, content quality or domain authority.

Sources

Our expertise

This is exactly what we build.

From advisory to rollout, Metasense designs, develops and delivers these experiences end to end.

Custom websites, applications & platforms

Found this useful? Share it.

One share helps other leaders discover our work.

LinkedInTwitterFacebookEmail