An audit of 50 major websites published this week found that most of them are easy for AI systems to fetch and hard for AI systems to understand. Retrievability scored an average of 74.4%. Attribution and meaning, the layer that tells a machine which number on the page is the price and who is making the claim, averaged 38.5%. The third layer, the one that lets an agent actually do something, averaged 2.1%. That distribution is the most useful map of AI-search work published so far this year, and it points somewhere different from where most of the effort is currently going.
The three layers, and where everybody stalls
Reza Moaiandin of SALT.agency ran the audit and published the findings at Search Engine Journal. All the data was captured on a single day, 12 June 2026, using an instrumented browser that recorded live HTTP responses, the rendered DOM, raw server HTML and machine-discovery endpoints. Twelve established signals were scored, with emerging and frontier protocols tracked but excluded from the scoring.
His summary of the result is worth quoting in full:
Our audit of 50 major websites found that, while most have made it easier for AI to find them, almost none have made it possible for AI to truly understand them. And nearly two-thirds leave the question of which AI bots can access which content entirely to luck.
The layers break down like this.
Layer one, retrievability — can a machine fetch and parse the page. Robots directives for AI user agents, accessibility tree integrity, ARIA labelling, semantic HTML, server-rendered delivery, sitemap declaration. This overlaps almost entirely with conventional technical SEO, which is why sites do well here. Only three of the fifty scored below 50%.
Layer two, attribution and meaning — can a machine tell what the page is about and who owns it. JSON-LD schema and content signals policy. Scores fall off a cliff between the first layer and the second.
Layer three, agent transaction and discovery — can an agent carry out a task on your behalf. Of 48 sites where endpoint testing was possible, 46 scored zero.
The overall mean was 56.6% and the median 58.3%. Airbnb topped the cohort at 79.2%, which is a reminder that nobody has finished this work, including the companies with the largest engineering teams in the sector.
The schema finding, and the part that surprised us
JSON-LD was present on the homepage of 35 of the 50 sites, and all but three of those scored the maximum. We have been writing about structured data since it was HTML5 microdata and rich snippets, and the argument for it has not changed much. What has changed is who is reading it. That leaves nearly a third of major commercial websites with no structured data on their homepage at all in mid-2026.
More striking is the second element in that layer. Only five of the fifty had implemented Cloudflare’s Content Signals Policy, the set of robots.txt directives that spell out separately what a crawler may do for search indexing, for live AI query responses, and for model training. Without it, a site’s AI policy is a binary: block everything or allow everything. Five out of fifty is not a slow adoption curve. It is a signal that most teams have not yet noticed the choice exists.
Does schema get you cited?
Not directly, and the honest version of that answer is more useful than the hopeful one. Loren Baker put it this way in his piece on schema for AI citations:
Schema is far less a ranking switch than a trust builder.
His framing is that four surfaces have to agree: the webpage a person can read, the schema that encodes those same facts for machines, the platform of record (Google Business Profile if you are local, the Merchant Center feed if you sell products), and third-party corroboration in reviews, directories and publications. When those agree, an engine has one reliable reference point. When they disagree, elaborate markup makes things worse rather than better, because now there is a documented contradiction.
The examples he gives are the kind of detail that sounds trivial and is not. Writing “Suite” on the page and “Ste” in the markup is a mismatch. So is a business name, phone number, opening hours, SKU, price, job title or author name that shifts between surfaces with no explanation. On the local side, he flags a confusion we see constantly: on a Google Business Profile, “service area” means where you dispatch or deliver; in schema, areaServed means every area you serve. Different fields, different questions, and reusing generic markup across a multi-location business erases the distinctions that would let an engine recommend the right branch.
There is a commerce version of the same trap. If your schema flips to OutOfStock the moment inventory hits zero, you are telling Google you no longer sell the item, and the ranking you built can go with it even when you restock a few days later. Schema.org has values for temporary states, and using them correctly matters more than adding another dozen properties. The same discipline is what governed rich result eligibility long before answer engines existed.
Should you serve markdown files to AI crawlers?
This is the cargo cult of the moment, and it now has a data point against it. A person on Reddit asked whether anyone had actually seen a major AI bot request a markdown version of a page. Google’s John Mueller answered from his own testing, and Roger Montti wrote it up at Search Engine Journal:
On my test sites the only crawlers who claim to accept markdown are SEO tools. Ymmv.
He added a practical note that is better advice than the headline: server setups commonly do not log the accept header, so if you are curious whether anything on the web wants markdown from your site, work out how to log that first and check the metrics before generating anything.
This sits awkwardly against Cloudflare’s position. Cloudflare, which sells a service converting HTML to markdown on the fly, has written that “markdown has quickly become the lingua franca for agents and AI systems as a whole.” Both statements can be true in their own domain. Markdown genuinely is the standard format for agent instructions and context files, which is why AGENTS.md and Claude’s equivalents exist and work. That usefulness does not appear to extend to crawling, indexing and ranking, where every one of these systems has been reading HTML competently for thirty years.
The deeper reason to be sceptical is the one Montti raises: content served only to machines cannot be trusted by the machines, for the same reason the keyword meta tag died. There is no upside for an AI system in consuming a version of your page that your readers never see.
What about the agent layer?
Layer three is where the scores collapse and where the next two years happen. The two established elements are OAuth discovery and OAuth protected resource metadata, which together let a client application work out what it may access and how to identify itself. Airbnb and Vercel had implemented the first and not the second, scoring 50% each. Everyone else tested at zero.
Behind that sit the emerging protocols. The Model Context Protocol lets an AI query your systems directly rather than reassembling facts from product pages. It is worth building something small with it before forming an opinion; John wrote up connecting a personal knowledge base to Claude over MCP when the protocol was new. On the commerce side, Google’s Universal Commerce Protocol and OpenAI’s Agentic Commerce Protocol handle transactions inside an AI conversation.
UCP is moving quickly enough to be worth tracking if you sell anything. Matt G. Southern covered the latest release: the fourth overall and the first since April, adding grocery features, vendor-neutral 3D Secure 2 authentication, payment schedules with deposits and instalments, and split payments across methods. It carries breaking changes that require schema updates, so anyone who has already adopted it needs to read the release notes rather than assume a clean upgrade. A Food Technical Council was formed in July with Block, DoorDash, Google, Toast and Uber Eats; a Lodging Technical Council followed on 11 August with Amadeus, Booking.com, Expedia, Google, Hilton, Marriott and Trip.com. Stripe joined the Governing Council in April alongside permanent members Google and Shopify.
None of that means grocery ordering or hotel booking works through an AI interface today. It means the specifications are being written now, by the platforms that already run those industries.
A sensible order of work
The audit implies a sequence, and it is not the sequence most sites are following.
Finish layer one properly rather than partially. Server-rendered HTML, clean document hierarchy, a sitemap that is actually declared, and explicit robots directives for named AI user agents. Most of this is development work rather than marketing work, which is part of why it stalls. The last of those is the two-thirds-left-to-luck problem, and it is an afternoon of work.
Then reconcile the four surfaces. Before adding schema types, make the facts agree across the page, the markup, the platform of record and whatever third parties say about you. A contradiction repeated in JSON-LD is worse than no JSON-LD.
Add the content signals policy. It costs nothing, it lives in robots.txt, and it replaces a binary decision with a considered one.
Log your accept headers before building anything for bots. Mueller’s advice generalises. Measure what is actually asking for what, then decide.
Watch layer three; do not bet the quarter on it. There is a genuine first-mover advantage available here, and there is also a specification in its fourth revision with breaking changes. Both are true.
Frequently asked questions
Does structured data help with AI Overviews and AI Mode?
Google’s guidance is that structured data is not necessary for AI features but is still recommended as part of a broader SEO strategy. Microsoft has said schema helps its models understand content, and OpenAI has pointed to structured product data feeding what ChatGPT shows shoppers. It is corroboration rather than a switch.
Is it worth generating markdown versions of my pages for LLMs?
There is currently no evidence it improves AI citations or visibility. Google’s John Mueller found that on his test sites, the only crawlers claiming to accept markdown were SEO tools. Log your accept headers and check before investing.
What is the Content Signals Policy?
A set of robots.txt directives from Cloudflare that separate what a crawler may do for search indexing, for live AI answers, and for model training. Five of the fifty audited sites had implemented it.
What is UCP and do I need it?
The Universal Commerce Protocol is the specification behind agent-driven checkout in AI Mode and Gemini, governed by Google, Shopify and Stripe. Retailers already using it should read the breaking changes in the latest release. Everyone else can watch.
The short version
Fetchability is largely solved and comprehension is not. The gap between 74.4% and 38.5% in that audit is where the available advantage sits right now, and closing it is mostly a matter of making the facts on your page, in your markup, on your business profile and in third-party sources say the same thing. That work is unglamorous, it is checkable, and almost nobody has done it. Once it is done, Search Console’s AI performance report is where you watch it take effect.
