Recommendation Intelligence Research™ · Study #51

Schema.org JSON-LD on its own made AI repeat fewer facts than plain text. JSON with neutral field names did not.

“Add schema markup and AI will cite you more” is one of the most repeated tips in AI-visibility guides. We had never tested it; our earlier study compared bullets with prose. Here a model got the same three facts about a brand in five forms: plain prose, JSON-LD only, prose plus JSON-LD, JSON with neutral field names, and prose twice. We counted how many of the facts its answer repeated. JSON-LD on its own was repeated less than prose, on gpt-4o and again on gpt-4o-mini. The same JSON structure with neutral field names was not. JSON-LD next to prose changed nothing. Part of the drop depends on which schema property holds each fact.

50.6%JSON-LD only
64.4%plain prose
67.0%JSON, neutral field names
67.3%prose + JSON-LD
Where this comes from

Everyone recommends schema. We had no data of our own.

Adding structured data is among the most common recommendations in AI-visibility checklists and vendor guides. Structured Markup (Study #38) tested something close to it: the same facts as bullets instead of prose, where bullets cost 5.2 points of citation across two rounds. It did not test real schema.org JSON-LD, the markup the advice is actually about.

We wrote the predictions and contrasts down before the data came in, ran the study twice with independent seeds, and then added a second part, also written down in advance: does the result depend on which schema property holds each fact, and does it hold on a second model?

The test

Same three facts, five forms

Four brands, 20 shopping intents each. For every brand the model saw three facts in its system message: what the brand does, a mention in a named publication, and a statement that it is one of the best-known brands in its category. The wording and the order of the facts were identical in every form. The user asked an open question (“I’m looking for… What would you recommend and why?”) and we counted how many of the three facts the answer repeated.

Model: gpt-4o (replicated on gpt-4o-mini), temperature 0.9
Forms: plain prose; prose + JSON-LD; JSON-LD only; JSON with neutral field names; prose twice
Brands and intents: 4 brands × 20 intents (80 clusters)
Scoring: a GPT-4o judge per fact, plus checks that need no judge (word overlap, distinctive terms, publication name)
Calls per round: 2,000 answers and 6,000 judge calls, two independent rounds
Part 2: four property mappings (gpt-4o, 2,400 answers per round) and gpt-4o-mini (2,000 per round)
Statistics: cluster bootstrap and permutation tests over brand × intent, Bonferroni-corrected
Rule: a finding needs the same direction and significance in both rounds
Pre-registration. Hypotheses, contrasts, and the rule for a finding were written before each part ran; the script and brand files were fingerprinted (SHA-256) at the start of every round. Part 2 was written after Part 1 and before any Part 2 data. The raw answers and judge verdicts are published with the study.
The result: JSON-LD against prose

JSON-LD alone was repeated less than prose

As JSON-LD only, the model repeated 50.6% of the facts, against 64.4% from plain prose: −15.0 points in round 1 (95% CI −17.7 to −12.3) and −12.6 in round 2 (−15.4 to −9.8), p = 0.0001 in both. All four brands moved the same way, between −7.0 and −22.3 points. The same JSON structure with neutral field names (brand, fact_1, fact_2, fact_3) did not hurt: 67.0%, +1.8 and +3.3 points against prose, neither significant. So the drop is not about JSON syntax or length; JSON-LD with its schema.org vocabulary scored −16.8 and −15.9 points below the neutral version.

When the prose was there as well, JSON-LD did no harm and no good: prose plus JSON-LD scored 67.3%, and compared with the same prose shown twice (68.1%, which controls for the facts appearing twice) the difference was −2.1 and +0.4 points, inside ±5 in both rounds.

Share of the three facts repeated in the answer
gpt-4o · mean of two rounds · 2,000 answers per round
JSON-LD only
50.6%
Plain prose
64.4%
JSON, neutral field names
67.0%
Prose + JSON-LD
67.3%
Prose twice
68.1%
Not an artifact of the judge or of answer length. Measures that need no judge point the same way: word overlap with the facts was 9.2 and 7.7 points lower for JSON-LD only than for prose, and distinctive terms 6.6 and 3.8 points lower. Answers were shorter with JSON-LD only (129 words against 145 for prose), but neutral JSON was shorter still (112) and did not drop.
The result: what drops

The reputation claims fall most. The main description barely moves.

Split by fact, the loss is not even. What the brand does changed little (86.4% from prose, 83.6% from JSON-LD). The two claims about reputation fell most: “featured in a named publication” from 71.4% to 49.4%, and “one of the most recognized brands” from 35.5% to 18.9%. A check that needs no judge, whether the answer contains the publication’s name, shows the same picture: 78.1% from prose, 59.0% from JSON-LD only, 79.4% from neutral JSON.

Fact (schema property in the JSON-LD)ProseJSON-LD onlyNeutral JSON
What the brand does (description)86.4%83.6%93.8%
Featured in a named publication (subjectOf → headline)71.4%49.4%69.6%
One of the best-known brands (disambiguatingDescription)35.5%18.9%37.6%
Share of answers in which the judge found each fact, mean of two rounds, gpt-4o. The mapping of facts to schema properties was fixed before any data (description, subjectOf headline, disambiguatingDescription).
The result: is it the markup or where the facts sit?

Part of the drop depends on the property

The drop was concentrated in the two facts we had put into secondary schema properties. So Part 2 moved them. With the original mapping, JSON-LD was again −13.9 and −13.2 points below prose. With all three facts in a single property (description), the gap shrank to −7.3 and −5.2 points, still significant, and +6.7 and +8.0 points better than the original mapping. Moving only the reputation claim to additionalProperty recovered about 16.1 points on that fact (+16.8 and +15.5); moving the publication mention to award helped only partly (+5.3, not significant in round 1, and +10.0).

The pattern: facts in the main description are used the most, and the more metadata-like the property, the less the model repeats what is in it. The property mapping explains roughly half of the drop; the schema wrapper itself explains the rest. This is a pattern in the numbers, not a mechanism we have verified.

Share of the three facts repeated, by how the facts are written
gpt-4o · Part 2 · mean of two rounds · 2,400 answers per round
JSON, neutral field names
68.3%
Plain prose
63.0%
JSON-LD, all facts in one property
56.8%
JSON-LD, reputation claim moved
54.3%
JSON-LD, publication mention moved
51.0%
JSON-LD, original mapping
49.5%
The result: a second model

The same pattern on gpt-4o-mini

Part 2 repeated the design on gpt-4o-mini. JSON-LD only: 48.5% against 59.2% for prose (−11.3 and −10.3 points) and 77.1% for neutral JSON (−28.3 and −29.0). The reputation claim fell most again (−18.8 and −19.8 points). The comparison of prose plus JSON-LD against prose twice gave no consistent result (−3.7 and +1.6). One result we did not predict and cannot explain: on gpt-4o-mini, neutral JSON was repeated more than prose (+17.0 and +18.8 points); on gpt-4o it was not.

All confirmatory tests

Every test, both rounds

Difference in share of facts repeated (points)Round 1 (95% CI)Round 2 (95% CI)Result
gpt-4o: JSON-LD only − prose−15.0 (−17.7 to −12.3)−12.6 (−15.4 to −9.8)Finding
gpt-4o: JSON-LD only − neutral JSON−16.8 (−21.8 to −12.2)−15.9 (−21.0 to −11.1)Finding
gpt-4o: prose + JSON-LD − prose twice−2.1 (−4.9 to +0.7)+0.4 (−2.4 to +3.4)No effect beyond ±5
Part 2, gpt-4o: JSON-LD (original) − prose−13.9 (−16.9 to −11.0)−13.2 (−16.7 to −9.8)Finding
Part 2, gpt-4o: all facts in one property − prose−7.3 (−10.3 to −4.4)−5.2 (−8.2 to −2.2)Finding
Part 2, gpt-4o: one property − original mapping+6.7 (+3.8 to +9.4)+8.0 (+5.0 to +11.1)Finding
gpt-4o-mini: JSON-LD only − prose−11.3 (−14.7 to −7.8)−10.3 (−13.6 to −7.0)Finding
gpt-4o-mini: JSON-LD only − neutral JSON−28.3 (−33.4 to −23.3)−29.0 (−34.4 to −23.9)Finding
gpt-4o-mini: prose + JSON-LD − prose twice−3.7 (−7.5 to −0.2)+1.6 (−1.0 to +4.1)Inconclusive
“Finding” means the same direction and a corrected p below the threshold (0.0125 in Part 1, 0.0167 in Part 2) in both rounds. “No effect beyond ±5” means the 95% interval stayed inside ±5 points in both rounds. A head-to-head test of which brand the model picks was also planned; it is not reported because it could not discriminate (see limitations).
How far to trust the judge

The judge was checked by two humans

A GPT-4o judge decided, fact by fact, whether the answer repeated it. We checked it blind three times. Claude re-scored 100 items per round: agreement with the judge was 89% and 91%. A first human rater, the study author, scored 42 items: agreement was 67%, below the 80% we had set in advance, because the rater was more lenient than the judge. We then ran a second audit, with the rules written down before any label existed: an independent human rater scored 40 new items (20 prose, 20 JSON-LD only) after a stricter instruction that a related but different fact does not count. Agreement with the judge was 92.5% (kappa 0.84), above the 80% bar.

On the independent rater’s own labels, the claim was found in 85% of prose answers and 30% of JSON-LD-only answers. The judge gave 85% and 45%. So the judge, if anything, was more generous to JSON-LD than the person was; the measured gap is not an artifact of a strict judge. Each cell has only 20 items, and the study also reports measures that need no judge (word overlap, distinctive terms, the publication name), which point the same way.

Why it matters

Say it in text. Markup is not a substitute.

On this evidence, adding JSON-LD to a page does not make a model repeat your facts more. When the markup is the only place a fact appears, it is repeated less than the same fact in plain text, and claims about reputation suffer most. Next to well-written text, markup did no harm and no good. For a store, the practical reading is narrow but clear: do not count on structured data to carry the claims you want a model to repeat; put them in the visible copy as well.

What we did not test is whether the systems that fetch your pages pass your markup to the model at all, or whether entity schema helps a system recognise who you are. Those are different questions from the one measured here.

Limitations

What this does not establish

Markup in the prompt not on a page

The JSON-LD was placed as text in the system message. This measures what the model does when it sees markup, not whether ChatGPT, Google or an agent passes the markup of a fetched page to the model, or drops it first.

Repeating is not recommending

We measured how many facts the answer repeats. A head-to-head test of which brand the model picks was run too, but the competitor was too weak and the target won about 100% of the time in every form, so it says nothing about markup. It needs a separate study.

Four brands one vendor

Four brands, two models from the same vendor, no web search. The size of the effect varies by brand (−7.0 to −22.3 points in Part 1). Other models, real pages and live retrieval may behave differently.

Our mapping not the only one

Which fact sits in which schema property was our choice, and Part 2 shows it matters. Other properties, other schema types and richer markup were not tested. Judge agreement with a person was below our target (see above).

Do AI models read what your page says?

Free AI Commerce Score™ in 10 seconds.

See whether your key claims are visible as plain text AI models can read, not only as markup, before you spend on awareness.

Free · No signup · Results in 10 seconds
Explore the series

More mechanism studies

The study this one builds on and sits next to.