Recommendation Intelligence Research™ · Study #23

The model knows 6 facts about your brand. It uses one.

We asked gpt-4o directly what it knows about 9 brands. It answered confidently every time, certifications, price positioning, specific product features, the kind of detail a brand would want a shopper to hear. Then we checked whether any of that showed up in the moment that actually matters: a real buyer question, no brand named, where the model brought that brand up on its own. Across 1,392 fact checks, 75.8% of the facts a model just claimed to know never appeared. Knowing about a brand and using what it knows are two different things, and most of what gets stored never gets said.

Atom Foundry · September 2026 · Extends the Memory question raised on How AI Decides, reuses the deployment data behind the Recommendation Reports series, part of the Recommendation Intelligence Research™ series

Cite this study: 10.5281/zenodo.22758106

75.8%Facts possessed, never deployed

24.2%Actually used, when mentioned

9Brands tested

1,392Fact-cell checks

Where this comes from

Two questions, asked separately

Every prior study in this series measures one call type: what does the model recommend, under one condition or another. This one measures two call types against each other. First, a possession probe: in a fresh conversation with no purchase context, gpt-4o was asked to list the most distinctive, checkable facts it knows about a brand, its products, its pricing, what sets it apart from named competitors. Run three times per brand and merged down to the claims that showed up, in substance, across at least two of the three runs, so a one-off phrasing doesn't get counted as a stable fact.

Second, deployment: did any of those facts actually show up when the model made a real recommendation? This is where the design gets cheap. The deployment side already existed, 400 real gpt-4o buyer-question responses per brand, collected for the published Recommendation Reports on Bellroy, Caraway, and the rest of the Wave-1 cohort, full response text and a brand-mentioned flag already saved for every one of 3,600 observations across 9 brands. Instead of re-collecting that at AIVO-scale cost, every cell where the brand was actually mentioned was scored against the brand's own possessed-fact list: does this specific fact show up, even paraphrased, in this specific response. A tenth brand, Topicals, was excluded outright, it was mentioned in 0 of its 400 observations, so there is nothing to check deployment against, a complete Candidacy failure rather than a Linkage Gap case.

Model: gpt-4o throughout

Possession probe: 27 new calls, 9 brands × 3 runs

Deployment data: reused, 3,600 already-published observations

Scoring calls: ~280 new calls, one per sampled mentioned cell

Cells scored: 308, capped at 40 per brand

Fact-cell pairs checked: 1,392

Why reuse instead of re-collect. The expensive part of this design, real buyer-question responses across enough categories to be meaningful, was already paid for and already validated by ten published Recommendation Reports. Re-running it from scratch would have meant roughly 3,600 wasted calls for data that already exists. The only new collection is the possession side, and the scoring pass that checks the two against each other.

The finding

Most possessed facts never show up

Across the full cohort, 75.8% of possession-deployment pairs never appear (95% CI 69.9-81.6%, n=1,392). The deployment rate, the flip side of that number, is 24.2%: roughly one fact in four that the model just told us it knew actually made it into a real recommendation where the brand was mentioned. The gap holds up brand by brand, though the size of it varies a lot.

Deployment rate by brand

Share of possessed facts that actually appear, per sampled mentioned cell · capped at 40 cells per brand

Onyx Coffee Lab · n=40 cells

8.8%

Peak Design · n=40 cells

17%

Branch · n=23 cells

20.7%

Bellroy · n=40 cells

22.5%

Rumpl · n=40 cells

23%

Wild One · n=40 cells

27%

Boll & Branch · n=40 cells

35.6%

Caraway · n=40 cells

37.5%

Zigpoll · n=5 cells, wide CI

40%

Too few cells to trust alone

Zigpoll is the one estimate still worth discounting, everything else moved. Zigpoll was recognized in only 5 of 400 buyer-question runs, so its 40% sits on the smallest base in the cohort and is the noisiest single number here. Branch, at 23 mentions, is thinner than the rest but no longer an outlier, its 20.7% lands squarely inside the range of the fully-sampled brands. The seven brands scored on the full 40 cells range from 8.8% (Onyx Coffee Lab) to 37.5% (Caraway), with Boll & Branch the biggest mover in this data, 35.6%, near the top of the cohort rather than near the bottom.

A follow-up question

Do brands that win more also deploy more?

A natural guess: maybe the brands that get recommended most often are also the ones whose facts get used most reliably, winning and fact-deployment could be the same underlying mechanism seen twice. Correlating each brand's deployment rate here against its published recommend rate from the Recommendation Reports series gives r = -0.04 (p = 0.91, n = 9), no relationship at all. At nine brands this is underpowered by design, not a confirmed null, but there is no visible sign that winning more and deploying more of your own facts are the same thing.

That is a genuinely useful negative result. It means a brand cannot assume that simply winning more often will also fix its fact-deployment rate, and a brand that is losing cannot assume its facts are the reason, the mechanisms look separable rather than the same lever measured twice.

Why it matters

Possession was never the bottleneck

This series has already shown, in Candidate Evaluation, that when a specific comparison fact is put directly in front of the model at the moment it is deciding, a better rating flips the outcome 100% of the time. That is the deployment side working exactly as expected, once a fact is placed at the decision point, it gets used. This study shows the other half: a fact the model already has, sitting in memory, unprompted, is used only about one time in four when the moment to use it actually arrives. The bottleneck was never whether the model knows something about a brand. Every brand here returned specific, largely accurate facts the instant it was asked. The bottleneck is whether that fact makes it from memory into the specific sentence being generated at the moment of recommendation.

This lands almost exactly on an unrelated industry measurement of the same phenomenon published elsewhere in 2026 (a comparable, non-overlapping brand cohort, n=592, reported a 75.7% gap rate against this study's 75.8%), which is worth noting as a sanity check on the scale of the effect, not as a claim of shared methodology.

What this doesn't prove

Self-grading is the real risk here

Stating the limits up front. The same model, gpt-4o, runs the possession probe, produces the deployment response, and judges whether a fact shows up in that response. That is a real limitation, not a hypothetical one, a model could in principle be systematically lenient or strict with itself in ways a human or a different model would not be. It was not hypothetical here: an independent read of the judge calls before publishing this page found the first version of the scoring prompt was systematically too strict, it required close-to-exact wording and missed clear paraphrases, most severely for one brand where 26 of 27 "no facts used" verdicts turned out to have a plain-text match the judge had missed (for example, a response that says "GOTS certified organic cotton" scored as zero facts used). All 308 scored cells were re-judged with a corrected prompt, one that spells out what counts as a paraphrase match, before any number on this page was set. The figures above are the corrected pass. This is exactly the kind of error a same-model self-grading setup is prone to, and the fix was a full re-score, not a small sample.

"Six distinctive facts" is a fixed target, not a natural unit, a looser or tighter extraction prompt would shift the denominator and the rate along with it (the actual merged, stable fact count per brand ranged 4-5, not a flat 6). The deployment-side data was collected one to three weeks before the possession probe, reused rather than paired in time, though gpt-4o's behavior is not expected to have shifted materially in that window. Nine brands, one category mix, consumer ecommerce, one model. Cross-platform and cross-model versions of this question are a separate, larger, and currently API-key-blocked follow-up.

Supporting evidence

Two checks that the setup was clean

308 of 308 Re-judged after a scoring bug was found

An independent read of the judge calls before publishing found the original scoring prompt was too strict, missing plain paraphrases (worst case: 26 of 27 "zero facts used" verdicts for one brand had a clear textual match). Every one of the 308 scored cells was re-judged with a corrected prompt. The numbers on this page are the corrected pass.

0 of 400 Topicals, excluded honestly

Topicals was mentioned in 0 of its 400 buyer-question observations, so there was nothing to check deployment against. Rather than drop it silently, it stays out of the cohort by name, a total Candidacy failure is a different mechanism than a Linkage Gap.