Recommendation Intelligence Research™ · Study #6

Every store we tested was already in the game. We never checked who doesn't make it.

Every correlation in this series so far, including the ones that found nothing, was measured only among brands the model already recommends. That is a range-restricted sample. We had never asked whether a better store gets recommended at all. So we tested the full population: 599 stores that appeared in at least one recommendation against 60,325 that never appeared in any. One factor separates the two groups. It still predicts nothing about who wins once you are in it.

Atom Foundry · July 2026 · Part six of the Recommendation Intelligence Research™ series

Cite this study: 10.5281/zenodo.22772390

60,924Stores, full population

599Ever recommended

0 vs 0Total score, recommended vs never

+0%Intent gap, the one real separator

Where this comes from

The catch: we only ever looked at brands already in the room

Every prior test that correlated store quality with recommendation frequency ran the comparison among brands the model already recommends. That answers whether a better store, once it is being recommended, gets recommended more often. It never asks the more basic question: does a better store get recommended in the first place, or does it never even enter consideration?

Those are different questions with different samples. Restricting to brands already recommended can hide a real effect that only shows up at the gate, before frequency ever becomes relevant. So we dropped the restriction and compared the full population of scanned stores: everyone who ever appeared in a recommendation against everyone who never did.

The test

599 recommended. 60,325 never.

We split every scanned store into two groups based on a single question: did this store appear in at least one AI recommendation, ever, or not. Then we compared the AI Commerce Score and each of its components across the full population, no restriction to brands already winning recommendations.

Total stores scanned: 60,924

Recommended at least once: 599

Never recommended: 60,325

Compared: total score and 7 score components

Intent robustness check: 9 of 9 niches

Frequency correlation: r = 0.112, R² = 1.2%

What we are not reporting. An earlier pass also compared raw review counts and average price between the two groups. We are leaving both out. A later data-quality check found review count populated for only 9.4% of the 66,085 scanned stores and average price for 66.7%, which means those two comparisons were mostly comparing empty fields, not real values. Only the score components below are computed for every scanned store regardless of raw-field coverage, so only those are reported here.

The result

Recommended stores are not better. They are marginally worse

Across the full population, recommended stores scored 55.4 on average against 56.0 for stores that were never recommended. That is a real number now, not a restricted-sample artifact. The 0.7% correlation this series already reported was measured among recommended brands only. This test removes that restriction entirely, across 60,924 stores, and the gap still does not appear. If anything it points the wrong way.

Score components, recommended vs never recommended

5.13

3.70

Intent

7.43

7.22

Visual

3.83

3.53

Schema

11.27

12.13

Technical

10.42

11.06

Trust

8.25

8.94

Price

3.07

3.14

Brand

Recommended (599) Never recommended (60,325)

Recommended stores score higher on intent, visual, and schema, and lower on technical, trust, and price. Same total either way. Intent is the only gap large enough to matter, roughly 39% higher for recommended stores. The rest move by single-digit percentages.

Testing the one real gap

Intent holds up under every check we ran

A single gap this size could easily be an artifact of category mix, circular scoring, or which brands happen to own their own store. We tested for all three, plus whether the gap actually predicts anything once a store is in the recommended set.

9 of 9

Niches show the gap, range +0.78 to +1.59 points. Not a category artifact.

0.20 to 0.29

Intent's correlation with the store's other score factors. A free-floating brand-recognition signal would not correlate with anything.

1.40 vs 1.53

Gap size in brand-owned stores versus independent ones, nearly identical. Not a brand-selection artifact.

R² = 1.2%

Once a store is in the recommended set, intent does not predict how often it gets picked, r = 0.112. Same magnitude as the fame study, on the null.

Why it matters

Store quality gets you candidacy, not selection

One store factor genuinely separates the brands that ever get recommended from the brands that never do, and it survives niche breakdown, circularity checks, and a brand-ownership split. That is real. But the same factor, measured only among brands already in the recommended set, predicts almost nothing about which of them wins more often. Getting into the game and winning it are governed by different things.

One confound is worth naming directly: 3.70% of Magento stores appear in the recommended group against 0.79% of Shopify stores, a 4.7x difference. That looks dramatic until you notice Magento stores in this dataset skew older and larger, the platform itself is not doing the work. It is a marker for something else, not a lever anyone can pull.

What correlation can never settle: direction. Brands with sharply positioned stores may be recommended because their positioning is clear, or their positioning may be clear because they are already successful enough to invest in it. Both stories produce the same correlation. Nothing in this data, or any observational data, can tell them apart.

A closer look, two brands

Two brands, opposite anomalies

Reusing already-published data from four other studies in this series (Possession vs Deployment, Multi-turn Displacement, Fact Injection, Hidden Context, same Wave-1 cohort, zero new calls), two brands sit at completely different ends of the Candidacy vs Selection split. Brand names anonymized here, same convention as Multi-turn Displacement.

10% to 100%

Brand J, overall recommend rate 1.2% across 400 real buyer-question observations, wins only 10% of head-to-head Hidden Context tests with no added context. Give it any exposure condition at all and it wins 100% of the time, every condition tested. The bottleneck is entirely getting into the room.

1.25% to 97.5%

A single verified fact injected into context lifts Brand J's mention rate from 1.25% to 97.5%, a 96-point swing, the largest of any brand in Fact Injection. Nothing else about the brand changed. Same story as the card to the left, from a different study.

100% then 5%

Brand H reaches full candidacy just as easily as Brand J under the same supportive conditions. But in multi-turn conversation, explicitly recommended at turn one, it survives to the final pick only 5% of the time, displaced by the same single competitor in 18 of 20 runs.

100% but only 0-53%

Even guaranteed a seat in the set, Brand H's Hidden Context winner rate ranges from 0% (brand exposure) to 53% (campaign origin), nowhere near Brand J's 100% across every condition. Getting into the room does not fix Brand H's problem. Something in the specific matchup does.

This is not the same correlation problem as above. Hidden Context and Fact Injection are controlled manipulations, not observed correlations, so for these two brands specifically the Candidacy/Selection split is more than a pattern in the data, it is a result of actually changing the input and watching the outcome move. Brand J's bottleneck is Candidacy. Brand H's bottleneck is Selection. The same fix, adding context, resolves one almost completely and barely moves the other, which is a real, actionable difference in what each brand would need.