Our earlier lock-in work measured brands. A reader asked whether the stability also holds at the level of the specific product, the thing a customer actually buys. We asked 50 shopping questions 60 times each, across two sweeps a week apart, and let the model name one brand and one product. Brand answers were the most stable. Specific products were less stable, and the gap was largest where product identity was judged strictly. When two calls named the same brand, more than a quarter still named a different variant.
Recommendation Lock-In showed that for many shopping questions a model returns the same brand again and again. A reader on LinkedIn asked the natural follow-up: is the stability visible at the level of the SKU, or only the brand? We had only measured brands, and said so. Our untested guess was that the product would be less stable.
We wrote the primary test down before running: the difference between brand stability and product stability, per intent, with a bootstrap interval over intents and a permutation test. The guess held, and the direction was the same in every intent where the two differed.
Each of 50 shopping intents (ten categories, five per category, 16 broad and 34 narrow) was asked 30 times in sweep A and 30 times in sweep B, seven days later. The prompt asked for one specific product and returned a brand and a product name. No web search. Product names come from the model's memory, so some may be real variants and some plausible ones.
Across the 50 intents, the most common brand made up 80.9% of calls. The most common specific variant made up 66.7%, a gap of 14.2 points (95% CI 9.8 to 18.8, p < 0.001). Judged by product line the gap was 10.5 points; judged by exact name it was 28.8. In 35 of 50 intents the brand share was higher than the product share, in 15 they were equal, and in none was the product more stable.
Pairwise agreement asks: if two calls are drawn at random for the same question, how often do they give the same answer? For brands, 73.3%. For specific variants, 53.9% (difference 19.4 points, CI 14.1 to 25.1, p < 0.001). Pairs drawn from different sweeps agreed just as often as pairs inside one sweep (brand 73.3%, variant 53.9%), so the pattern held a week later.
Among pairs of calls that named the same brand, 26.3% named a different specific variant (CI 20.2 to 32.6). Judged by product line the rotation is 18.4%, and by exact name 50.3%. Lock-in on a brand does not mean lock-in on a SKU: the model tends to settle on who, less on which.
Few intents are fully locked. 46% of intents had a most common brand at 90% or more of calls. For the most common specific variant, 22% did (12% by exact name, 28% by product line). The gap between brand and product was similar for broad intents (15.3 points, CI 6.7 to 25.5) and narrow ones (13.6 points, CI 8.9 to 18.8).
| Metric | Estimate | 95% CI (bootstrap over intents) |
|---|---|---|
| Most common brand | 80.9% | 75.5–85.9% |
| Most common product, exact name | 52.0% | 44.8–59.4% |
| Most common product, product line | 70.3% | 64.1–76.5% |
| Most common product, specific variant | 66.7% | 60.8–72.6% |
| Gap brand minus variant (modal share) | 14.2 points | 9.8–18.8 |
| Gap brand minus line (modal share) | 10.5 points | 5.8–16.0 |
| Gap brand minus exact name (modal share) | 28.8 points | 22.6–35.5 |
| Two calls, same brand | 73.3% | 66.5–79.7% |
| Two calls, same specific variant | 53.9% | 46.9–61.1% |
| Two calls, same product line | 59.2% | 52.0–66.6% |
| Two calls, same exact name | 37.7% | 29.9–45.9% |
| Gap brand minus variant (pair agreement) | 19.4 points | 14.1–25.1 |
| Same brand, different variant (rotation) | 26.3% | 20.2–32.6% |
| Same brand, different line (rotation) | 18.4% | 12.4–25.1% |
| Same brand, different exact name (rotation) | 50.3% | 42.3–58.3% |
| Cross-sweep agreement, brand | 73.3% | 66.5–79.7% |
| Cross-sweep agreement, specific variant | 53.9% | 46.9–61.2% |
When a model keeps returning your brand, that does not mean it keeps returning the same listing. In this setting, about a quarter of repeated brand picks rotated to a different variant, and only one intent in five had a variant that held at 90% of calls. For a store with several products in one category, the practical question is which of them the model has a stable association with, not only whether the brand is in the answer.
This extends what lock-in studies measured at brand level. It also fits the picture from mechanism studies: the model's decision is shaped by what it knows about the brand, while the choice of a specific item inside it looks softer.
Product names come from the model's memory with no retrieval. Some named variants may not exist as sold. Real shopping agents with live product data may be steadier or less steady at the SKU level.
Whether two names are one product depends on the definition. We report strict, line and variant, and the direction holds at all three, but the size of the gap does not.
Single model, one prompt wording, temperature 0.7. Other models, wordings and temperatures may differ.
Sweep B repeats the same question a week later. It tests repeatability, not changes in a real catalog. The effective sample is 50 intents.
See how clearly your store and products are presented to AI models before you spend on awareness.
The study this one builds on and the series it sits in.
Illustrative example · single-site signal for atomfoundry.dev.
View full signals →