The honest limits of synthetic customer research
A running list of what synthetic research structurally can't do — not caveats buried in fine print, but the actual boundary of the method, stated plainly.
Every method has a boundary. Most vendors put theirs in a footnote. We're putting ours on its own page.
Synthetic customer research, done honestly, is useful for a specific, narrower set of jobs than the pitch for it often implies. This page is a direct accounting of where that boundary actually sits — not to undersell the method, but because pretending the boundary doesn't exist is how people get burned by it.
It can't surprise you the way a real person can
A synthetic persona's answer is drawn from patterns in a language model's training data plus whatever material you grounded it in. It can produce a plausible-sounding answer you didn't expect, but it can't produce genuine surprise the way a real customer occasionally does — a reaction that comes from lived experience the model never had access to.
It can't validate a number with statistical confidence
A five-persona panel agreeing on something is a directional signal, not statistical proof at any meaningful confidence level. Cochran's formula is explicit about how many real respondents a genuinely precise result requires, and it's almost always more than a synthetic panel's practical size.
It can't replace a specific real conversation
If you need to understand one particular customer's specific frustration with your product, a synthetic panel modeling a general persona type can't substitute for actually talking to that customer.
It can replace a guess
What it reliably beats is the default alternative for most early-stage decisions: no read at all, made on pure instinct. A directional signal beats a guess, even when it doesn't beat a real study.
These aren't edge cases — they're the core, structural limits of what this category of tool can do, true for any synthetic panel product, not a weakness specific to thepanelist.
Common questions about these limits
Does naming these limits mean thepanelist is less useful than it claims?
It means thepanelist is exactly as useful as it claims, because the claims are scoped to fit inside these limits, not stretched past them.
Should these limits stop me from using synthetic research?
No — they should shape what you use it for: a fast directional filter before a real decision, not a substitute for real customer conversations or statistically validated studies.
Are these limits specific to thepanelist, or true of the whole category?
They're true of synthetic customer research generally, not a thepanelist-specific weakness. Any tool claiming otherwise is overstating what the method can do.
See what a directional read is actually good for
A fast filter before a real decision — read the methodology behind it.
Read the methodologyMore Critical Perspectives
Synthetic panels are a gut-check, not a replacement
It would be easy to let the marketing copy drift toward 'skip user research entirely.' Here's exactly what a synthetic panel can and can't tell you, and why we keep saying so even when it costs us a cleaner pitch.
AI market research: how accurate is it really?
Accuracy isn't one number. It's a set of separate, checkable claims — and most of what gets called 'accuracy' in AI research marketing is actually a claim about something else entirely.
Why we said no to fake respondent counts
It would be easy to show '10,000 simulated respondents' on a panel result. We don't, because a panel of five distinct personas and a panel of ten thousand near-identical copies would look the same on that number.