Synthetic panels are a gut-check, not a replacement
It would be easy to let the marketing copy drift toward 'skip user research entirely.' Here's exactly what a synthetic panel can and can't tell you, and why we keep saying so even when it costs us a cleaner pitch.
The gap between 'directionally useful' and 'a substitute for real customers' is exactly the gap that gets people into trouble.
There's an obvious, tempting way to sell this product: skip the slow, expensive part of user research and just ask the AI instead. We don't say that, and we're not going to, because it isn't true.
What a synthetic panel actually is: a fast way to pressure-test a question before you spend real time and money finding out the answer the slow way. It can surface the obvious version of an objection in minutes. It cannot tell you whether your actual customers — the ones who exist — will behave the way five generated personas predicted.
What the diversity check proves
The diversity check on every panel measures something real and narrow: does this specific panel disagree with itself the way a real population would, or has it quietly collapsed into one opinion wearing different names.
That's a check on the panel's internal validity — a fully checkable, measurable claim about the tool's own output.
What it can never prove
It is not, and can't be, a check on whether the panel's answers match reality outside the tool. A panel can be genuinely varied and still be wrong about what real people would say, because it's built from a language model's training data, not from your customers.
No amount of internal diversity proves external accuracy — that distinction is the entire point of this page.
Three ways this actually goes wrong
First, treating a synthetic panel's enthusiasm for an idea as validation instead of as one input among several: five generated personas liking your pricing page is not the same signal as five paying customers not churning.
Second, using it to skip the step of talking to real users entirely, rather than to decide which three questions are worth their limited time.
Third, quoting a panel's exact numbers as if they were survey data, when what actually happened is five personas each gave one qualitative answer and someone did the math on five data points.
Every claim we make instead is about something checkable inside the tool itself: whether this panel disagrees with itself, whether its age spread looks like a real population's, whether its answers to your specific question are diverse or collapsed. That's a smaller, less exciting claim than 'replace your research team.' It's also the only one we can actually stand behind.
The honest use case
A cheap way to find out what you don't know yet. Run the panel before you write the real survey, not instead of it. If four personas raise the same objection independently, that's worth taking into the room with real users, framed as a hypothesis, not an answer.
If the panel is unanimous, that's often more suspicious than reassuring: real audiences rarely agree that cleanly, and the methodology page exists partly to help you notice when your panel is being suspiciously agreeable instead of actually diverse.
Common questions about this limitation
Does thepanelist ever claim to replace real user research?
No, explicitly. This page exists specifically to state the line we won't cross, even when a cleaner, more ambitious claim would make for easier marketing copy.
What can the diversity check actually prove?
Only that a specific panel's answers vary the way a real population's might, not that those answers match what real people outside the tool would actually say.
How should I actually use a panel's unanimous answer?
Treat it with more suspicion, not less. Real audiences rarely agree that cleanly — unanimous agreement is often a sign of a collapsed panel rather than genuine consensus.
See what a diversity check actually measures
Read the full methodology behind every panel's internal validity check.
Read the methodologyMore Critical Perspectives
AI market research: how accurate is it really?
Accuracy isn't one number. It's a set of separate, checkable claims — and most of what gets called 'accuracy' in AI research marketing is actually a claim about something else entirely.
Why we said no to fake respondent counts
It would be easy to show '10,000 simulated respondents' on a panel result. We don't, because a panel of five distinct personas and a panel of ten thousand near-identical copies would look the same on that number.
The honest limits of synthetic customer research
A running list of what synthetic research structurally can't do — not caveats buried in fine print, but the actual boundary of the method, stated plainly.