thepanelist
Operational Playbooks
EXECUTION PROTOCOL
Use Cases

Feature prioritization between sprints

When the backlog has five plausible next features and no clear winner, run a trade-off test against a panel grounded in your actual users to see which one a panel reaches for — and how strongly.

Protocol SpecificationsVerified
Run Speed
~3 Minutes
Panel Depth
5–8 Personas
Audit Mode
Entropy 0.9+
Input Asset
Pitch / Brief
Deploy this panel in /app
Phase 01 // The Strategic Blindspot

Five plausible features. One sprint. A directional tiebreaker.

Diagnostic Note 01

Most feature-prioritization debates aren't between one obviously right answer and four obviously wrong ones — they're between several plausible options, each defensible, with the team genuinely split.

Diagnostic Note 02

thepanelist's trade-off test is built for exactly that moment: describe the candidate features, generate a panel grounded in your actual user base, and rank the options directly — with a measure of how strongly the panel actually agreed on the winner, not just which one scored highest.

Phase 02 // Risk vs. Verification MatrixDecision Audit
Intuition Exposure
Unchecked Risk

Without a tiebreaker

Prioritization debates without outside input tend to resolve toward whoever argues loudest, or toward the feature that's easiest to build rather than the one users would value most.

A decision made that way can feel resolved internally while still being disconnected from what the actual user base would have picked.

Synthetic Protocol
Validated Process

With thepanelist

Describe each candidate feature briefly, generate a panel grounded in your user base, and run a trade-off test that ranks them directly against each other.

The result includes how much the panel actually agreed on the top choice — a landslide and a narrow split both produce a "winner," but they mean different things for how confidently you should act on it.

Phase 04 // Empirical Metric Benchmark
Standardized SLA
Kendall's Tau
the statistic thepanelist uses to measure how much a panel actually agreed on a ranking

A clear top-ranked feature and a panel that genuinely agrees with each other are two different claims — this is the number that tells you which one you actually got.

Ready to run this playbook on your own concept?Initialize Panel Session
Phase 03 // Execution Playbook Rules

How to run a useful prioritization test

#1

Describe each candidate feature the way you'd describe it to a real user, not the way you'd describe it internally — "lets you export your data as CSV" reads more clearly to a panel than an internal ticket title does.

#2

Ground the panel in your actual user base as specifically as you can, not a generic description of your market — the ranking is only as relevant as how closely the panel resembles the people who'll actually use the feature.

#3

Read the agreement score alongside the ranking, not instead of it. A narrow-margin winner with low panel agreement is a genuinely different signal than a landslide with high agreement, even if both technically produce the same top pick.

Feature prioritization between sprints — thepanelist