Shreyas Doshi's Guide to Validating & Prioritizing Ideas
In April 2021, a Product Leader from Stripe named Shreyas Doshi published a story called “Destined to Fail: A B2B Product Management story”. It was the first time I came across the idea of Customer Problem Stack Ranking as a way to think about customer discovery or idea validation — needless to say, it changed the way I’ve thought about building products ever since. Original artwork shown below was produced by Shaun Miller.
I hope you find it as impactful as I did!
Daniel
Once upon a time, there was an idea for a new product.
The Product Manager (or founder) diligently goes out and talks to customers to understand whether this product will solve their problems. And customers say yes!
The PM reports her findings to the executive team and everyone gets very excited. Staffing and budget are successfully obtained.
Soon after, following all the best Lean Startup and Product Management principles, they ship the first version of the new product.
But hardly any customers adopt it…
At the first Product Review post-launch, the PM directs attention toward the positives. Her learnings include things like “Our MVP isn’t sufficient,” “We need to make it easier to implement/adopt” and — most commonly — “We just need to build features X, Y and Z.”
The team supports the PM and she gets her mandate to build those features.
Doing all of that takes a lot of extra time…
So fast forward a couple of quarters and, eventually, the second version gets shipped !!! 🎉
Yet again, product adoption is anemic.
At the next product review meeting, sales and marketing start getting the blame pointed at them. The PM says something like “We know from talking to customers that we have the right product. We just need to improve our Go-To-Market strategy.”
The team is “pot-committed” at this point and it feels like there’s no turning back. Ideas about how to better sell the product are discussed: reduce prices, cross-sell, bundle, email campaigns, re-organize the Sales team…
Changes are made. Expectations are set.
Yet growth remains flat and lifeless.
By the time this point is reached, the original PM has already left the team. A new PM joins and starts with a “customer listening tour” for their first 90 days. They identify some additional issues, present new findings and recommendations to the executive team, and get a fresh mandate to execute the revised plan.
This can go on and on with new product teams, revised strategies, more rounds of discovery interviews…
Ultimately, projects like this eventually get “sunset”. Learnings are captured and shared widely in the company. The team tells themselves "We haven't failed, we have learned." An Edison quote gets shared at some point.
So, what really happened here?
There are many possible reasons for this saga, but the most common ones:
A. The product should not have been built in the first place
B. The original product was ill-conceived & the later pivots had to inherit this original error
Let's look at Option A 👇
The product solved a problem, but not the problem.
Often, product teams aim to be in the top right of this quadrant → executing to the highest quality an idea that solves a problem for someone.
But, this model assumes that all problems are equally important. In reality, there’s a third axis → the importance of the problem.
And the best products tackle the problems that matter most (and do it well!)
Daniel Kahneman famously described The Focusing Illusion, which says that “Nothing in life is as important as you think it is, while you are thinking about it.”
For people in business, this quote could easily be rewritten as “Nothing in business is as important as it actually is, while you are talking about it.”
When you talk to a customer about a specific problem, they will naturally “focus” on that problem — at the exclusion of other problems they (or their team/business) are facing.
Your focus brings a disproportionate emphasis to the discussion on solving THAT specific problem. Hence the customer’s initial excitement about your possible product solution. Hence the tepid response and adoption once you’ve built said product.
If you were to zoom out and look at this problem in comparison to all of the other problems that this customer is currently dealing with, you might quickly realize that what you thought of as a “hair-on-fire problem” is something they see as just a “mild inconvenience”.
And nobody goes out of their way to find solutions for a mild inconvenience.
This approach is called “Customer Problems Stack Rank” (CPSR). You ask the customer to stack rank the problem versus all the other problems they are trying to solve for their business and org.
This way of thinking can be applied across functions — you should also get the CPSR from other personas involved, like the VP Support, VP Marketing, VP Sales, etc.
You are now closer to truth.
When I first read Destined to Fail, I knew immediately that Shreyas was describing my EXACT circumstances. We had launched 6 months earlier and that same week had lost our only subscription customer. Things were looking pretty bleak. We had conducted about 150 interviews by that point to figure out what problem to solve, but clearly it just wasn’t working.
We decided to run one last experiment before giving up, where we would ask a bunch of our target customers to stack rank a list of problems so we could see where the key problem we were focused on solving ranked in their list of priorities.
In just one evening, we drafted a list of problem statements based on our customer discovery interviews, shared them with 600 target customers, and could already see that our problem statement was ranking dead last out of 45 options. But the biggest surprise for us was that 5 of the top 7 highest-ranking problems were actually very similar to the one we were trying to solve!
So the next morning we rewrote our landing page and product onboarding flow to match these 5 high-ranking problems instead. Within a week of starting this stack ranking experiment, we had multiple paying customers, our landing page was converting to trial 3x better than before, and we even had our first customer testimonials!
To learn more about this “customer problem stack ranking” experiment, check out the short video case study below or read this viral blog post explaining the process we followed.
For more about using stack ranking in your own research, check out these guides:
Discovery Sandwich: How The Best Product Teams Use Mixed Methods Research
Pairwise Comparison: A Robust Method For Running Stack Ranking Surveys At Scale
Ultimate Guide to Needs-Based Segmentation Using Stack Ranked Data
Value-Adoption Matrix: How To Find Untapped Opportunities In Your Existing Features
If you’re looking for a tool to run your stack ranking experiments, check out OpinionX. We built it specifically to help teams run stack ranking surveys to understand what matters most to their customers, colleagues or communities (it’s free to use!).
If you liked this post, consider subscribing to The Full-Stack Researcher — our newsletter that shares actionable research advice with thousands of product teams and startup founders: