Never Say Usually: How To Ask Questions In User Interviews
And other tips for discovering your customer’s detailed, messy reality — featuring Teresa Torres, The Mom Test, and Indian Matchmaking.
—
“How do you usually … ?”
Hypothetical questions like this are easier for everyone in a research interview.
Asking someone what they “usually” do in a given scenario removes any pressure on them to reveal personal details. You want your interviewee to feel at ease, and that’s exactly what a “usually” question accomplishes.
But “usually” questions also encourage people to present the ideal version of themselves — the person who goes to the gym at least 3x/week, rarely procrastinates, always gets out of bed after their first alarm, and is never influenced by things like ego, corporate politics, or pettiness.
It’s obvious that this “ideal” profile of a person isn’t useful for researchers. That’s not what this post is about. The bigger problem with these hypothetical questions is that they encourage respondents to be vague — and vagueness is the enemy of discovery research.
Discovery research is about uncovering the unmet needs, pains, and desires that drive people’s behavior. Things like needs, pains and desires are not “ideal”. In fact, they’re often a little embarrassing to admit — like wasting an hour browsing Netflix and never actually picking something to watch…
The Netflix Question
Imagine you’re a Product Manager at Netflix trying to increase the number of hours of content watched per user (their North Star Metric). To identify product opportunities, you decide to interview some Netflix users to learn more about how they’re currently using the product:
In the hypothetical approach, I ask Sam how he tends to use Netflix and Sam says he recently finished watching Love is Blind, Too Hot To Handle, and Indian Matchmaking. My takeaway is that Sam loves reality dating shows and Netflix should suggest more content in this genre to users.
In the specific approach, I ask Sam to tell me about the last time he used Netflix. Sam tells me that he finished watching Indian Matchmaking two nights ago and spent more time yesterday evening searching for something new to start than actually watching anything. My takeaway is that Sam is struggling with content discovery and I need to learn more about how the existing recommendation features may not be meeting his needs.
Such a small shift in question can lead to dramatically different insights from your research.
Avoiding vague questions
There are a couple of common fluffy question types that lead to vague answers:
Generic: usually, always, never…
Hypothetical: might, could...
Future-Tense: would, will...
Indirect: Do you ever…, Have you considered…
Whenever you catch yourself asking a fluffy question, find a way to anchor the respondent with a specific question instead:
When was the last time you …?
Can you tell me more about your most recent … project?
What was a memorable time that you tried to …?
What were you doing right before you decided to …?
Your goal is to find a loose thread, an interesting hook, or an unexpected insight. You’re looking for responses that could open a door and allow you to dig deeper into their experience. Once you find that open door, there’s only one question you really need to know…
The Most Important Question
Opportunities don’t live in the “usually”, they live in the “specifically”. Researchers are trying to uncover the person’s messy, problematic reality in all its glorious detail. When that messy reality is (very generously) revealed to us, our job is to understand why it is the way it is.
Thankfully, the only thing we need to do at that point is ask “Why?” — many times and in a way that makes the other person comfortable sharing their needs, pains and desires with you. To do that, they need to be reassured you won’t judge them, publicly reveal their answer, or share it in a way that will come back to haunt them.
Learning which questions not to ask is the easy part. Building trust in a user interview is the real challenge.
The principle of “specificity” in discovery research is covered in the books The Mom Test by Rob Fitzpatrick (which focuses on customer discovery) and Continuous Discovery Habits by Teresa Torres (which focuses on product discovery). The difference between customer discovery and product discovery is explained in this post. The Netflix example was taken from Teresa’s book.
PS. We started a new cohort of OpinionX Academy this week, where we coach founders through the customer discovery process to help them validate their startup idea. Learn more about our 9-Step OpinionX Academy Framework and how we help teams with our research support services.
Subscribe to The Full-Stack Researcher newsletter where we share research strategies for founders and product teams: