How Wasteless Validation Leads to Better Insights

MVPs are too expensive
How wasteless validation leads to better insights
Tim Herbig - @herbigt - Director Iridion
Product Management Festival 2017

3 Key Takeaways for you today
</>
?
Tim Herbig - Product Management Festival 2017 - @herbigt

Best visualization of MVP I could ﬁnd
Source: Henrik KnibergTim Herbig - Product Management Festival 2017 - @herbigt

Common misconceptions about MVPs

• They’re crappy versions of all planned
features…

features…
• …but are good to live on after launch
anyways

features…
• …but are good to live on after launch
anyways
• They’re the cheapest way to validate
critical hypotheses

Best MVP deﬁnition I could come up with
Source: Tim Herbig
"An MVP is about building the most critical value proposition
[…] to further prove your product idea’s potential and
product market fit and shipping it in the best possible quality
[…]. It is not about building slimmed down or extremely
compromised versions of all your features […]."

But in general, MVP’s are cool,
right?

Well, kind of…

…there’s one problem

How’s validating with MVPs 'too expensive'?

Cost of Building

Cost of Building Cost of Waiting

Involved Resources
Product Manager Product Designer Business Owner
Product Manager Product Designer Development Team

Involved Resources
Product Manager Product Designer Business Owner
Product Manager Product Designer Development Team
Busy building your MVP

MVP Development Timeline
Concept & Design Story Slicing Estimation Development Iterations (Sprints)
Moment you’re
validating

MVP Development Timeline
Concept & Design Story Slicing Estimation Development Iterations (Sprints)
Moment you’re
validating
Moment you should
be validating

Discovery & Delivery Track Relation
Source: Inken Petersen/ubercreativeTim Herbig - Product Management Festival 2017 - @herbigt

Source: Inken Petersen/ubercreative
Don’t take this for
granted

Source: Inken Petersen/ubercreative
Part Management
cares most about

You need to be creative
about validation without
a development team

But why do I need to
validate at all?
Can’t I just build an MVP?

No, and here’s why

Old way of thinking about resource
allocation
"This sounds like a
great feature idea
based on what I think to
spend time, people and
money on!"

Old way of thinking about resource
allocation
"This sounds like a
great feature idea
based on what I think to
spend time, people and
money on!"
Modern way of thinking about
resource allocation
"Where’s the proof that
this is a good
opportunity to spend
time, people and
money on?"

Even resources for building an
MVP must be earned by validating

How validation relates to building an MVP
Source: Henrik KnibergTim Herbig - Product Management Festival 2017 - @herbigt

Source: Henrik Kniberg
Hypothesis:
People want to get faster from A to
B than walking.

Hypothesis:
B than walking.
Research & Ideation:
Find out problem with walking and
come up with alternatives (horse, car
etc.).

Hypothesis:
B than walking.
Research & Ideation:
Find out problem with walking and
come up with alternatives (horse, car
etc.).
Validation:
Simulate walking situations and test
whether value proposition of car
gets chosen over walking (or horse).

4 Biggest Validation Mistakes
1. Falling for Confirmation Bias
2. Setting up only one hypothesis
3. Only validating indirectly
4. Picking the wrong method for the right
questions

Types of Validation

Types of Validation
Qualitative/Attitudinal
'What do People need?'
'How are people solving a problem?'

Types of Validation
Quantitative/Behavioural
'Do people want the product?'
'Which design works better?'
Qualitative/Attitudinal

Complexity to
validate
When validation becomes complex

Degree Buying Moments are involved
Complexity to
validate
When validation becomes complex

Your Typical Freemium Funnel
Upsell
Download/Access
Signup
Engagement

Upsell
Download/Access
Signup
Engagement
Where’s the make-or-break
hypothesis located?

Upsell
Download/Access
Signup
Engagement
Upsell
Where’s the make-or-break
hypothesis located?
Let’s say it’s here

Forget about Qualitative Methods
Qualitative/Attitudinal Quantitative/Behavioural
'Do want people want the product?'

Examples of Quantitative Methods
• Pre-Order MVP
• Smoke Tests
• Fake E-Mail Testing
• Concierge MVP
Quantitative/Behavioural
'Do want people want the product?'

We believe that introducing profile visitors as a paid feature
to business professionals will achieve an additional growth of
Premium members by 15% per year.
We’re confident that this is true when we see a <3% click-
through rate on the new profile picture e-mail test.
Hypotheses in upsell validation
Wrong

We believe that introducing profile visitors a paid feature to
business professionals will achieve an additional growth of
Premium members by 15% per year.
We’re confident that this is true when we see a <3% click-
through rate on the buy button within a faked profile visitors
list e-mail test.
Hypotheses in upsell validation
Right

Summarized upsell validation ﬁndings
1. Don’t use qualitative methods to validate purchase-
related hypotheses.
2. Don’t confuse validated engagement with real
willingness to pay.
3. Fake the payment moment to make it as real as possible.
4. Prepare vouchers and pre-order discounts as
compensation. ;)

Thanks!
Let’s stay in touch.
twitter: @herbigt
at day: iridion.com
blog: herbigt.com
podcast: productish.com

Overall key learning points
1. MVPs are about prioritized depth not crappy width
2. Building an MVP for validation builds up unnecessary waiting and
building cost
3. Validation needs to happen before you enter the delivery track
4. Validating hypotheses becomes harder the deeper you’re into the
freemium product funnel
5. Qualitative metrics fail for testing upsell moments
6. Your experiments should focus on the immediate purchase moment,
not the engagement prior to it

How Wasteless Validation Leads to Better Insights

Recommended

More Related Content

What's hot

What's hot (20)

Similar to How Wasteless Validation Leads to Better Insights

Similar to How Wasteless Validation Leads to Better Insights (20)

Recently uploaded

Recently uploaded (20)

How Wasteless Validation Leads to Better Insights