A talk I gave at Product Camp Cascadia 2021 about how to build the right products faster by running lean experiments and staying objective with lean expectations.
3. What you'll get out of this talk:
How to build less of the wrong products by running lean
experiments.
FOUR types of lean experiments through FIVE real-life
examples.
The importance of lean expectations in staying objective.
When to walk away from an idea.
4. Lena Sesardic
7 yrs startups/startup-like teams (3.5 yrs PM).
Entrepreneur, Creator & Writer.
Mentor & advise venture teams.
Author of The Making of Product Managers.
I write on lenasesardic.com & for Product
Coalition & The Startup on Medium.
5. Think in experiments, not products.
What is the quickest way to test an idea?
Riskiest assumptions.
Hypothesis (specific, measurable, disprovable, time-bound).
Expected outcome—stay objective & accountable.
6. Divorce yourself from your idea
9/10 startups fail.
Even more product features fail.
Most experiments fail.
Failed experiments = learning.
7. Lean experiments
Few days/weeks to run.
Test a specific hypothesis.
Quantitative/qualitative data as feedback.
Prioritize learning.
12. 2. Place orders with restaurants on the day manually.
Resource: Veni.ca
Real-life Example: Veni
13. 3. Deliver all meals manually.
Resource: Veni.ca
Real-life Example: Veni
14. Spent $25.
Batching meal orders is valuable to restaurants.
Collected feedback on the customer experience.
Going forward: Wizard of Oz + landing page analytics
Results:
Resource: Veni.ca
Real-life Example: Veni
15. Pros & Cons: Wizard of Oz method
No need to build
technology.
Quick to ship.
High-touch service =
learning opportunity.
Easy to pivot.
Requires manual work.
Have to build product
eventually.
May not be polished.
May not be possible for
all product concepts.
17. Fake Door Demand Test
Fake the existence of a product & measure the
actions users take.
18. Real-life Example: Kettle & Fire
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
19. Real-life Example: Kettle & Fire
1. Keyword search & trend analysis.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
20. 2. Simple landing page with "Order Bone Broth Now!" button &
$50 worth of Bing ads.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire
21. 3. Redirected to "PayPall $29.99 to xxxxxx@gmail.com."
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire
22. 7814 2384 30.51%
Visitors Conversions Conversion rate
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Results:
Real-life Example: Kettle & Fire
23. 4. Out of stock follow-up email.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire
28. Resource: Customer Camp.
Quantitative data: waitlist sign-ups.
Customer discovery interviews w/ interested parties.
Qualitative data >> deep insights.
Connections with potential future customers.
Results:
Real-life Example: Customer Camp
29. Pros & Cons: Fake Door Demand Test
Quick to ship landing
page/button.
No need to create or
source product.
Quantitative/qualitative
data for analysis.
Scalable (landing
pages/button).
Paid ads expensive &
require expertise.
False negatives due to
poor UX/ad copy or choice
of ad platform.
Potentially frustrated
customers (use sparingly).
34. 100+ leads for each of 5 niche industries.
2.
Real-life Example: Scheduling App X
35. 5 cold email campaign to the 5 niche industries with clickable
prototype included.
3.
Real-life Example: Scheduling App X
36. Identified 2 niche industries with highest demand (carpet
cleaning & painting).
50 early customers with signed Letters of Intent (LOIs).
Results:
Real-life Example: Scheduling App X
37. Pros & Cons: Cold Outreach
Quick to ship.
Quantitative/qualitative
data for analysis.
Scalable (cold emails).
Some lead generation can be
time-consuming & expensive.
False negatives due to poor
email copy or cold calling.
Easy to pivot.
False negatives due to poorly
formed value prop.
41. 1. Gauged interest with a tweet & blog post.
Real-life Example: Lean SEO Course
Resource: Starter Story
42. 2. Positive qualitative feedback & insight into customer pains.
Real-life Example: Lean SEO Course
Resource: Starter Story
43. 3. Gauge product format with a tweet.
Real-life Example: Lean SEO Course
Resource: Starter Story
44. 4. Pre-orders to validate willingness to pay.
Real-life Example: Lean SEO Course
Resource: Starter Story
45. Real-life Example: Lean SEO Course
$2,054 in presales in 2 weeks.
$30k in sales within 2 months.
Results:
Resource: Starter Story
46. Pros & Cons: Inchworm Medley
Multiple checkpoints for
validation.
Builds community.
No need to create or
source product right
away.
Quantitative/qualitative
data for analysis.
More effective with an existing
audience & credibility.
May be difficult to back out of
(social reputation).
Risk of someone stealing
your idea.
47. Recap of Lean Experiments
#1 Wizard of Oz method #2 Fake door demand test
#3 Cold outreach #4 Inchworm medley
48. Walking Away From an Idea
Pre-desired outcomes are not met.
We tend to see what we want to see.
Exceptions are rare.
49. The Value of Lean Experiments
Speed.
Learning.
Hidden ROI.