Slides from Montreal's Startup Weekend NEXT on lean analytics, market sizing, and how to build startups into great companies

1. Startup Weekend
NEXT
Pre-accelerator session 3

2. Agenda
Why build something people want?
What’s product/market fit?
What’s an MVP and why you care
Curated MVP
Pirate metrics
The One Metric that Matters

3. Why build something people want?
Everything else is a hobby or a nonprofit.

4. Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.

5. The core of Lean
is iteration.

6. Most startups don’t know what they’ll
be when they grow up.
Hotmail
was a
database
company
Flickr
was going to
be an MMO
Twitter
was a
podcasting
company
Autodesk
made
desktop
automation
Paypal
first built for
Palmpilots
Freshbooks
was invoicing
for a web
design firm
Wikipedia
was to be
written by
experts only
Mitel
was a
lawnmower
company

7. “What information
consumes is rather
obvious: it consumes
the attention of its
recipients.
Hence a wealth of information
creates a poverty of attention,
and a need to allocate that
attention efficiently among the
overabundance of information
sources that might consume it.”
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon

8. The mythical product/market fit
And other startup unicorns.

9. Product
(new “what”)
Market
(new “who”)
Method
(new “how”)
3 kinds of
innovation

10. Product
(new “what”)
Market
(new “who”)
Method
(new “how”)
Pure
startup
Distribution
innovation
Market
diversification
Channel
expansion
Another view
Inventive
Adjacent
Disruptive
Core

11. What’s an MVP?
The least you need to do to quantify an assumption.

12. Minimum

13. Viable

14. qidiq streamlines invites
Survey owner adds recipient to group
Survey owner asks question
Recipient reads survey question
Recipient responds to question
Recipient sees survey results
(Later, if needed…)
Recipient visits site; no password!
Recipient does password recovery
One-time link sent to email
Recipient creates password
Recipient can edit profile, etc.
Survey owner adds recipient to group
Survey owner asks question
Recipient gets invite
Recipient reads survey question
Recipient responds to question
Recipient installs mobile app
Recipient creates account, profile
Recipient sees survey results
Recipient can edit profile, etc.
10-25%RESPONSERATE
70-90%RESPONSERATE

15. Delightful?
If that’s required for viability

16. Curated MVP
Localmind, Airbnb, roof racks and steering wheels.

17. Do AirBnB hosts
get more business
if their property is
professionally
photographed?

18. Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts

19. 5,000 shoots per month
by February 2012

20. Localmind hacks Twitter
Needed to find out if a core assumption—strangers answering
questions—was valid.
Ran Twitter experiment instead of writing code
Asked senders of geolocated Tweets from Times Square random
questions; counted response rate
Conclusion: high enough to proceed

21. LikeBright’s mechanical turk
Used Mechanical Turk, Google Voice to speak w/
100 single women; paid $2. The interviews
lasted typically around 10-15 minutes.
Simple interview script with open-ended
questions, since he was digging into the problem
validation stage of his startup.
Founder Nick Soman: “I was amazed at the
feedback I got. We were able to speak with one
hundred single women that met our criteria in
four hours on one evening.”
Went back to TechStars and got accepted.
LikeBright’s website is now live with a 50%
female user base, and recently raised a round of
funding.
“Since that first foray into interviewing customers,
I’ve probably spoken with over a thousand
people through Mechanical Turk,”

22. Pirate metrics

23. Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...

24. The One Metric that Matters
Focus matters most of all

25. Unfortunately,
we’re all liars.

26. Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.

27. Analytics can help.

28. Analytics is the measurement of
movement towards your business
goals.

29. In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.

30. The
simplest
rule
bad
metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/

31. MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁

32. January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohort 1 2 3 4 5
January
February
March
April
May
$5 $3 $2 $1 $0.5
$6 $4 $2 $1
$7 $6 $5
$8 $7
$9
How about
this one?

33. Cohort 1 2 3 4 5
January
February
March
April
May
Averages
$5 $3 $2 $1 $0.5
$6 $4 $2 $1
$7 $6 $5
$8 $7
$9
$7 $5 $3 $1 $0.5
Look at the
same data
in cohorts

34. The Lean Analytics framework.

35. Six business models
E-commerce SaaS Media
Mobile
app
User-
generated
content
2-sided
market
Your company

36. Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages

37. (Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation
rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient
Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over

38. Model + Stage = One Metric That Matters.
One Metric
That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat

39. Really? Just one?

40. Yes, one.

41. In a startup, focus is hard to achieve.

42. Having only one metric
addresses this problem.

43. Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/

44. Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile
app
User-gen
content
2-sided
market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
(Money from transactions)
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
(Money from active users)
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
(Money from ad clicks)

45. Better: bit.ly/BigLeanTable

46. Market Sizing
Top-down and bottoms-up

47. Top-down
Start with the whole planet, then narrow it down.

48. Population of Canada

49. Percent mobile phones by country

50. IOS penetration

51. Put this all together
Metric/Assumption Value
Population of Canada
Mobile phone percentage
Mobile phones in Canada
IOS percentage
IOS phones in Canada
IOS devices per account
IOS accounts we can sell to in Canada
(this is our Total Addressable Market)
34.88M
80%
27,904M
15.5%
4.32M
3.1*
1.39M
* I made this up. Don’t believe everything you read.

52. How much is the TAM worth?
Metric/Assumption Value
TAM
Percent we will claim
Number of users
User lifetime
Revenue per month
CLV
Expected total revenues
1.39M
5%
69,500
40 months
$5 per month
$200
$13.9M
* I made this up. Don’t believe everything you read.

53. Bottoms-up
Start with one buyer, then add it up

54. Bottoms-up
Metric/Assumption Value
Impressions (acquisition)
Percent that install
Installs (activation)
Percent still using after 30 days
Become regular users (retention)
Percent that buy
Buy premium version (revenue)
Percent that churn each month
Customer lifespan
Revenue per month
Customer lifetime value
TAM value
9,266,667
10%
926,667
25%
231,667
30%
69,500
2.5%
40
5
200
$13,900,000

55. Reality check
Bottoms-up (TAM) Value
Impressions (acquisition) 9.3 M impressions
Top-down (TAM) Value
IOS accounts we can sell to in Canada
(this is our Total Addressable Market) 1.39 M users in Canada
You Shall Not Pass.

56. More realistic
Metric/Assumption Value
Impressions (acquisition)
Percent that install
Installs (activation)
Percent still using after 30 days
Become regular users (retention)
Percent that buy
Buy premium version (revenue)
Percent that churn each month
Customer lifespan
Revenue per month
Customer lifetime value
TAM value
1,300,000
10%
130,000
25%
32,500
30%
9,750
2.5%
40
5
200
$1,950,000

57. Some more resources

63. Thanks!
@acroll