Alistair Croll's presentation of Lean Analytics hightlights at the Montreal Lean Startup Circle meetup September 2018

1. Lean Analytics, the
good bits.
Lean Startup Circle -
Sept 18, 2018
Alistair Croll
@acroll

3. Steve Blank says
“An organization formed to search for
a sustainable, repeatable business
model.”

4. Startup founder
Search Execute
Executive
Product/market fit

5. Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.

6. Analytics can help.

7. But analyze what?

8. MVP

9. (Not everyone agrees)

10. What’s the
biggest
problem
in a hair
salon?

11. The empty chair.
Knowing this, how
might you change
the client/customer
relationship?

12. How Lean UX did it

13. Analytics is the measurement of
movement towards your business
goals.

14. In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.

15. The
simplest
rule
bad
metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/

16. Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%
of retailers
20%
of retailers
10%
of retailers
You are
just like
Customers that
buy >1x in 90d
Once
2-2.5
per year
>2.5
per year
Your customers
will buy from you
Then you are
in this mode
1-15%
15-30%
>30%
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)

17. MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁

18. Which of these two companies
is doing better?

19. January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or
stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
How about
this one?

20. January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or
stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
How about
this one?

21. Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
Averages $7 $5 $3 $1 $0.5
Look at the
same data
in cohorts

22. Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spend money to
get customers.
Customers are
worth more than
they cost.
Stickiness
Keep people
coming back.
Approach
Get customers
faster than you
lose them.
Math that
matters

23. Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...

24. Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages

25. Six business model archetypes.
E-commerce SaaS Media
Mobile
app
User-gen
content
2-sided
market
The business you’re in

26. (Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation
rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient
Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over

27. Model + Stage = One Metric That Matters.
One Metric
That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat

28. Really? Just one?

29. Yes, one.

30. In a startup, focus is hard to achieve.

31. Having only one metric
addresses this problem.

32. www.theeastsiderla.com

33. Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/

34. Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile
app
User-gen
content
2-sided
market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
(Money from transactions)
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
(Money from active users)
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
(Money from ad clicks)

35. Better: bit.ly/BigLeanTable

36. Drawing lines in the sand

37. A company loses a quarter of its
customers every year.
Is this good or bad?

38. Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.

39. Not knowing what normal is
makes you do stupid things.

40. Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits

41. The Lean Analytics cycle

42. Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI

43. Do AirBnB hosts
get more business
if their property is
professionally
photographed?

44. Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts

45. 5,000 shoots per month
by February 2012

46. Hang on a second.

47. Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?

48. Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI

49. “Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses have in
common?”
Camera model.
With data:
find a commonality
Without data: make a
good guess

50. (No good deed goes unpunished.)

51. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844

52. ARCHIMEDES
HAD TAKEN
BATHS BEFORE.

53. Once, a leader convinced others
in the absence of data.

54. Now, a leader knows
what questions to ask.

55. Thanks!
• Twitter: @acroll
• LinkedIn: linkedin.com/in/alistaircroll
• Mail: acroll@gmail.com
• Web: solveforinteresting.com
• Medium: https://medium.com/@acroll
• Events: pandemon.io, strataconf.com, startupfest.com
• Books: leananalyticsbook.com, tiltthewindmill.com

56. Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko

57. Maybe they don’t love you
like they said they do.
N
Your offering doesn’t make them want to
brag or their contact isn’t really a friend
N
Advocates can’t learn & convey
your message easily
N
They don’t trust you entirely
N
Woohoo! Scalable, viral,
explainable product!
Y
Get a meeting
Y
Grab the phone
Y
They pitch it
Y
Call them now?
Y
Intro to a friend?
Interview