Presentation during The Lean Startup SXSW by David Binetti, Votizen .

1. The Sacred, The
Profane, and the Pivot
A Brief Case Study of Applied
CDM, AARRR Metrics, and Lean Principles
(c) 2009 David Binetti
Monday, March 7, 2011

2. Definitions
Sacred
Steve Blank’s Customer Development Model (CDM).
I follow it religiously.
Profane
Dave McClure swears every third word. Oh, and
his Acquisition, Activation, Retention, Referral,
Revenue (AARRR) model is awesome.
Pivot
Eric Ries’s concept of strategic shifts predicated on
learnings. (Sorry, no pithy witticism for this.)
(c) 2009 David Binetti
Monday, March 7, 2011

3. CDM (Brief Version)
A) Establish Hypotheses
Test -> (Iterate or Exit)
B) Sell
Test -> (Iterate or Exit)
(c) 2009 David Binetti
Monday, March 7, 2011

4. CDM (Discovery - Phase A)
Category Hypothesis
Product State what you are selling.
Problem State the customer problem you are solving.
Channel State how you are going to sell your product.
State how you are going to generate demand for
Demand
your product.
State the market type (existing, re-segmented,
Market Type
new) you’re in.
Competitive State the competition you face.
(c) 2009 David Binetti
Monday, March 7, 2011

5. CDM (Validation - Phase B)
I could explain in detail, but
basically boils down to:
prove you can sell by actually
selling before you scale.
(c) 2009 David Binetti
Monday, March 7, 2011

6. AARRR Model
(AKA, Metrics for Pirates)
Metric Description
Acquisition Users visit
Activation Users sign up
Retention Users come back
Referral Users like and refer
Revenue User pay something for something
(c) 2009 David Binetti
Monday, March 7, 2011

7. AARRR Model (cont.)
The Model measures:
Conversion rates for each step
Expected value at each step (not
included here.)
A ‘one-slide pitch deck’.
Have these numbers, and you need
nothing else.
(c) 2009 David Binetti
Monday, March 7, 2011

8. Slight Differences
The CDM has you look at and
test the whole business model.
The AARRR assumes the model
rolls up in the metrics.
(c) 2009 David Binetti
Monday, March 7, 2011

9. How They Complement
The CDM teaches you to test
assumptions, but isn’t as specific
on how to measure results.
The AARRR gives metrics to
know what’s good, but not how
to identify what’s broken.
(c) 2009 David Binetti
Monday, March 7, 2011

10. Case: Votizen
Social Network for Registered Voters
Civic Participation Made Simple and
Meaningful
Voter-Focused (as opposed to politician-
focused) approach to $5B Political Market.
Independent, non-partisan, simple and fun!
(c) 2009 David Binetti
Monday, March 7, 2011

11. CDM (Brief)
Category Hypothesis
Social Network for Registered Voters (“LinkedIn
Product
for Politics”)
Traditional campaign techniques #fail in a web
Problem
2.0 world.
The internet is the primary source of voter
Channel
information.
Demand Social networks are both the product and the
demand-generator.
Resegmented; displace current market for
Market Type
direct-mail & robocalls.
Competitive Hyper-fragmented.
(c) 2009 David Binetti
Monday, March 7, 2011

12. AARRR (Brief)
Metric Description
Creates account on system. (Precursor-
Acquisition
requirement)
Certifies voting record (Authenticates - gives
Activation
voter the power)
Referrals Forwards to friends (Acquisition model)
Uses system to affect change (Validates lifetime
Retention
value)
Private! (Sorry, not appropriate for a public
Revenue
document.)
(c) 2009 David Binetti
Monday, March 7, 2011

13. Applied
Started with MVP - Minimum Viable Product.
Just enough to get feedback (cost: $1206)
Metric MVP
Acquisition 5%
Activation 17%
Referrals -
Retention -
Revenue -
Monday, March 7, 2011

14. Improve
Tried a bunch of A/B testing on home page.
Found an optimal message.
Learning: When given the option to explore,
people explored. When given a direction,
they took the direction.
If you want something done, make it part
of registration flow.
(c) 2009 David Binetti
Monday, March 7, 2011

15. Applied
Significant, immediate jump in initial metrics.
(Spend ~$10K)
Metric MVP V.1
Acquisition 5% 17%
Activation 17% 90%
Referrals - 4%
Retention - 5%
Revenue - -
Monday, March 7, 2011

16. Improve Retention and Referrals
Same Process, Different Results
A/B, Feedback, Analytics, etc.
Incremental improvement on retention
and referrals.
Was improving, but marginal value of
efforts were asymptotic.
Time for a pivot!
(c) 2009 David Binetti
Monday, March 7, 2011

17. The Pivot
Eric Ries concept: shift direction while
grounded in learning
Implementation of “back to start of CDM”
I took learning and created new MVP.
So, $10K sunk cost -- buh-bye! (ugh)
Needed to fire current developers,
hire new skills. (double-ugh)
(c) 2009 David Binetti
Monday, March 7, 2011

18. The Pivot (cont.)
But, not a waste!
Time, money and effort brought learning.
Look beyond the features to the problem.
“I always wanted to get more involved.
This makes it so much easier. Thanks!”
(c) 2009 David Binetti
Monday, March 7, 2011

19. Applied
New product launched as MVP - @2gov on Twitter
Contact all public officials at a single address.
Results? Much better R&R, and better conversion.
Metric MVP V.1 New
Product
Acquisition 5% 17% 42%
Activation 17% 90% 83%
Referrals - 4% 54%
Retention - 5% 21%
Revenue - - -
Monday, March 7, 2011

20. Applied
Next step: test revenue assumptions
Provide a channel for voters to support causes and
candidates in which they believe.
Another pivot! - No data yet
Metric MVP V.1 New Next
Product Iteration
Acquisition 5% 17% 42% ?
Activation 17% 90% 83% ?
Referrals - 4% 54% ?
Retention - 5% 21% ?
Revenue - - - ?
Monday, March 7, 2011

21. Takeaways
Set up your CDM hypotheses so that you know
what you are trying to accomplish and how to
tell if you are finished or need to start over.
The AARRR approach serves as a great ‘startup
dashboard.’ Numbers don’t lie (as long as you
don’t lie to yourself.)
Should you need to iterate, then pivot -- don’t
flail. Measure success by lessons learned, not
by the results themselves (oh, and restate CDM.)
(c) 2009 David Binetti
Monday, March 7, 2011

22. Learn from the Source
CDM: http://www.steveblank.com
AARRR: http:/ /www.slideshare.net/
Startonomics/startup-metrics-for-pirates-
presentation
Pivot: http://
www.startuplessonslearned.com/2009/06/
pivot-dont-jump-to-new-vision.html
(c) 2009 David Binetti
Monday, March 7, 2011

23. Feedback Wanted!
Would love feedback!
Want to get better, so criticism is preferred.
“If you don’t have anything bad to say,
then don’t say anything.”
dbinetti@gmail.com
(c) 2009 David Binetti
Monday, March 7, 2011