SlideShare a Scribd company logo
1 of 113
Lean Analytics
    Use data to build a
   better business faster.




                             www.leananalyticsbook.com
  @byosko | @acroll
                                   @leananalytics
Silicon Valley
doesn’t love failure.
It just hates it
less than the alternative:
Making something nobody needs.
Most startups don’t know what they’ll be
when they grow up.


                 Freshbooks                          Mitel
                 was invoicing    Wikipedia         was a
  Paypal
                   for a web      was to be      lawnmower
first built for                                     company
                  design firm      written by
 Palmpilots                      experts only


                   Flickr
 Hotmail                           Twitter       Autodesk
                 was going to
  was a                             was a       made desktop
                 be an MMO
 database                        podcasting      automation
 company                          company
OnLab Japan introduction to Lean Analytics
Kevin Costner is a lousy entrepreneur.




 Don’t sell what you can make.
 Make what you can sell.
Analytics is the measurement
of movement towards your
business goals.




                  http://www.flickr.com/photos/itsgreg/446061432/
Small business example:
Solare watches the
numbers

•   Stage: Revenue
•   Model: Retailer

•   Solare is an Italian fine-dining restaurant under new management. The new team
    is trying to identify the key metrics and leading indicators
Solare watches the numbers

• A line in the sand: Gross Revenue to Labor Cost
   • Under 30% is good
   • Below 24% is great
   • Lower than 20% and you may be under-staffing, leading to dissatisfied
     customers
• A leading indicator: Total covers is 5x reservations at 5PM
   • If you have 50 reservations at 5, you’ll have 250 covers that night.
   • This ratio varies by restaurant.
In a startup, the purpose of
analytics is to iterate to a
product/market fit before
the money runs out.
Qualitative or Quantitative
5 things you    Exploratory or Reporting
need to know    Vanity or Actionable
about metrics   Correlated or Causal
                Leading or Lagging
Qualitative                                    Quantitative
  Unstructured,                                    Numbers and stats;
  anecdotal,                                       hard facts but less
  revealing, hard to                               insight.
  aggregate.
  Warm and fuzzy.                                  Cold and hard.




http://www.flickr.com/photos/zooboing/8388257248/   http://www.flickr.com/photos/x1brett/4665645157/
Simply: you can’t count smiles.
Discover qualitatively, prove quantitatively.
Qualitative is inspiration, quantitative is verification.
Exploratory                                          Reporting
   Speculative, trying                                 Predictable, keeping
   to find unexpected                                   you abreast of
   or interesting                                      normal, managerial
   insights.                                           operations.

http://www.flickr.com/photos/50755773@N06/5415295449/    http://www.flickr.com/photos/elwillo/4737933662/
Donald Rumsfeld on analytics

                               Are facts which may be wrong and
                    we know    should be checked against data.

            know
                    we don’t   Are questions we can answer by
                               reporting, which we should baseline
                       know    & automate.
Things we
                               Are intuition which we should
                    we know    quantify and teach to improve
            don’t              effectiveness, efficiency.
            know
                    we don’t   Are exploration which is where
                               unfair advantage and interesting
                       know    epiphanies live.
                                    (Or rather, Avinash Kaushik channeling Rumsfeld)
Vanity                                       Actionable
                                                          Picks a
                                                          direction.



Makes you feel
good, but doesn’t
change how you’ll
act.
http://www.flickr.com/photos/lostseouls/807253220/   http://www.flickr.com/photos/aussiegall/6382775153/
A metric from the early, foolish days of the Web.
      Hits
                   Count people instead.
                   Marginally better than hits. Unless you’re displaying
  Page views
                   ad inventory, count people.
                   Is this one person visiting a hundred times, or are a
     Visits
                   hundred people visiting once? Fail.
                   This tells you nothing about what they did, why they
Unique visitors
                   stuck around, or if they left.
   Followers/      Count actions instead. Find out how many followers
  friends/likes    will do your bidding.
Time on site, or   Poor version of engagement. Lots of time spent on
  pages/visit      support pages is actually a bad sign.
                   How many recipients will act on what’s in them?
Emails collected

  Number of        Outside app stores, downloads alone don’t lead to
  downloads        lifetime value. Measure activations/active accounts.
2-sided market model:
AirBnB and photography

•   Stage: Revenue
•   Model: 2-sided marketplace

•   Rental-by-owner marketplace that allows property owners to list and market
    their houses. Offers a variety of related services as well.
AirBnB tests a hypothesis

• The hypothesis: “Hosts with professional photography will get more business.
  And hosts will sign up for professional photography as a service.”


• Built a concierge MVP


• Found that professionally photographed listings got 2-3x more bookings than the
  market average.


• In mid-to-late 2011, AirBnB had 20 photographers in the field taking pictures for
  hosts.
NIGHTS BOOKED

10 million


 8 million


   6 million


                                      20 photographers
    4 million




     2 million




           2008   2009         2010           2011       2012
http://www.flickr.com/photos/circasassy/7858155676/




                           it’s a
                           how you behave,
                           If it won’t change




bad
metric.
A few words on causality.




               http://www.flickr.com/photos/roryfinneren/65729247
50


37.5


 25


12.5


  0
       1         2         3         4          5         6          7         8         9         10
                                              Seat rentals
           http://www.rvca.com/anp/wp-content/plugins/wp-o-matic/cache/57226_07+proof+1a+hb+beach+day.jpg
http://www.imdb.com/media/rm3768753408/tt0073195
http://www.flickr.com/photos/kapungo/2287237966
10000


1000


 100


  10


   1
        Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec


                Ice cream consumption    Drownings
http://www.flickr.com/photos/25159787@N07/3766111564
http://www.flickr.com/photos/wheressteve/3284532080
http://www.flickr.com/photos/wtlphotos/1086968783
10000


1000


 100


  10


   1
        Jan Feb Mar Apr May Jun      Jul   Aug Sept Oct Nov Dec
             Ice cream consumption   Drownings    Temperature
http://www.flickr.com/photos/stuttermonkey/57096884
http://www.flickr.com/photos/germanuncut77/3785152581
http://www.flickr.com/photos/fasteddie42/2421039207
Correlated                     Causal
Two variables that       An independent
change in similar        factor that directly
ways , perhaps           impacts a
because they’re          dependent one.
linked to something
else.
                   Summer
              al




                                Ca
            us




                                 us
           Ca




                   Correlated    al   Drowning
 Ice cream
consumption
http://www.flickr.com/photos/bootbearwdc/1243690099/
Causality is a superpower, because it lets you
change the future.


 Correlation lets you              Causality lets you
 predict the future                change the future


“I will have 420                 “If I can make more
engaged users and                first-time visitors stay
75 paying customers              on for 17 minutes I
next month.”                     will increase sales in
                                 90 days.”

                                           Optimize the
Find correlation        Test causality
                                           causal factor
Leading               Lagging
Number today that   Historical metric that
shows metric        shows how you’re
tomorrow—makes      doing—reports the
the news.           news.
A leading indicator for e-commerce

  How many of
 your customers    Then you are in   Your customers      You are just
                                                                               Focus on
  buy a second       this mode       will buy from you       like
time in 90 days?

                                                                               Low CAC,
   1-15%           Acquisition           Once              70%                   high
                                                         of retailers          checkout




  15-30%              Hybrid             2-2.5             20%                 Increasing
                                         per year        of retailers            returns



                                                                                Loyalty,
   >30%              Loyalty              >2.5             10%                 inventory
                                         per year        of retailers          expansion
                                                             (Thanks to Kevin Hilstrom for this.)
Think about a car

•60MPH is twice as fast as 30MPH
•Speed limits and mileage are well understood
  •Kilometres are conveniently decimal; miles map to hours
•Ratios everywhere
  •Miles travelled is good;
  •Miles per hour is better;
  •Accelerating or decelerating changes your gas pedal
•Custom metrics: “MPH divided by speeding tickets” as a
 metric of “driving fast without losing my license”
The Lean Analytics Framework.
Eric Ries’
Three engines


             Stickiness           Virality             Price


Approach    Keep people        Make people        Spend revenue
            coming back.       invite friends.   getting customers.


Math that   Get customers     How many they        Customers are
 matters    faster than you     tell, how fast    worth more than
              lose them.       they tell them.    they cost to get.
Dave McClure’s Pirate metrics


                           How do your users become aware of you?
          Acquisition
      AARRR
                             SEO, SEM, widgets, email, PR, campaigns, blogs ...


                           Do drive-by visitors subscribe, use, etc?
              Activation     Features, design, tone, compensation, affirmation ...


                           Does a one-time user become engaged?
              Retention      Notifications, alerts, reminders, emails, updates...


                           Do you make money from user activity?
               Revenue       Transactions, clicks, subscriptions, DLC, analytics...


                           Do users promote your product?
                Referral     Email, widgets, campaigns, likes, RTs, affiliates...
The five Stages of Lean Analytics




                         Empathy
The stage you’re at




                        Stickiness

                           Virality

                         Revenue

                            Scale
Example: a restaurant

• Empathy: Before opening, the owner first learns about the diners in its area,
  their desires, what foods aren’t available, and trends in eating.
• Stickiness: Then he develops a menu and tests it out with consumers, making
  frequent adjustments until tables are full and patrons return regularly. He’s giving
  things away, testing things, asking diners what they think. Costs are high
  because of variance and uncertain inventory.
• Virality: He starts loyalty programs to bring frequent diners back, or to
  encourage people to share with their friends. He engages on Yelp and
  Foursquare.
• Revenue: With virality kicked off, he works on margins—fewer free meals,
  tighter controls on costs, more standardization.
• Scale: Finally, knowing he can run a profitable business, he pours some of the
  revenues into marketing and promotion. He reaches out to food reviewers, travel
  magazines, and radio stations. He launches a second restaurant, or a franchise
  based on the initial one.
Example: a software company

• Empathy: The founder finds an unmet need, often because she has a background in a
  particular industry or has worked with existing solutions that are being disrupted.
• Stickiness: She meets with an initial group of prospects, and signs contracts that look more
  like consulting agreements, which she uses to build an initial product. She’s careful not to
  commit to exclusivity, and tries to steer customers towards standardized solutions, charging
  heavily for custom features. She supports the customers directly from the engineering team
  until the product is stable and usable.
• Virality: Product in hand, she asks for references from satisfied customers, and uses them as
  testimonials. She starts direct sales, and grows the customer base. She launches a user group,
  and starts to automate support. She releases an API, encouraging third-party development and
  scaling potential market size without direct development.
• Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling
  costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored
  based on anticipated payoff and development cost. Recurring license and support revenue
  becomes an increasingly large component of overall revenues.
• Scale: She signs deals with large distributors, and works with global consulting firms to have
  them deploy and integrate her tool. She attends trade shows to collect leads, carefully
  measuring cost of acquisition against close rate and lead value.
Empathy
Get inside their head.
Mobile app model:
Localmind hacks Twitter

•   Stage: Empathy
•   Model: UGC/mobile

•   Real-time question and answer platform tied to locations.
•   Needed to find out if a core behavior—answering questions about a place—
    happened enough to make the business real
Localmind hacks Twitter

• Before writing a line of code, Localmind was concerned that people would never
  answer questions.
   • This was their biggest risk: if questions went unanswered users would have a
     terrible experience and stop using Localmind.
• Ran an experiment on Twitter
   • Tracked geolocated tweets in Times Square
   • Sent @ messages to people who had just tweeted, asking questions about
     the area: how busy is it; is the subway running on time; is something open;
     etc.
• The response rate to their tweeted questions was very high.
   • Good enough proxy to de-risk the solution, and convince the team and
     investors that it was worth building Localmind.
When it’s time to move on

•   Have you conducted enough quality customer interviews to feel confident that
    I’ve found a problem worth solving?
•   Do you understand your customer well enough?
•   Do you believe your solution will meet the needs of customers?
Stickiness
The dogs like the dogfood.
1995   Hits
1997   Visits
1999   Visitors
2002   Conversions   Who did you add? Where from? Why?
2010   Engagement    What did they do? How did it benefit?

                     Who did you lose? Why did they leave?
Stickiness stage:
WP Engine discovers the
2% cancellation rate

•   Stage: Stickiness
•   Model: SaaS

•   Wordpress hosting company founded in July 2010, it raised $1.2M in November
    2011
WP-Engine discovers the 2%
cancellation rate

• All companies have cancellations, but founder Jason Cohen was alarmed that he
  was losing a quarter of customers every year.


• Jason called customers himself. “Not everyone wanted to speak with me, but
  enough people were willing to talk, even after they had left, that I learned a lot
  about why they were leaving.”


• Asked around. Turns out 2% is best case for most hosting companies.


• Without this, the company would have been getting diminishing returns over-
  optimizing churn; instead, they could focus on maximizing revenues or lowering
  acquisition costs.
When it’s time to move on
•
 Are people using the product as expected?
•
 Define an active user. What percentage of your users/customers is active?
Write this down. Could this be higher? What can you do to improve engagement?
•
 Evaluate your feature roadmap against the 7 questions to ask before building
more features. Does this change the priorities of feature development?
•
 Evaluate the complaints you’re getting from users. How does this impact
feature development going forward?
Virality
I told two friends.
Virality stage:
Timehop’s content sharing

•   Stage: Virality
•   Model: Mobile app

•   Social network around the past
•   Focused on virality (but not necessarily the coefficient!)
The one metric that matters: content
sharing

• Focused on percent of daily active users that share their content

• Aiming for 20-30% of DAU sharing




“All that matters now is virality.
Everything else—be it press,
publicity stunts or something else—
is like pushing a rock up a
mountain: it will never scale. But
being viral will.”
          - Jonathan Wegener, co-founder
3 kinds of virality
•
 Inherent virality is built into the product, and happens as a function of use.
•
 Artificial virality is forced, and often built into a reward system.
•
 Word of mouth virality is simply conversations generated by satisfied users.
When it’s time to move on

• Are you using one of the three types of virality (inherent, artificial, word of mouth)
  for your startup? Describe how. If virality is a weak aspect of your startup, write
  down 3-5 ideas for how you could build more virality into your product.
• What’s your viral coefficient? Even if it’s below 1 (which it likely is), do you feel like
  the virality that exists is good enough to help sustain growth and lower customer
  acquisition costs?
• What’s your viral cycle time? How could you speed it up?
Revenue
Pour some of the money back into acquisition.
Revenue stage:
Backupify’s customer
lifecycle

•   Stage: Revenue
•   Model: SaaS

•   Leading backup provider for cloud based data.
•   The company was founded in 2008 by Robert May and Vik Chadha
•   Has gone on to raise $19.5M in several rounds of financing.
Shifting to Customer Acquisition
Payback as a key metric

• Initially focused on site visitors

• Then focused on trials

• Then switched to signups

• Today, MRR

• In early 2010, CAC was $243 and ARPU was only $39

   • Pivoted to target business users

   • CLV-to-CAC today is 5-6x

• Now they track Customer Acquisition Payback

   • Target is less than 12 months
Sergio Zyman’s many “mores”

• If you’re dependent on physical, per-transaction costs (like direct sales, or
  shipping products to a buyer, or signing up merchants) then more efficiently will
  figure prominently on either the supply or demand side of your business model.
• If you’ve found a high viral coefficient, then more people makes sense, because
  you’ve got a strong force multiplier added to every dollar you pour into customer
  acquisition.
• If you’ve got a loyal, returning set of customers who buy from you every time,
  then more often makes sense, and you’re going to emphasize getting them to
  come back more frequently.
• If you’ve got a one-time, big-ticket transaction, then more money will help a lot,
  because you’ve only got one chance to extract revenue from the customer and
  need to leave as little money as possible on the table.
• If you’re a subscription model, and you’re fighting churn, then upselling
  customers to higher-capacity packages with broader features to additional
  subscribers within their organization is your best way of growing existing
  revenues, so you’ll spend a lot of time on more stuff.
When it’s time to move on
You’re making money
You’re sustainable
You’re tracking growth metrics
Scale
Let others grow your business for you.
The hole in the middle


           Differentiation         Efficiency


               Apple               Costco
                         Here be
                         dragons


                        Your local
                 sustainable gluten-free
                     cupcake shop
                         Niche
Guerrilla             Data-
marketing                    driven
                             learning

               GROWTH
               HACKING




       Subversiveness
The trust equation


            Your expertise &   Do what you         Are you one of
              reputation;      say; say what       us? Morality &
             certifications        you do.           personality.



                    Credibility + Reliability      +   Intimacy
Trustworthiness =
                               Self-orientation

                                    What’s in it
                                     for you?

                                                       Maister, Green and Galford
• A Facebook user reaching 7 friends within 10 days of
  signing up (Chamath Palihapitiya)
• If someone comes back to Zynga a day after signing up
  for a game, they’ll probably become an engaged, paying
  user (Nabeel Hyatt)
• A Dropbox user who puts at least one file in one folder
  on one device (ChenLi Wang)
• Twitter user following a certain number of people, and a
  certain percentage of those people following the user
  back (Josh Elman)
• A LinkedIn user getting to X connections in Y days (Elliot
  Schmukler)

  (These are also great segments to analyze.)
                                 (from the 2012 Growth Hacking conference)
The growth hack

• Growth hacking is simply what marketing should have been doing, but it fell in
  love with Don Draper and opinions along the way


• At its most basic: Optimize a factor you think is correlated with growth
AirBnB and Craigslist
Getsatisfaction extortion
Coveting GMail invites
Farmville and wall posts
The five Stages of Lean Analytics

                       The business you’re in

             E-    2-sided          Mobile   User-gen
                             SaaS                       Media
          commerce market            app      content
UGC model:
Reddit goes from links to
community

•   Stage: Virality
•   Model: UGC
•   A graduate of the first YCombinator class, reddit was acquired by Conde Nast
    but left largely to its own devices. Thanks to a vibrant community and some
    good guidance by its founders, it’s a traffic powerhouse.
Reddit goes from links to community

• Product evolution
   • Started as a simple link-sharing site with voting
   • Then added the ability to comment, with votes on comments
   • Then created the ability to make “self-posts” rather than only comment on off-
     site traffic
   • Now self-posts are more than half of all posts
Reddit goes from links to community

• Revenue from ads and “reddit gold”
   • Started as a joke, but turned into a revenue source
      • One person paid $1000 for a month; some paid $0.01. Avg. around $4.
   • Paying users get early access to features, since they’re an engaged beta
The five Stages of Lean Analytics

                                                   The business you’re in

                                         E-    2-sided          Mobile   User-gen
                                                         SaaS                       Media
                                      commerce market            app      content
                         Empathy
The stage you’re at




                                              One Metric
                        Stickiness

                           Virality

                         Revenue             That Matters.
                            Scale
Choose only one metric.
Yes, one metric.
It will soon change.
In a startup,
focus is hard to achieve.
Having only one metric
addresses this problem.
Metrics are like
      squeeze toys.




http://www.flickr.com/photos/connortarter/4791605202/
What’s your OMTM?
                 E-            2-sided                           Mobile           User-gen
                                                  SaaS                                             Media
              commerce         market                             app              content

 Empathy                       Interviews; qualitative results; quantitative scoring; surveys


              Loyalty,       Inventory,       Engagement, Downloads,             Content,       Traffic, visits,
Stickiness    conversion     listings         churn       churn, virality        spam           returns


              CAC, shares,                    Inherent        WoM, app           Invites,       Content
   Virality   reactivation
                           SEM, sharing
                                              virality, CAC   ratings, CAC       sharing        virality, SEM


               (Money from transactions)        (Money from active users)           (Money from ad clicks)

              Transaction,   Transactions,    Upselling,      CLV,               Ads,           CPE, affiliate
 Revenue      CLV            commission       CAC, CLV        ARPDAU             donations      %, eyeballs


              Affiliates,     Other            API, magic      Spinoffs,          Analytics,     Syndication,
    Scale     white-label    verticals        #, mktplace     publishers         user data      licenses
B2B: Selling to the enterprise
The B2B stereotype

• Domain expert knows
  industry and the problem
  domain. Has a Rolodex;
  proxy for customers.




                                                                http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html
• Disruption expert knows
  tech that will produce a
  change Sees beyond the
  current model.




                             Domain   Disruption
                             expert     expert     Operations
Three typical approaches

                    Create a popular consumer          Dropbox
 Enterprise pivot   product then pivot to tackle the
                    enterprise


                    Take an existing consumer or       Yammer,
 Copy and rebuild   open source idea and make it       MapR
                    enterprise-ready


                    Convince the enterprise to         Taleo,
Disrupt a problem   discard the old way because of     Google
                    overwhelming advantages.           Apps
Enterprise example:
Coradiant pivots from
service to appliance

•   Stage: Revenue
•   Model: Enterprise sale

•   Coradiant started as a research firm, then moved into managed services. As the
    market’s needs changed and data center preferences ossified, the company
    switched from services to a physical appliance for web performance
    management.
Coradiant pivots from service to
appliance

• Started as an MSP in colocated spaces, offering service and virtual
  infrastructure.
• Data center partners became competitors
• Talked to customers, who liked the monitoring interface and performance
  management
• Hibernated the company and turned the internal tool for monitoring web health
  (OutSight) into an appliance
   • MVP focused on the core value—what was actually happening on the wire
   • Reporting etc. was introduced as Excel exports initially
   • Made it easy to get data off the box to mitigate limited feature sets
• Scaled through channels and partnerships (Splunk, Akamai, etc.)
Lean Analytics lifecycle
for an enterprise-focused startup

     Stage                  Do this                      Fear this
                  Consulting to test ideas and         Lock-in, IP
    Empathy       bootstrap the business               control, overfitting

                  Standardization and integration;     Ability to
    Stickiness    shift from custom to generic         integrate; support

                  Word of mouth, references, case      Bad vibes;
     Virality     studies                              exclusivity

                  Growing direct sales, professional   Pipeline, revenue
    Revenue       services, support                    recognition, comp

                  Channels, analysts, ecosystems,      Crossing the
      Scale       APIs, vertically targeted products   chasm; Gorillas
The Zero Overhead principle

A central theme to this new wave of
innovation is the application of core product
tenets from the consumer space to the
enterprise.
In particular, a universal lesson that I keep
sharing with all entrepreneurs building for the
enterprise is the Zero Overhead Principle: no
feature may add training costs to the
user.                                        DJ Patil
Intrapreneur
Skunk Works for intrapreneurs	

• The Lockheed Martin Skunk Works
Span of control and the railroads

• Daniel C. McCallum
The BCG matrix

• How businesses think
  about products or                       Question marks! increase

                                                                   Pivot to 

                                                                                       Stars!
  companies                              (low market share, market
              (high growth rate,
                                                                    share

                                          high growth rate)
 through
            high market share)
                                        May be the next big thing. virality,
   What everyone wants. As
• Lean is about moving                  Consumes investment, but attention
      market invariably stops
                                            will require money to               growing, should become
  up and to the right    Growth rate
     increase market share.
                     cash cows.

                                                                                                   Milk with

                                                  Pivot to
                    Pivot to

                                                                                                    revenue

                                             redefine problem/
             increase growth

                                                                                                optimization as

                                              solution through
              rate through

                                                                                                 growth slows
                                                  empathy
                    disruption

                                                Dogs!                               Cash cows!
                                         (low market share,                     (high market share,
                                           low growth rate)
                      low growth rate)
                                         Barely breaks even, may                Boring sources of cash, to
                                        be a distraction from better             be milked but not worth
                                         opportunities. Sell off or               additional investment.
                                                shut down.
                                  




                                                                  Market share
Intrapreneur example:
P&G changes the mop
instead of the soap

•   Stage: Empathy
•   Model: Retail/consumer packaged goods

•   P&G is constantly looking for better soaps. But innovation was slowing.
    Frustrated, they hired a design team to help them.
P&G changes the mop
instead of the soap

• Heavy internal investment in R&D, but limited results
• Brought in an outside agency (Continuum) to help
• The team watched people as they mopped, recording and iterating their
  research approach


• Watched someone pick up spilled coffee. Rather than mopping, the person
  swept up with a broom, then wiped with a cloth
• Realized the mop, not the liquid, mattered
• Studied the makeup of floor dirt; realized much of it is dust


• Swiffer is a $500M innovation in a stalled industry
The Lean Analytics lifecycle
for an Intrapreneur
   Stage                  Do this                        Fear this
                 Get buy-in                             Political fallout
  Beforehand
                 Find problems; don’t test demand.      Entitled, aggrieved
   Empathy       Skip the business case, do analytics   customers

                 Know your real minimum based on        Hidden “must haves”,
  Stickiness     expectations, regulations              feature creep

                 Build inherent virality in from the    Luddites who don’t
    Virality     start; attention is the new currency   understand sharing

                 Consider the ecosystem, channels,      Channel conflict,
   Revenue       and established agreements             resistance, contracts

                 Hand the baton to others gracefully    Hating what happens
    Scale                                               to your baby
Metrics in practice:
The Lean Analytics Cycle

Success!                         Pick OMTM                Draw a line
                                                          in the sand
               Pivot or
               give up          Draw a new line             Find a
                                                           potential
                                    Try again            improvement

Did we move
the needle?                                         Without       With data:
                                                  data: make a      find a
                                                  good guess     commonality
                          Design a test
  Measure
 the results                                              Hypothesis
                          Make changes
                          in production
Virality stage:
Circle of Moms finds an
engaged market

•   Stage: Stickiness
•   Model: UGC

•   Launched as Circle of Friends in 2007, it was a way for small groups to interact
    atop Facebook’s platform; but when engagement wasn’t good enough, the
    founders decided to dig deeper.
The problem: Not enough engagement

• Too few people were actually using the product


• Less than 20% of any circles had any activity after their initial creation


• A few million monthly uniques from 10M registered users, but no sustained
  traction
What Circle of Moms found

• They found moms were far more engaged
   • Their messages to one another were on average 50% longer
   • They were 115% more likely to attach a picture to a post they wrote
   • They were 110% more likely to engage in a threaded (i.e. deep) conversation
   • Circle owners’ friends were 50% more likely to engage with the circle
   • They were 75% more likely to click on Facebook notifications
   • They were 180% more likely to click on Facebook news feed items
   • They were 60% more likely to accept invitations to the app
• Pivoted to the new market, including a name change
• By late 2009, 4.5M users and strong engagement
• Sold to Sugar, inc. in early 2012
Virality stage:
qidiq streamlines invites

•   Stage: Virality
•   Model: SaaS

•   Tool to poll small groups, built in the Year One Labs accelerator
Initial design                                          Redesigned workflow
                       Survey owner adds recipient to group                           Survey owner adds recipient to group




                                                               70-90% RESPONSE RATE
                           Survey owner asks question                                     Survey owner asks question

                               Recipient gets invite                                    Recipient reads survey question
10-25% RESPONSE RATE




                           Recipient installs mobile app                                 Recipient responds to question

                                                                                          Recipient sees survey results
                         Recipient creates account, profile

                        Recipient can edit profile, view past                                  (Later, if needed…)
                                  questions, etc.
                                                                                             Recipient visits website
                         Recipient reads survey question
                                                                                          Recipient has no password!
                          Recipient responds to question
                                                                                       Recipient does password recovery
                           Recipient sees survey results
                                                                                           One-time link sent to email

                                                                                          Recipient creates password

                                                                                       Recipient can edit profile, view past
                                                                                                 questions, etc.
“The most important figures that
one needs for management are
unknown or unknowable, but
successful management must
nevertheless take account of
them.”


                      Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
  HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others in
the absence of data.
Now, a leader knows what
questions to ask.
Ben Yoskovitz
byosko@gmail.com
@byosko




Alistair Croll
acroll@gmail.com
@acroll

More Related Content

What's hot

Metrics for early stage startups
Metrics for early stage startupsMetrics for early stage startups
Metrics for early stage startupsAndreas Klinger
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics
 
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다승화 양
 
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)Minwoo Kim
 
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )정혁 권
 
Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassLean Analytics
 
데이터가 흐르는 조직 만들기 - 마이리얼트립
데이터가 흐르는 조직 만들기 - 마이리얼트립데이터가 흐르는 조직 만들기 - 마이리얼트립
데이터가 흐르는 조직 만들기 - 마이리얼트립승화 양
 
올바른 분석을 방해하는 함정 카드 피해가기
올바른 분석을 방해하는 함정 카드 피해가기올바른 분석을 방해하는 함정 카드 피해가기
올바른 분석을 방해하는 함정 카드 피해가기Minho Lee
 
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)Trigger
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth HackerTim Homuth
 
Finding the one growth metric that matters
Finding the one growth metric that mattersFinding the one growth metric that matters
Finding the one growth metric that mattersSean Ellis
 
29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins29 Growth Hacking Quick Wins
29 Growth Hacking Quick WinsMattan Griffel
 
Data analytics as a service
Data analytics as a serviceData analytics as a service
Data analytics as a serviceStanley Wang
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash CourseDavid Fallarme
 
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)Growth Hacking Asia
 
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)Jaikwang Lee
 
Data Storytelling: The only way to unlock true insight from your data
Data Storytelling: The only way to unlock true insight from your dataData Storytelling: The only way to unlock true insight from your data
Data Storytelling: The only way to unlock true insight from your dataBright North
 
10 Principles for Data Storytelling
10 Principles for Data Storytelling10 Principles for Data Storytelling
10 Principles for Data StorytellingDamian Radcliffe
 
Startup Metrics for Pirates (FounderX, May 2016)
Startup Metrics for Pirates (FounderX, May 2016)Startup Metrics for Pirates (FounderX, May 2016)
Startup Metrics for Pirates (FounderX, May 2016)Dave McClure
 
From Mice to Whales – 5 Ways to Build a $100 Million SaaS Business
From Mice to Whales – 5 Ways to Build a $100 Million SaaS BusinessFrom Mice to Whales – 5 Ways to Build a $100 Million SaaS Business
From Mice to Whales – 5 Ways to Build a $100 Million SaaS BusinessChristoph Janz
 

What's hot (20)

Metrics for early stage startups
Metrics for early stage startupsMetrics for early stage startups
Metrics for early stage startups
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and Enterprises
 
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다
실전 스타트업 데이터분석: 소셜데이팅 이음은 이렇게 한다
 
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)
데이터 분석에 필요한 기본 개념: 지표, Funnel 등 데이터를 이해하기 위한 멘탈 모델(Mental Model)
 
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )
린분석 with 레진코믹스 ( Lean Analytics with Lezhin Comics )
 
Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclass
 
데이터가 흐르는 조직 만들기 - 마이리얼트립
데이터가 흐르는 조직 만들기 - 마이리얼트립데이터가 흐르는 조직 만들기 - 마이리얼트립
데이터가 흐르는 조직 만들기 - 마이리얼트립
 
올바른 분석을 방해하는 함정 카드 피해가기
올바른 분석을 방해하는 함정 카드 피해가기올바른 분석을 방해하는 함정 카드 피해가기
올바른 분석을 방해하는 함정 카드 피해가기
 
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)
12 Steps to Effective Growth Hacking (www.wepullthetrigger.com)
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth Hacker
 
Finding the one growth metric that matters
Finding the one growth metric that mattersFinding the one growth metric that matters
Finding the one growth metric that matters
 
29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins
 
Data analytics as a service
Data analytics as a serviceData analytics as a service
Data analytics as a service
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash Course
 
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)
Growth Hacking Fundamentals @ Echelon Jakarta (by Growth Hacking Asia)
 
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)
대용량 로그분석 Bigquery로 간단히 사용하기 (20170215 T아카데미)
 
Data Storytelling: The only way to unlock true insight from your data
Data Storytelling: The only way to unlock true insight from your dataData Storytelling: The only way to unlock true insight from your data
Data Storytelling: The only way to unlock true insight from your data
 
10 Principles for Data Storytelling
10 Principles for Data Storytelling10 Principles for Data Storytelling
10 Principles for Data Storytelling
 
Startup Metrics for Pirates (FounderX, May 2016)
Startup Metrics for Pirates (FounderX, May 2016)Startup Metrics for Pirates (FounderX, May 2016)
Startup Metrics for Pirates (FounderX, May 2016)
 
From Mice to Whales – 5 Ways to Build a $100 Million SaaS Business
From Mice to Whales – 5 Ways to Build a $100 Million SaaS BusinessFrom Mice to Whales – 5 Ways to Build a $100 Million SaaS Business
From Mice to Whales – 5 Ways to Build a $100 Million SaaS Business
 

Viewers also liked

Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Lean Analytics
 
プレゼンビフォアアフタ
プレゼンビフォアアフタプレゼンビフォアアフタ
プレゼンビフォアアフタTsuyoshi Ushio
 
Lean Analytics at Lean Startup Update!! 2015
Lean Analytics at Lean Startup Update!! 2015Lean Analytics at Lean Startup Update!! 2015
Lean Analytics at Lean Startup Update!! 2015Masanori Kado
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
 
とあるスタートアップの評価指標(メトリクス)
とあるスタートアップの評価指標(メトリクス)とあるスタートアップの評価指標(メトリクス)
とあるスタートアップの評価指標(メトリクス)Takaaki Umada
 
Build Less Patterns AgileRoots 2014
Build Less Patterns AgileRoots 2014Build Less Patterns AgileRoots 2014
Build Less Patterns AgileRoots 2014Tsuyoshi Ushio
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics
 
FlyData Autoload: 事例集
FlyData Autoload: 事例集FlyData Autoload: 事例集
FlyData Autoload: 事例集FlyData Inc.
 
Directory Write Leases in MagFS
Directory Write Leases in MagFSDirectory Write Leases in MagFS
Directory Write Leases in MagFSMaginatics
 
2014-01-28 Operation in the future
2014-01-28 Operation in the future2014-01-28 Operation in the future
2014-01-28 Operation in the futureOperation Lab, LLC.
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014Lean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
 
Getting Started with Performance Co-Pilot
Getting Started with Performance Co-PilotGetting Started with Performance Co-Pilot
Getting Started with Performance Co-PilotPaul V. Novarese
 
PyCon JP 2014 plone terada
PyCon JP 2014 plone teradaPyCon JP 2014 plone terada
PyCon JP 2014 plone teradaManabu Terada
 
SI業界の営業の役割と存在意義を一緒に考えよう
SI業界の営業の役割と存在意義を一緒に考えようSI業界の営業の役割と存在意義を一緒に考えよう
SI業界の営業の役割と存在意義を一緒に考えようManabu Terada
 
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」WebSig24/7
 
Pyconjp2014_implementations
Pyconjp2014_implementationsPyconjp2014_implementations
Pyconjp2014_implementationsmasahitojp
 
Site Search Analytics in a Nutshell
Site Search Analytics in a NutshellSite Search Analytics in a Nutshell
Site Search Analytics in a NutshellLouis Rosenfeld
 

Viewers also liked (20)

Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
 
プレゼンビフォアアフタ
プレゼンビフォアアフタプレゼンビフォアアフタ
プレゼンビフォアアフタ
 
Lean Analytics at Lean Startup Update!! 2015
Lean Analytics at Lean Startup Update!! 2015Lean Analytics at Lean Startup Update!! 2015
Lean Analytics at Lean Startup Update!! 2015
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
とあるスタートアップの評価指標(メトリクス)
とあるスタートアップの評価指標(メトリクス)とあるスタートアップの評価指標(メトリクス)
とあるスタートアップの評価指標(メトリクス)
 
Build Less Patterns AgileRoots 2014
Build Less Patterns AgileRoots 2014Build Less Patterns AgileRoots 2014
Build Less Patterns AgileRoots 2014
 
Cracking PRNG
Cracking PRNGCracking PRNG
Cracking PRNG
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
 
FlyData Autoload: 事例集
FlyData Autoload: 事例集FlyData Autoload: 事例集
FlyData Autoload: 事例集
 
Directory Write Leases in MagFS
Directory Write Leases in MagFSDirectory Write Leases in MagFS
Directory Write Leases in MagFS
 
2014-01-28 Operation in the future
2014-01-28 Operation in the future2014-01-28 Operation in the future
2014-01-28 Operation in the future
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
PCP
PCPPCP
PCP
 
Getting Started with Performance Co-Pilot
Getting Started with Performance Co-PilotGetting Started with Performance Co-Pilot
Getting Started with Performance Co-Pilot
 
PyCon JP 2014 plone terada
PyCon JP 2014 plone teradaPyCon JP 2014 plone terada
PyCon JP 2014 plone terada
 
SI業界の営業の役割と存在意義を一緒に考えよう
SI業界の営業の役割と存在意義を一緒に考えようSI業界の営業の役割と存在意義を一緒に考えよう
SI業界の営業の役割と存在意義を一緒に考えよう
 
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」
第29回WebSig会議「効率化だけではない!中小~中堅ECサイトの成果を上げる「メディア編集力」とは」
 
Pyconjp2014_implementations
Pyconjp2014_implementationsPyconjp2014_implementations
Pyconjp2014_implementations
 
Site Search Analytics in a Nutshell
Site Search Analytics in a NutshellSite Search Analytics in a Nutshell
Site Search Analytics in a Nutshell
 

Similar to OnLab Japan introduction to Lean Analytics

7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can HelpLean Analytics
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
Advanced Social Media Monitoring
Advanced Social Media MonitoringAdvanced Social Media Monitoring
Advanced Social Media MonitoringBlake Robinson
 
The Future of Innovative Marketing - Carole Lamarque
The Future of Innovative Marketing - Carole LamarqueThe Future of Innovative Marketing - Carole Lamarque
The Future of Innovative Marketing - Carole LamarqueCarole Lamarque
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungMattan Griffel
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking jeroentjepkema
 
Making the Right Decisions within a Continuously Changing Digital World - Google
Making the Right Decisions within a Continuously Changing Digital World - GoogleMaking the Right Decisions within a Continuously Changing Digital World - Google
Making the Right Decisions within a Continuously Changing Digital World - GoogleSavage Marketing
 
Rockstar Entrepreneur for RISE Austin
Rockstar Entrepreneur for RISE AustinRockstar Entrepreneur for RISE Austin
Rockstar Entrepreneur for RISE AustinLaura Capes
 
Top 10 Mistakes CMOs Will Make in 2013 - webinar
Top 10 Mistakes CMOs Will Make in 2013 - webinarTop 10 Mistakes CMOs Will Make in 2013 - webinar
Top 10 Mistakes CMOs Will Make in 2013 - webinarDemandWave
 
How to get MVP Right by Health Fidelity Sr Product Manager
How to get MVP Right by Health Fidelity Sr Product ManagerHow to get MVP Right by Health Fidelity Sr Product Manager
How to get MVP Right by Health Fidelity Sr Product ManagerProduct School
 
Oli Gardner SMD Warsaw 2014 - Advanced Landing Page Optimization With Conve...
Oli Gardner  SMD Warsaw 2014  - Advanced Landing Page Optimization With Conve...Oli Gardner  SMD Warsaw 2014  - Advanced Landing Page Optimization With Conve...
Oli Gardner SMD Warsaw 2014 - Advanced Landing Page Optimization With Conve...Joanna Gęsicka
 
Online Marketing and Metrics Presentation at UEIA, 2012
Online Marketing and Metrics Presentation at UEIA, 2012Online Marketing and Metrics Presentation at UEIA, 2012
Online Marketing and Metrics Presentation at UEIA, 2012Justo Hidalgo
 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Andreas Klinger
 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS MetricsChristoph Janz
 
Optimizely Experience Chicago - Oli Gardner, Unbounce
Optimizely Experience Chicago - Oli Gardner, UnbounceOptimizely Experience Chicago - Oli Gardner, Unbounce
Optimizely Experience Chicago - Oli Gardner, UnbounceOptimizely
 
Advanced Content Creation, SEO & Storytelling
Advanced Content Creation, SEO & StorytellingAdvanced Content Creation, SEO & Storytelling
Advanced Content Creation, SEO & StorytellingCasey Armstrong
 
Reporting! Reporting! Reporting!
Reporting! Reporting! Reporting!Reporting! Reporting! Reporting!
Reporting! Reporting! Reporting!Kerry Dean SEO
 

Similar to OnLab Japan introduction to Lean Analytics (20)

7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Advanced Social Media Monitoring
Advanced Social Media MonitoringAdvanced Social Media Monitoring
Advanced Social Media Monitoring
 
The Future of Innovative Marketing - Carole Lamarque
The Future of Innovative Marketing - Carole LamarqueThe Future of Innovative Marketing - Carole Lamarque
The Future of Innovative Marketing - Carole Lamarque
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-Young
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
 
Making the Right Decisions within a Continuously Changing Digital World - Google
Making the Right Decisions within a Continuously Changing Digital World - GoogleMaking the Right Decisions within a Continuously Changing Digital World - Google
Making the Right Decisions within a Continuously Changing Digital World - Google
 
Rockstar Entrepreneur for RISE Austin
Rockstar Entrepreneur for RISE AustinRockstar Entrepreneur for RISE Austin
Rockstar Entrepreneur for RISE Austin
 
Digi Marketing The Recession Breaker
Digi Marketing  The  Recession  BreakerDigi Marketing  The  Recession  Breaker
Digi Marketing The Recession Breaker
 
Top 10 Mistakes CMOs Will Make in 2013 - webinar
Top 10 Mistakes CMOs Will Make in 2013 - webinarTop 10 Mistakes CMOs Will Make in 2013 - webinar
Top 10 Mistakes CMOs Will Make in 2013 - webinar
 
How to get MVP Right by Health Fidelity Sr Product Manager
How to get MVP Right by Health Fidelity Sr Product ManagerHow to get MVP Right by Health Fidelity Sr Product Manager
How to get MVP Right by Health Fidelity Sr Product Manager
 
Oli Gardner SMD Warsaw 2014 - Advanced Landing Page Optimization With Conve...
Oli Gardner  SMD Warsaw 2014  - Advanced Landing Page Optimization With Conve...Oli Gardner  SMD Warsaw 2014  - Advanced Landing Page Optimization With Conve...
Oli Gardner SMD Warsaw 2014 - Advanced Landing Page Optimization With Conve...
 
Online Marketing and Metrics Presentation at UEIA, 2012
Online Marketing and Metrics Presentation at UEIA, 2012Online Marketing and Metrics Presentation at UEIA, 2012
Online Marketing and Metrics Presentation at UEIA, 2012
 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics
 
Optimizely Experience Chicago - Oli Gardner, Unbounce
Optimizely Experience Chicago - Oli Gardner, UnbounceOptimizely Experience Chicago - Oli Gardner, Unbounce
Optimizely Experience Chicago - Oli Gardner, Unbounce
 
Advanced Content Creation, SEO & Storytelling
Advanced Content Creation, SEO & StorytellingAdvanced Content Creation, SEO & Storytelling
Advanced Content Creation, SEO & Storytelling
 
Reporting! Reporting! Reporting!
Reporting! Reporting! Reporting!Reporting! Reporting! Reporting!
Reporting! Reporting! Reporting!
 
Changing Content, Changing Culture
Changing Content, Changing CultureChanging Content, Changing Culture
Changing Content, Changing Culture
 

More from Lean Analytics

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsLean Analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean Analytics
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)Lean Analytics
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Lean Analytics
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine MontrealLean Analytics
 

More from Lean Analytics (7)

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine Montreal
 

Recently uploaded

How The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersHow The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersFlyyx Tech
 
Importance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxImportance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxBonano Insurance
 
Presented by Sabri international .......
Presented by Sabri international .......Presented by Sabri international .......
Presented by Sabri international .......SABRI INTERNATIONAL
 
Young Woman Entrepreneur - Kaviya Cherian
Young Woman Entrepreneur - Kaviya CherianYoung Woman Entrepreneur - Kaviya Cherian
Young Woman Entrepreneur - Kaviya CherianCDEEPANVITA
 
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)CXO 2.0 Conference
 
NVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxNVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxKrutik Rakade
 
HOW TO START EARNING WITH AFFILIATE MARKETING
HOW TO START EARNING WITH AFFILIATE MARKETINGHOW TO START EARNING WITH AFFILIATE MARKETING
HOW TO START EARNING WITH AFFILIATE MARKETINGNATHAN SPEAKS
 
The 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfThe 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfInsightsSuccess4
 
Optimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondOptimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondBoundify
 
AirOxi - Pioneering Aquaculture Advancements Through NFDB Empanelment.pptx
AirOxi -  Pioneering Aquaculture Advancements Through NFDB Empanelment.pptxAirOxi -  Pioneering Aquaculture Advancements Through NFDB Empanelment.pptx
AirOxi - Pioneering Aquaculture Advancements Through NFDB Empanelment.pptxAirOxi Tube
 
EPC Contractors aspects Presentation.pdf
EPC Contractors  aspects Presentation.pdfEPC Contractors  aspects Presentation.pdf
EPC Contractors aspects Presentation.pdfGiuseppe Tommasone
 
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdf
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdfCORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdf
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdfLouis Malaybalay
 
Mist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in EgyptMist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in Egyptopstechsanjanasingh
 
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024The Vietnam Believer_Newsletter_Vol.001_Mar12 2024
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024believeminhh
 
Wallet Pitch for startup fintech and loan
Wallet Pitch for startup fintech and loanWallet Pitch for startup fintech and loan
Wallet Pitch for startup fintech and loansujat8807
 
Green Innovations: Wristbands Ireland's Eco-Friendly Products
Green Innovations: Wristbands Ireland's Eco-Friendly ProductsGreen Innovations: Wristbands Ireland's Eco-Friendly Products
Green Innovations: Wristbands Ireland's Eco-Friendly ProductsWristbands Ireland
 
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...BilalAhmed717
 
L-1 VISA Business (Plan Sample) - Plan Writers
L-1 VISA Business (Plan Sample) - Plan WritersL-1 VISA Business (Plan Sample) - Plan Writers
L-1 VISA Business (Plan Sample) - Plan WritersPlan Writers
 

Recently uploaded (20)

How The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersHow The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K Subscribers
 
Importance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxImportance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptx
 
Presented by Sabri international .......
Presented by Sabri international .......Presented by Sabri international .......
Presented by Sabri international .......
 
Young Woman Entrepreneur - Kaviya Cherian
Young Woman Entrepreneur - Kaviya CherianYoung Woman Entrepreneur - Kaviya Cherian
Young Woman Entrepreneur - Kaviya Cherian
 
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)
CXO 2.0 Conference (Event Information Deck | Dec'24-Mar'25)
 
WAM Corporate Presentation Mar 12 2024_Video.pdf
WAM Corporate Presentation Mar 12 2024_Video.pdfWAM Corporate Presentation Mar 12 2024_Video.pdf
WAM Corporate Presentation Mar 12 2024_Video.pdf
 
NVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxNVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptx
 
HOW TO START EARNING WITH AFFILIATE MARKETING
HOW TO START EARNING WITH AFFILIATE MARKETINGHOW TO START EARNING WITH AFFILIATE MARKETING
HOW TO START EARNING WITH AFFILIATE MARKETING
 
The 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfThe 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdf
 
Optimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondOptimize Your CRM Customization and Beyond
Optimize Your CRM Customization and Beyond
 
AirOxi - Pioneering Aquaculture Advancements Through NFDB Empanelment.pptx
AirOxi -  Pioneering Aquaculture Advancements Through NFDB Empanelment.pptxAirOxi -  Pioneering Aquaculture Advancements Through NFDB Empanelment.pptx
AirOxi - Pioneering Aquaculture Advancements Through NFDB Empanelment.pptx
 
EPC Contractors aspects Presentation.pdf
EPC Contractors  aspects Presentation.pdfEPC Contractors  aspects Presentation.pdf
EPC Contractors aspects Presentation.pdf
 
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdf
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdfCORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdf
CORPORATE SOCIAL RESPONSIBILITY - FINAL REQUIREMENT.pdf
 
WAM Corporate Presentation Mar 12 2024.pdf
WAM Corporate Presentation Mar 12 2024.pdfWAM Corporate Presentation Mar 12 2024.pdf
WAM Corporate Presentation Mar 12 2024.pdf
 
Mist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in EgyptMist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in Egypt
 
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024The Vietnam Believer_Newsletter_Vol.001_Mar12 2024
The Vietnam Believer_Newsletter_Vol.001_Mar12 2024
 
Wallet Pitch for startup fintech and loan
Wallet Pitch for startup fintech and loanWallet Pitch for startup fintech and loan
Wallet Pitch for startup fintech and loan
 
Green Innovations: Wristbands Ireland's Eco-Friendly Products
Green Innovations: Wristbands Ireland's Eco-Friendly ProductsGreen Innovations: Wristbands Ireland's Eco-Friendly Products
Green Innovations: Wristbands Ireland's Eco-Friendly Products
 
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...
Project Work on Consumer Behavior in Fast Food Restaurants. Their behavior to...
 
L-1 VISA Business (Plan Sample) - Plan Writers
L-1 VISA Business (Plan Sample) - Plan WritersL-1 VISA Business (Plan Sample) - Plan Writers
L-1 VISA Business (Plan Sample) - Plan Writers
 

OnLab Japan introduction to Lean Analytics

  • 1. Lean Analytics Use data to build a better business faster. www.leananalyticsbook.com @byosko | @acroll @leananalytics
  • 3. It just hates it less than the alternative:
  • 5. Most startups don’t know what they’ll be when they grow up. Freshbooks Mitel was invoicing Wikipedia was a Paypal for a web was to be lawnmower first built for company design firm written by Palmpilots experts only Flickr Hotmail Twitter Autodesk was going to was a was a made desktop be an MMO database podcasting automation company company
  • 7. Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
  • 8. Analytics is the measurement of movement towards your business goals. http://www.flickr.com/photos/itsgreg/446061432/
  • 9. Small business example: Solare watches the numbers • Stage: Revenue • Model: Retailer • Solare is an Italian fine-dining restaurant under new management. The new team is trying to identify the key metrics and leading indicators
  • 10. Solare watches the numbers • A line in the sand: Gross Revenue to Labor Cost • Under 30% is good • Below 24% is great • Lower than 20% and you may be under-staffing, leading to dissatisfied customers • A leading indicator: Total covers is 5x reservations at 5PM • If you have 50 reservations at 5, you’ll have 250 covers that night. • This ratio varies by restaurant.
  • 11. In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.
  • 12. Qualitative or Quantitative 5 things you Exploratory or Reporting need to know Vanity or Actionable about metrics Correlated or Causal Leading or Lagging
  • 13. Qualitative Quantitative Unstructured, Numbers and stats; anecdotal, hard facts but less revealing, hard to insight. aggregate. Warm and fuzzy. Cold and hard. http://www.flickr.com/photos/zooboing/8388257248/ http://www.flickr.com/photos/x1brett/4665645157/
  • 14. Simply: you can’t count smiles. Discover qualitatively, prove quantitatively. Qualitative is inspiration, quantitative is verification.
  • 15. Exploratory Reporting Speculative, trying Predictable, keeping to find unexpected you abreast of or interesting normal, managerial insights. operations. http://www.flickr.com/photos/50755773@N06/5415295449/ http://www.flickr.com/photos/elwillo/4737933662/
  • 16. Donald Rumsfeld on analytics Are facts which may be wrong and we know should be checked against data. know we don’t Are questions we can answer by reporting, which we should baseline know & automate. Things we Are intuition which we should we know quantify and teach to improve don’t effectiveness, efficiency. know we don’t Are exploration which is where unfair advantage and interesting know epiphanies live. (Or rather, Avinash Kaushik channeling Rumsfeld)
  • 17. Vanity Actionable Picks a direction. Makes you feel good, but doesn’t change how you’ll act. http://www.flickr.com/photos/lostseouls/807253220/ http://www.flickr.com/photos/aussiegall/6382775153/
  • 18. A metric from the early, foolish days of the Web. Hits Count people instead. Marginally better than hits. Unless you’re displaying Page views ad inventory, count people. Is this one person visiting a hundred times, or are a Visits hundred people visiting once? Fail. This tells you nothing about what they did, why they Unique visitors stuck around, or if they left. Followers/ Count actions instead. Find out how many followers friends/likes will do your bidding. Time on site, or Poor version of engagement. Lots of time spent on pages/visit support pages is actually a bad sign. How many recipients will act on what’s in them? Emails collected Number of Outside app stores, downloads alone don’t lead to downloads lifetime value. Measure activations/active accounts.
  • 19. 2-sided market model: AirBnB and photography • Stage: Revenue • Model: 2-sided marketplace • Rental-by-owner marketplace that allows property owners to list and market their houses. Offers a variety of related services as well.
  • 20. AirBnB tests a hypothesis • The hypothesis: “Hosts with professional photography will get more business. And hosts will sign up for professional photography as a service.” • Built a concierge MVP • Found that professionally photographed listings got 2-3x more bookings than the market average. • In mid-to-late 2011, AirBnB had 20 photographers in the field taking pictures for hosts.
  • 21. NIGHTS BOOKED 10 million 8 million 6 million 20 photographers 4 million 2 million 2008 2009 2010 2011 2012
  • 22. http://www.flickr.com/photos/circasassy/7858155676/ it’s a how you behave, If it won’t change bad metric.
  • 23. A few words on causality. http://www.flickr.com/photos/roryfinneren/65729247
  • 24. 50 37.5 25 12.5 0 1 2 3 4 5 6 7 8 9 10 Seat rentals http://www.rvca.com/anp/wp-content/plugins/wp-o-matic/cache/57226_07+proof+1a+hb+beach+day.jpg
  • 27. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption Drownings
  • 31. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption Drownings Temperature
  • 35. Correlated Causal Two variables that An independent change in similar factor that directly ways , perhaps impacts a because they’re dependent one. linked to something else. Summer al Ca us us Ca Correlated al Drowning Ice cream consumption
  • 37. Causality is a superpower, because it lets you change the future. Correlation lets you Causality lets you predict the future change the future “I will have 420 “If I can make more engaged users and first-time visitors stay 75 paying customers on for 17 minutes I next month.” will increase sales in 90 days.” Optimize the Find correlation Test causality causal factor
  • 38. Leading Lagging Number today that Historical metric that shows metric shows how you’re tomorrow—makes doing—reports the the news. news.
  • 39. A leading indicator for e-commerce How many of your customers Then you are in Your customers You are just Focus on buy a second this mode will buy from you like time in 90 days? Low CAC, 1-15% Acquisition Once 70% high of retailers checkout 15-30% Hybrid 2-2.5 20% Increasing per year of retailers returns Loyalty, >30% Loyalty >2.5 10% inventory per year of retailers expansion (Thanks to Kevin Hilstrom for this.)
  • 40. Think about a car •60MPH is twice as fast as 30MPH •Speed limits and mileage are well understood •Kilometres are conveniently decimal; miles map to hours •Ratios everywhere •Miles travelled is good; •Miles per hour is better; •Accelerating or decelerating changes your gas pedal •Custom metrics: “MPH divided by speeding tickets” as a metric of “driving fast without losing my license”
  • 41. The Lean Analytics Framework.
  • 42. Eric Ries’ Three engines Stickiness Virality Price Approach Keep people Make people Spend revenue coming back. invite friends. getting customers. Math that Get customers How many they Customers are matters faster than you tell, how fast worth more than lose them. they tell them. they cost to get.
  • 43. Dave McClure’s Pirate metrics How do your users become aware of you? Acquisition AARRR SEO, SEM, widgets, email, PR, campaigns, blogs ... Do drive-by visitors subscribe, use, etc? Activation Features, design, tone, compensation, affirmation ... Does a one-time user become engaged? Retention Notifications, alerts, reminders, emails, updates... Do you make money from user activity? Revenue Transactions, clicks, subscriptions, DLC, analytics... Do users promote your product? Referral Email, widgets, campaigns, likes, RTs, affiliates...
  • 44. The five Stages of Lean Analytics Empathy The stage you’re at Stickiness Virality Revenue Scale
  • 45. Example: a restaurant • Empathy: Before opening, the owner first learns about the diners in its area, their desires, what foods aren’t available, and trends in eating. • Stickiness: Then he develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. He’s giving things away, testing things, asking diners what they think. Costs are high because of variance and uncertain inventory. • Virality: He starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. He engages on Yelp and Foursquare. • Revenue: With virality kicked off, he works on margins—fewer free meals, tighter controls on costs, more standardization. • Scale: Finally, knowing he can run a profitable business, he pours some of the revenues into marketing and promotion. He reaches out to food reviewers, travel magazines, and radio stations. He launches a second restaurant, or a franchise based on the initial one.
  • 46. Example: a software company • Empathy: The founder finds an unmet need, often because she has a background in a particular industry or has worked with existing solutions that are being disrupted. • Stickiness: She meets with an initial group of prospects, and signs contracts that look more like consulting agreements, which she uses to build an initial product. She’s careful not to commit to exclusivity, and tries to steer customers towards standardized solutions, charging heavily for custom features. She supports the customers directly from the engineering team until the product is stable and usable. • Virality: Product in hand, she asks for references from satisfied customers, and uses them as testimonials. She starts direct sales, and grows the customer base. She launches a user group, and starts to automate support. She releases an API, encouraging third-party development and scaling potential market size without direct development. • Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored based on anticipated payoff and development cost. Recurring license and support revenue becomes an increasingly large component of overall revenues. • Scale: She signs deals with large distributors, and works with global consulting firms to have them deploy and integrate her tool. She attends trade shows to collect leads, carefully measuring cost of acquisition against close rate and lead value.
  • 48. Mobile app model: Localmind hacks Twitter • Stage: Empathy • Model: UGC/mobile • Real-time question and answer platform tied to locations. • Needed to find out if a core behavior—answering questions about a place— happened enough to make the business real
  • 49. Localmind hacks Twitter • Before writing a line of code, Localmind was concerned that people would never answer questions. • This was their biggest risk: if questions went unanswered users would have a terrible experience and stop using Localmind. • Ran an experiment on Twitter • Tracked geolocated tweets in Times Square • Sent @ messages to people who had just tweeted, asking questions about the area: how busy is it; is the subway running on time; is something open; etc. • The response rate to their tweeted questions was very high. • Good enough proxy to de-risk the solution, and convince the team and investors that it was worth building Localmind.
  • 50. When it’s time to move on • Have you conducted enough quality customer interviews to feel confident that I’ve found a problem worth solving? • Do you understand your customer well enough? • Do you believe your solution will meet the needs of customers?
  • 51. Stickiness The dogs like the dogfood.
  • 52. 1995 Hits 1997 Visits 1999 Visitors 2002 Conversions Who did you add? Where from? Why? 2010 Engagement What did they do? How did it benefit? Who did you lose? Why did they leave?
  • 53. Stickiness stage: WP Engine discovers the 2% cancellation rate • Stage: Stickiness • Model: SaaS • Wordpress hosting company founded in July 2010, it raised $1.2M in November 2011
  • 54. WP-Engine discovers the 2% cancellation rate • All companies have cancellations, but founder Jason Cohen was alarmed that he was losing a quarter of customers every year. • Jason called customers himself. “Not everyone wanted to speak with me, but enough people were willing to talk, even after they had left, that I learned a lot about why they were leaving.” • Asked around. Turns out 2% is best case for most hosting companies. • Without this, the company would have been getting diminishing returns over- optimizing churn; instead, they could focus on maximizing revenues or lowering acquisition costs.
  • 55. When it’s time to move on • Are people using the product as expected? • Define an active user. What percentage of your users/customers is active? Write this down. Could this be higher? What can you do to improve engagement? • Evaluate your feature roadmap against the 7 questions to ask before building more features. Does this change the priorities of feature development? • Evaluate the complaints you’re getting from users. How does this impact feature development going forward?
  • 57. Virality stage: Timehop’s content sharing • Stage: Virality • Model: Mobile app • Social network around the past • Focused on virality (but not necessarily the coefficient!)
  • 58. The one metric that matters: content sharing • Focused on percent of daily active users that share their content • Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else— is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
  • 59. 3 kinds of virality • Inherent virality is built into the product, and happens as a function of use. • Artificial virality is forced, and often built into a reward system. • Word of mouth virality is simply conversations generated by satisfied users.
  • 60. When it’s time to move on • Are you using one of the three types of virality (inherent, artificial, word of mouth) for your startup? Describe how. If virality is a weak aspect of your startup, write down 3-5 ideas for how you could build more virality into your product. • What’s your viral coefficient? Even if it’s below 1 (which it likely is), do you feel like the virality that exists is good enough to help sustain growth and lower customer acquisition costs? • What’s your viral cycle time? How could you speed it up?
  • 61. Revenue Pour some of the money back into acquisition.
  • 62. Revenue stage: Backupify’s customer lifecycle • Stage: Revenue • Model: SaaS • Leading backup provider for cloud based data. • The company was founded in 2008 by Robert May and Vik Chadha • Has gone on to raise $19.5M in several rounds of financing.
  • 63. Shifting to Customer Acquisition Payback as a key metric • Initially focused on site visitors • Then focused on trials • Then switched to signups • Today, MRR • In early 2010, CAC was $243 and ARPU was only $39 • Pivoted to target business users • CLV-to-CAC today is 5-6x • Now they track Customer Acquisition Payback • Target is less than 12 months
  • 64. Sergio Zyman’s many “mores” • If you’re dependent on physical, per-transaction costs (like direct sales, or shipping products to a buyer, or signing up merchants) then more efficiently will figure prominently on either the supply or demand side of your business model. • If you’ve found a high viral coefficient, then more people makes sense, because you’ve got a strong force multiplier added to every dollar you pour into customer acquisition. • If you’ve got a loyal, returning set of customers who buy from you every time, then more often makes sense, and you’re going to emphasize getting them to come back more frequently. • If you’ve got a one-time, big-ticket transaction, then more money will help a lot, because you’ve only got one chance to extract revenue from the customer and need to leave as little money as possible on the table. • If you’re a subscription model, and you’re fighting churn, then upselling customers to higher-capacity packages with broader features to additional subscribers within their organization is your best way of growing existing revenues, so you’ll spend a lot of time on more stuff.
  • 65. When it’s time to move on You’re making money You’re sustainable You’re tracking growth metrics
  • 66. Scale Let others grow your business for you.
  • 67. The hole in the middle Differentiation Efficiency Apple Costco Here be dragons Your local sustainable gluten-free cupcake shop Niche
  • 68. Guerrilla Data- marketing driven learning GROWTH HACKING Subversiveness
  • 69. The trust equation Your expertise & Do what you Are you one of reputation; say; say what us? Morality & certifications you do. personality. Credibility + Reliability + Intimacy Trustworthiness = Self-orientation What’s in it for you? Maister, Green and Galford
  • 70. • A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) • If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) • A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) • Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) • A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (These are also great segments to analyze.) (from the 2012 Growth Hacking conference)
  • 71. The growth hack • Growth hacking is simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way • At its most basic: Optimize a factor you think is correlated with growth
  • 76. The five Stages of Lean Analytics The business you’re in E- 2-sided Mobile User-gen SaaS Media commerce market app content
  • 77. UGC model: Reddit goes from links to community • Stage: Virality • Model: UGC • A graduate of the first YCombinator class, reddit was acquired by Conde Nast but left largely to its own devices. Thanks to a vibrant community and some good guidance by its founders, it’s a traffic powerhouse.
  • 78. Reddit goes from links to community • Product evolution • Started as a simple link-sharing site with voting • Then added the ability to comment, with votes on comments • Then created the ability to make “self-posts” rather than only comment on off- site traffic • Now self-posts are more than half of all posts
  • 79. Reddit goes from links to community • Revenue from ads and “reddit gold” • Started as a joke, but turned into a revenue source • One person paid $1000 for a month; some paid $0.01. Avg. around $4. • Paying users get early access to features, since they’re an engaged beta
  • 80. The five Stages of Lean Analytics The business you’re in E- 2-sided Mobile User-gen SaaS Media commerce market app content Empathy The stage you’re at One Metric Stickiness Virality Revenue That Matters. Scale
  • 81. Choose only one metric.
  • 83. It will soon change.
  • 84. In a startup, focus is hard to achieve.
  • 85. Having only one metric addresses this problem.
  • 86. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 87. What’s your OMTM? E- 2-sided Mobile User-gen SaaS Media commerce market app content Empathy Interviews; qualitative results; quantitative scoring; surveys Loyalty, Inventory, Engagement, Downloads, Content, Traffic, visits, Stickiness conversion listings churn churn, virality spam returns CAC, shares, Inherent WoM, app Invites, Content Virality reactivation SEM, sharing virality, CAC ratings, CAC sharing virality, SEM (Money from transactions) (Money from active users) (Money from ad clicks) Transaction, Transactions, Upselling, CLV, Ads, CPE, affiliate Revenue CLV commission CAC, CLV ARPDAU donations %, eyeballs Affiliates, Other API, magic Spinoffs, Analytics, Syndication, Scale white-label verticals #, mktplace publishers user data licenses
  • 88. B2B: Selling to the enterprise
  • 89. The B2B stereotype • Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers. http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html • Disruption expert knows tech that will produce a change Sees beyond the current model. Domain Disruption expert expert Operations
  • 90. Three typical approaches Create a popular consumer Dropbox Enterprise pivot product then pivot to tackle the enterprise Take an existing consumer or Yammer, Copy and rebuild open source idea and make it MapR enterprise-ready Convince the enterprise to Taleo, Disrupt a problem discard the old way because of Google overwhelming advantages. Apps
  • 91. Enterprise example: Coradiant pivots from service to appliance • Stage: Revenue • Model: Enterprise sale • Coradiant started as a research firm, then moved into managed services. As the market’s needs changed and data center preferences ossified, the company switched from services to a physical appliance for web performance management.
  • 92. Coradiant pivots from service to appliance • Started as an MSP in colocated spaces, offering service and virtual infrastructure. • Data center partners became competitors • Talked to customers, who liked the monitoring interface and performance management • Hibernated the company and turned the internal tool for monitoring web health (OutSight) into an appliance • MVP focused on the core value—what was actually happening on the wire • Reporting etc. was introduced as Excel exports initially • Made it easy to get data off the box to mitigate limited feature sets • Scaled through channels and partnerships (Splunk, Akamai, etc.)
  • 93. Lean Analytics lifecycle for an enterprise-focused startup Stage Do this Fear this Consulting to test ideas and Lock-in, IP Empathy bootstrap the business control, overfitting Standardization and integration; Ability to Stickiness shift from custom to generic integrate; support Word of mouth, references, case Bad vibes; Virality studies exclusivity Growing direct sales, professional Pipeline, revenue Revenue services, support recognition, comp Channels, analysts, ecosystems, Crossing the Scale APIs, vertically targeted products chasm; Gorillas
  • 94. The Zero Overhead principle A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil
  • 96. Skunk Works for intrapreneurs • The Lockheed Martin Skunk Works
  • 97. Span of control and the railroads • Daniel C. McCallum
  • 98. The BCG matrix • How businesses think about products or Question marks! increase
 Pivot to 
 Stars! companies (low market share, market
 (high growth rate, share
 high growth rate) through
 high market share) May be the next big thing. virality,
 What everyone wants. As • Lean is about moving Consumes investment, but attention market invariably stops will require money to growing, should become up and to the right Growth rate increase market share. cash cows. Milk with
 Pivot to
 Pivot to
 revenue
 redefine problem/
 increase growth
 optimization as
 solution through
 rate through
 growth slows empathy disruption Dogs! Cash cows! (low market share, (high market share, low growth rate) low growth rate) Barely breaks even, may Boring sources of cash, to be a distraction from better be milked but not worth opportunities. Sell off or additional investment. shut down. Market share
  • 99. Intrapreneur example: P&G changes the mop instead of the soap • Stage: Empathy • Model: Retail/consumer packaged goods • P&G is constantly looking for better soaps. But innovation was slowing. Frustrated, they hired a design team to help them.
  • 100. P&G changes the mop instead of the soap • Heavy internal investment in R&D, but limited results • Brought in an outside agency (Continuum) to help • The team watched people as they mopped, recording and iterating their research approach • Watched someone pick up spilled coffee. Rather than mopping, the person swept up with a broom, then wiped with a cloth • Realized the mop, not the liquid, mattered • Studied the makeup of floor dirt; realized much of it is dust • Swiffer is a $500M innovation in a stalled industry
  • 101. The Lean Analytics lifecycle for an Intrapreneur Stage Do this Fear this Get buy-in Political fallout Beforehand Find problems; don’t test demand. Entitled, aggrieved Empathy Skip the business case, do analytics customers Know your real minimum based on Hidden “must haves”, Stickiness expectations, regulations feature creep Build inherent virality in from the Luddites who don’t Virality start; attention is the new currency understand sharing Consider the ecosystem, channels, Channel conflict, Revenue and established agreements resistance, contracts Hand the baton to others gracefully Hating what happens Scale to your baby
  • 102. Metrics in practice: The Lean Analytics Cycle Success! Pick OMTM Draw a line in the sand Pivot or give up Draw a new line Find a potential Try again improvement Did we move the needle? Without With data: data: make a find a good guess commonality Design a test Measure the results Hypothesis Make changes in production
  • 103. Virality stage: Circle of Moms finds an engaged market • Stage: Stickiness • Model: UGC • Launched as Circle of Friends in 2007, it was a way for small groups to interact atop Facebook’s platform; but when engagement wasn’t good enough, the founders decided to dig deeper.
  • 104. The problem: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction
  • 105. What Circle of Moms found • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  • 106. Virality stage: qidiq streamlines invites • Stage: Virality • Model: SaaS • Tool to poll small groups, built in the Year One Labs accelerator
  • 107. Initial design Redesigned workflow Survey owner adds recipient to group Survey owner adds recipient to group 70-90% RESPONSE RATE Survey owner asks question Survey owner asks question Recipient gets invite Recipient reads survey question 10-25% RESPONSE RATE Recipient installs mobile app Recipient responds to question Recipient sees survey results Recipient creates account, profile Recipient can edit profile, view past (Later, if needed…) questions, etc. Recipient visits website Recipient reads survey question Recipient has no password! Recipient responds to question Recipient does password recovery Recipient sees survey results One-time link sent to email Recipient creates password Recipient can edit profile, view past questions, etc.
  • 108. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 109. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 110. ARCHIMEDES HAD TAKEN BATHS BEFORE.
  • 111. Once, a leader convinced others in the absence of data.
  • 112. Now, a leader knows what questions to ask.