Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Measure What Matters: Focus on the One Metric That Matters
Similar to Measure What Matters: Focus on the One Metric That Matters (20)
More from Lean Analytics
More from Lean Analytics (14)
Recently uploaded
Recently uploaded (20)
Measure What Matters: Focus on the One Metric That Matters
- 1. (or flail around blindly...it’s up to you) Measure What Matters Ben Yoskovitz @byosko
- 2. I am a product guy entrepreneur blogger angel investor http://instigatorblog.com
- 3. What I’ve learned It’s easy to get zombified. Recruitment sucks. Startup accelerators are fun. Eventually you get it right. TBD.
- 4. I’m not an analytics expert.
- 5. Or a data scientist.
- 6. http://leananalyticsbook.com But I wrote a book on analytics anyway.
- 7. Why did I write it? h"p://www.flickr.com/photos/horiavarlan/4290549806/sizes/l/in/photostream/
- 9. The basics of Lean Startup Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
- 10. What I Hope You Get From It h"p://www.flickr.com/photos/romtomtom/4382603005/sizes/o/in/photostream/
- 11. Follow the Lean model, and it becomes increasingly hard to lie, especially to yourself. The importance of intellectual honesty
- 12. Using your gut properly Instincts are experiments. Data is proof.
- 13. Better decision making abilities Everyone has data, the key is figuring out what pieces will improve your learning and decision making.
- 14. Focus Don’t chase shiny objects. You might succeed without focus, but it’ll be by accident.
- 15. Measure what matters 1 What makes a good metric? 2 Types of metrics 3 Analytical superpowers 4 Lean Analytics framework 5 The One Metric That Matters 6 Lean Analytics Cycle
- 16. What Makes a Good Metric? h"p://www.flickr.com/photos/artnoose/2263480871/sizes/l/in/photostream/
- 17. Analytics is the measurement of movement towards your business goals. What is analytics?
- 22. If it won’t change how you behave, it’s a bad metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
- 24. Warm and fuzzy. Cold and hard. Unstructured, anecdotal, revealing, hard to aggregate. Numbers and stats; hard facts but less insight. Qualitative vs. Quantitative
- 25. Discover qualitatively. Prove quantitatively. Quantitative vs. qualitative data
- 26. Do hosts with professional photography get more business? Airbnb experiments...
- 27. Professional photography helps Airbnb’s business Gut instinct Concierge MVP 20 photographers in the field Test results Two to three times more bookings! Back to the beginning Use additional data to keep experimenting
- 28. 5,000 shoots / month in Feb. 2012
- 29. Makes you feel good but doesn’t change how you’ll act. Helps you pick a direction and change your behavior. “Up and to the right.” These are good. Vanity Actionable vs.
- 30. Hits A metric from the early, foolish days of the Web. Count people instead. Page views Marginally better than hits. Unless you’re displaying ad inventory, count people. Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail. Users This tells you nothing about what they did, why they stuck around, or if they left. Followers/ friends/likes Count actions instead. Find out how many followers will do your bidding. Logins But what are they actually doing when they login? Logins don’t tell you about actions and value. Vanity metrics are bad!
- 31. Speculative, tries to find unexpected or interesting insights. Predictable, keeps you abreast of normal, day-to-day operations. The cool stuff. The necessary stuff. Exploratory Reporting vs.
- 32. Pivoting from friends to moms •Started as Circle of Friends •Leveraged Facebook early •Grew to 10M users But engagement sucked!
- 33. Moms are crazy! (in a good way) Engagement solved! • Messages to one another were on average 50% longer. • 115% more likely to attach a picture to a post they wrote. • 110% more likely to engage in a threaded (i.e. deep) conversation. • Friends, once invited, were 50% more likely to become engaged users. • 180% more likely to click on Facebook news feed items. • 60% more likely to accept invitations to the app.
- 34. Historical metric that shows you how you’re doing: reports the news. Number today that shows a metric tomorrow: makes the news. Try and get here.Start here. Lagging Leading vs.
- 35. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/ Analytical Superpowers (or what the heck is growth hacking?)
- 36. 1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
- 37. Summer Ice cream consumption Drowning Correlated Causal Causal Two variables that change in similar ways, perhaps because they’re linked to something else. Correlated An independent factor that directly impacts a dependent one. Causal vs.
- 38. Correlation lets you predict the future Causality lets you change the future “I will have 420 engaged users and 75 paying customers next month.” “If I can make more first-time visitors stay on for 17 minutes I will increase sales in 90 days.” Find correlation Test causality Optimize the causal factor Causality is a superpower, because it lets you change the future.
- 40. Your Business + Stage What business are you in? What stage are you at? •E-Commerce •SaaS •Free Mobile App •2-Sided Marketplace •Media •User-Generated Content •Empathy •Stickiness •Virality •Revenue •Scale
- 41. business models
- 42. The SaaS Customer Lifecycle Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
- 43. •Stage: Revenue / Scale •Model: SaaS (Paid) •Recruitment marketing and assessment software •Switched business models from monthly subscription to pay per job posting Does recurring revenue work for everyone?
- 44. 10x revenue increase off of 3x in sales volume “People don’t do subscriptions for haircuts, hamburgers, and hiring. You have to understand your customer, who they are, how and why they buy, and how they value your product or service.” - Ben Baldwin, co-founder Lots of money!
- 47. EMPATHY STICKINESS GROWTHRATE VIRALITY REVENUE SCALE Lean Analytics Stages I’ve found a real, poorly-met need that a reachable market faces. I’ve figured out how to solve the problem in a way they will adopt and pay for. I’ve built the right product/features/ functionality that keeps users around. The users and features fuel growth organically and artificially. I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. “Gates” needed to move forward
- 48. •Stage: Scale •Model: SaaS •Popular social sharing application •Focused primarily on customer acquisition •Charged from day one From Stickiness to Scale (through Revenue)
- 49. 20% 60% 20% 2% of visitors created an account (acquisition / Empathy) of sign-ups returned in the 1st month (engagement / Stickiness) of sign-ups were active after 6 months (engagement / Stickiness) convert from free to paid (Virality & Revenue) Buffer charges early to prove people want the problem solved
- 50. skip steps at your own risk
- 51. One Metric That Matters. How It All Comes Together The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
- 52. Choose only one metric and draw a line in the sand. Putting the pieces together...
- 53. •Stage: Revenue •Model: SaaS (Freemium) •Paper and digital collaboration •180,000 users •Paid churn is their One Metric That Matters (OMTM) Building a revenue engine
- 54. • Target < 4% paid churn (hitting 2% lately on a monthly basis) •Anything over 5% means they don’t have a business that will generate positive margin returns: the bucket is too leaky The OMTM: Paid Churn
- 55. • Can we acquire more valuable customers? •What product features can increase engagement? • Can we improve customer support? •Was a marketing campaign successful? •Were customer complaints lowered? •Was a product upgrade valuable? If Paid Churn: Why & Next Steps: Paid Churn = “business health” indicator •Are the new customers not the right segment? • Did a marketing campaign fail? • Did a product upgrade fail somehow? • Is customer support falling apart?
- 56. Some interesting benchmarks Growth 5% / week (revenue or active users) Churn 2% / month Engaged visitors 30% monthly users 10% daily users Time on site 17 minutes Page load time < 5 seconds CLV:CAC 3:1 Mobile file size < 50MB Free to paid 2% of free users
- 58. Identify a key business problem, pick the OMTM, draw a line in the sand, and get started. Time to experiment
- 59. Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI The Lean Analytics Cycle