SlideShare a Scribd company logo
1 of 59
Download to read offline
Metrics-Driven Design
                            by Joshua Porter

                            Twitter: bokardo


                                    bokardo
                                  52weeksofUX
                                  performable
                                   microcopy
                           designing for the social web
                                make them care!
                                                          by Joshua Porter

Saturday, March 12, 2011
Saturday, March 12, 2011
ā€œUnfortunately for me, there was one
                           small problem I didnā€™t see back then.ā€



Saturday, March 12, 2011
Doug Bowman on Design at Google




      ā€œ                    Yes, itā€™s true that a team at Google couldnā€™t decide between two
                           blues, so theyā€™re testing 41 shades between each blue to see
                           which one performs better. I had a recent debate over whether a
                           border should be 3, 4 or 5 pixels wide, and was asked to prove my
                           case. I canā€™t operate in an environment like that.




      Metrics-Driven Design                                                     Daring. Decisive. Conviction.   SXSW
Saturday, March 12, 2011
41 Shades of Blue Test




                                                        link color
        Gmail                                                                            Google Search




                           41 Bucket Split Test: ~2.5% of users each got a shade for 2 weeks.




Saturday, March 12, 2011
Saturday, March 12, 2011
#2200C1          #0044CC



                           Google             Bing



Saturday, March 12, 2011
ā€œ                    That blue was worth at
                           least $80 million.
                           Paul Ray, UX Manager, Bing




      Metrics-Driven Design                             Microcopy   SXSW
Saturday, March 12, 2011
Doug Bowman on Design at Google




      ā€œ                    When a company is ļ¬lled with engineers, it turns to engineering to
                           solve problems. Reduce each decision to a simple logic problem.
                           Remove all subjectivity and just look at the data. Data in your favor?
                           Ok, launch it. Data shows negative eļ¬€ects? Back to the drawing board.
                           And that data eventually becomes a crutch for every decision,
                           paralyzing the company and preventing it from making any daring
                           design decisions.




      Metrics-Driven Design                                                      Daring. Decisive. Conviction.   SXSW
Saturday, March 12, 2011
Spectrum of Design


   Intuition-Driven                                                                               Data-Driven



           Make best-guesses                                                Every design choice is tested
           Rely on previous experience                        Takes others experience with a grain of salt
           Study what others are doing                                         Design is a logic problem
           Use best practices, principles & patterns                   Rely on data for decision-making
           Aesthetics are integral                                             Aesthetics are secondary
           Rely on our gut                                                          Never trust your gut
           Creative, visionary                                                          Cold, calculating
           Inherently risky                                                                   Risk-averse




           Dougā€™s words:                                                                       Assumed:

           instinctive, subjective, daring                          deliberate, objective, safe



Saturday, March 12, 2011
Imagine that your design is a mountain...




      Metrics-Driven Design               The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...




                    EXISTING DESIGN




      Metrics-Driven Design               The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...




                               Now



                    EXISTING DESIGN




      Metrics-Driven Design               The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...




                 Optimized (local maxima)

                               Now



                    EXISTING DESIGN




      Metrics-Driven Design                 The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...



           Potential
           Improvement



                 Optimized (local maxima)

                               Now




      Metrics-Driven Design                 The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...


                                        Goal




                                                                    BETTER DESIGN



                       Current Design



                    EXISTING DESIGN




      Metrics-Driven Design                    The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Imagine that your design is a mountain...


                                      Goal

           Potential
           Improvement

                                                                  BETTER DESIGN



                              Now



                    EXISTING DESIGN




      Metrics-Driven Design                  The grass is always greener on other mountainsides.   SXSW
Saturday, March 12, 2011
Optimization asks:
                           What works best in the current model?



                                    Design innovation asks:
                             What is the best possible model?




Saturday, March 12, 2011
What are Metrics?




Saturday, March 12, 2011
DEFINITION


                  Metrics are simply numbers
                that measure the eļ¬€ectiveness
                       of your business.



Saturday, March 12, 2011
5 Reasons why Metrics are a
                          Designerā€™s Best Friend




Saturday, March 12, 2011
1. Metrics reduce arguments
                          based on opinion.




Saturday, March 12, 2011
2. Metrics give you answers
                            about what really works.




Saturday, March 12, 2011
3. Metrics show you where
                           youā€™re strong as a designer.




Saturday, March 12, 2011
4. Metrics allow you to test
                               anything you want.




Saturday, March 12, 2011
5. Clients love metrics.




Saturday, March 12, 2011
PRINCIPLE

                  Your metrics will be as unique
                        as your business.




Saturday, March 12, 2011
Saturday, March 12, 2011
Vanity Metrics




      Metrics-Driven Design   Ego   SXSW
Saturday, March 12, 2011
The Usage Lifecycle




Saturday, March 12, 2011
Interested   Trial/beta User   Customer   Passionate
                                                             Customer




Saturday, March 12, 2011
Acquisition          Conversion      Engagement        Satisfaction




           Interested               Trial/beta User      Customer        Passionate
                                                                         Customer




Saturday, March 12, 2011
With metrics you are essentially just measuring
                how well you move people along the lifecycle!




Saturday, March 12, 2011
Acquisition Metrics




Saturday, March 12, 2011
CPA - Cost per Acquisition

                           If your CPA is higher than your LTV
                                  then youā€™re in trouble.




Saturday, March 12, 2011
Acquisition vs. Referral: Dropbox


                                            Ran Google Adwords campaigns
                                            to drive traļ¬ƒc to dropbox.com

                                            The traļ¬ƒc that completed the lifecycle:

                                            1) Searched on a keyword
                                            2) Visited their site
                                            3) Signed-up for service
                                            4) Became a customer

                                            cost them $233-$388 per person!
                                            (for a $99 product)




      Metrics-Driven Design                                  LTV = Lifetime value   SXSW
Saturday, March 12, 2011
Dropbox Lifecycle




                              30 days prior to April 2010, Dropbox users
                              sent 2.8 million direct referral invites.


                              Dropboxā€™s referral program with 2-sided
                              incentive increased sign-ups by 60%
                              permanently.




      Metrics-Driven Design                         Referral = Word of Mouth   SXSW
Saturday, March 12, 2011
Performable Acquisition Metrics



                      Comparative Metrics

                           Revenue by Channel

                       Revenue by Keyword




      Metrics-Driven Design                     Microcopy   SXSW
Saturday, March 12, 2011
Conversion Metrics




Saturday, March 12, 2011
Trials / Conversions = Conversion Rate




Saturday, March 12, 2011
Conversion Funnel Analysis



                    A                A         100%




                     B               B     60%




                     C               C   20%


      Metrics-Driven Design                           SXSW
Saturday, March 12, 2011
Sign-up Conversion Funnel



       Original Flow                     1                    2                    3                      4
                                  conļ¬rm personal info   add your friends       invite others         getting started




        Original Conversion
        Funnel
                                                                            1                            100%

         of the 100% of people who
         started the sign-up process, only                                  2                         63%
         14% made it to the getting
         started screen.
                                                                            3                   26%


                                                                            4          14%


      Metrics-Driven Design                                                                                             SXSW
Saturday, March 12, 2011
Sign-up Conversion Funnel




          New Flow
                                                  2                          4
                                             add your friends           getting started




          New Conversion Funnel                                  2                                     100%
          of the 100% of people who
          started the sign-up process, 86%
          made it to the getting started
          screen.
                                                                 4                                86%




      Metrics-Driven Design                                Removing stuļ¬€ is a quick way to improve your funnel   SXSW
Saturday, March 12, 2011
Engagement Matters: Twitter




                                                     http://www.lukew.com/ff/entry.asp?1039



      Metrics-Driven Design           Speed without context is wasted.           SXSW
Saturday, March 12, 2011
Engagement Metrics




Saturday, March 12, 2011
ā€¢   Hits
                           ā€¢   Page views
                           ā€¢   Visits
                           ā€¢   Unique Visitors
                           ā€¢   Returning Visitors
                           ā€¢   Registered Users
                           ā€¢   Customers
                           ā€¢   Frequency
                           ā€¢   Time on Site
                           ā€¢   Daily Active Users




Saturday, March 12, 2011
Cohort Analysis

                                                                Engagement over time

                      Month 1     Month 2   Month 3   Month 4   Month 5   Month 6    Month 7    Month 8     Month 9      Month 10 Month 11 Month 12

      (Joined in)
                           100%      20%       19%       13%       13%       10%        12%        11%              7%           7%            7%
                                                                                                                                                         ?
      January


                           100%      21%       16%       13%       11%          9%         9%         7%            7%           7%
                                                                                                                                           ?
      February


                           100%      24%       20%       17%       15%       13%        11%        10%           10%
                                                                                                                             ?
      March


                           100%      31%       27%       24%       19%       15%        12%        12%
                                                                                                                ?
      April


                           100%      31%       27%       25%       21%       18%        16%
                                                                                                  ?
      May


                           100%      39%       28%       24%       20%       19%
                                                                                       ?
      June


                           100%      40%       33%       27%       23%
                                                                            ?
      July


                           100%      47%       41%       32%
                                                                  ?
      August


                           100%      52%       43%
                                                        ?
      September
                                                                          In this case engagement is improving nicely.
                           100%      53%
                                              ?                           Of the January cohort, only 20% were engaged in month 2.
      October                                                             Of the October cohort, 53% were engaged in month 2.

                           100%
                                    ?
      November

                           ?
      December                                                                                        http://redeye.ļ¬rstround.com/2008/01/after-the-techc.html



Saturday, March 12, 2011
Prevention: Facebook Deactivation

                                                      Design changes to
                                                    the deactivation page
                                                   accounted for 1 million
                                                   members not leaving the
                                                           service.
                                                                              Julie Zhou, Facebook




                                    http://www.readwriteweb.com/archives/what_happens_when_you_deactivate_your_facebook_acc.php


      Metrics-Driven Design                                          I will miss you after my talk today.             SXSW
Saturday, March 12, 2011
Satisfaction Metrics




Saturday, March 12, 2011
Referral: Net Promoter Score




            How likely is it that you would recommend
              our company to a friend or colleague?

                0             1   2   3   4   5   6            7            8           9            10




      Metrics-Driven Design                           Do you know what your net promoter score is?        SXSW
Saturday, March 12, 2011
Net Promoter Score




            How likely is it that you would recommend
              our company to a friend or colleague?

                0             1      2       3        4   5   6            7            8           9            10

                                         Detractors                          Passives               Promoters




                                  Score = % Promoters - % Detractors


      Metrics-Driven Design                                       Do you know what your net promoter score is?        SXSW
Saturday, March 12, 2011
Mint.com & Net Promoter Score




      ā€œ                    Maybe we didnā€™t have a high viral coeļ¬ƒcient
                           but we had a great net promoter score.
                           Jason Putorti, Lead Designer, Mint.com




      Metrics-Driven Design                                              SXSW
Saturday, March 12, 2011
Emergent Metrics




Saturday, March 12, 2011
Emergent Metrics: 5 Friends

                                      The magic number is 5.
                                      Once a FriendFeed user
                                      found ļ¬ve friends, they
                                       became active users.
                                               Bret Taylor, Friendfeed




      Metrics-Driven Design                                        SXSW
Saturday, March 12, 2011
Is there one metric that drives others?




      ā€œ                    At Blogger, we determined that our most critical
                           metric was number of posts. An increase in posts
                           meant that people were not just creating blogs, but
                           updating them, and more posts would drive more
                           readership, which would drive more users, which
                           would drive more posts.

                                                           Ev Williams
                                                           founder of Blogger & Twitter




      Metrics-Driven Design                                    Once youā€™re lucky, twice youā€™re good.   SXSW
Saturday, March 12, 2011
Principles of Metrics-Driven Design


               1. Optimize in small steps; innovate with daring leaps.
               2. No design survives contact with the user.
               3. Small improvements, taken together, yield amazing results.Ā 
               4. Testing is empowering, reversion is cleansing.
               5. Metrics are not creative: human beings are.
               6. All team members are responsible for the user experience.
               7. If metrics arenā€™t actionable, they arenā€™t useful.
               8. Design is never done.




      Metrics-Driven Design                           Reminder: Principal = Person & Principle = Thing   SXSW
Saturday, March 12, 2011
Saturday, March 12, 2011
This Talk.



       http://bokardo.com/talks/metrics-driven-design/




Saturday, March 12, 2011
My Life.




                    http://performable.com              http://oneļ¬‚ightbooks.com




Saturday, March 12, 2011

More Related Content

What's hot

UX Design + UI Design: Injecting a brand persona!
UX Design + UI Design: Injecting a brand persona!UX Design + UI Design: Injecting a brand persona!
UX Design + UI Design: Injecting a brand persona!Jayan Narayanan
Ā 
Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2...
Finding the Narrative in Numbers: Making the Most of Metrics  [UX Immersion 2...Finding the Narrative in Numbers: Making the Most of Metrics  [UX Immersion 2...
Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2...Kate Rutter
Ā 
Dropbox startup lessons learned 2011
Dropbox   startup lessons learned 2011Dropbox   startup lessons learned 2011
Dropbox startup lessons learned 2011Eric Ries
Ā 
The Other C Word: What makes great content marketing great
The Other C Word: What makes great content marketing greatThe Other C Word: What makes great content marketing great
The Other C Word: What makes great content marketing greatVelocity Partners
Ā 
Finding the one growth metric that matters
Finding the one growth metric that mattersFinding the one growth metric that matters
Finding the one growth metric that mattersSean Ellis
Ā 
A/B testing at Spotify
A/B testing at SpotifyA/B testing at Spotify
A/B testing at SpotifyAli Sarrafi
Ā 
Dropbox Startup Lessons Learned
Dropbox Startup Lessons LearnedDropbox Startup Lessons Learned
Dropbox Startup Lessons Learnedgueste94e4c
Ā 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS MetricsChristoph Janz
Ā 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Andreas Klinger
Ā 
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)Dave McClure
Ā 
Startup Metrics for Pirates (SeedCamp 2008)
Startup Metrics for Pirates (SeedCamp 2008)Startup Metrics for Pirates (SeedCamp 2008)
Startup Metrics for Pirates (SeedCamp 2008)Dave McClure
Ā 
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)Dave McClure
Ā 
30-Day Facebook PM Interview Study Guide
30-Day Facebook PM Interview Study Guide30-Day Facebook PM Interview Study Guide
30-Day Facebook PM Interview Study GuideLewis Lin šŸ¦Š
Ā 
Design System & Atomic Design
Design System & Atomic DesignDesign System & Atomic Design
Design System & Atomic DesignVani Jain
Ā 
Data-Driven @ Netflix
Data-Driven @ NetflixData-Driven @ Netflix
Data-Driven @ NetflixMichelle Ufford
Ā 
UX Lesson 1: User Centered Design
UX Lesson 1: User Centered DesignUX Lesson 1: User Centered Design
UX Lesson 1: User Centered DesignJoan Lumanauw
Ā 
Report on Beautiful Evidence by E. Tufte
Report on Beautiful Evidence by E. TufteReport on Beautiful Evidence by E. Tufte
Report on Beautiful Evidence by E. Tuftehckingmtuedu
Ā 
A Content Strategy Roadmap
A Content Strategy RoadmapA Content Strategy Roadmap
A Content Strategy RoadmapKristina Halvorson
Ā 

What's hot (20)

Good UX Bad UX
Good UX Bad UXGood UX Bad UX
Good UX Bad UX
Ā 
UX Design + UI Design: Injecting a brand persona!
UX Design + UI Design: Injecting a brand persona!UX Design + UI Design: Injecting a brand persona!
UX Design + UI Design: Injecting a brand persona!
Ā 
Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2...
Finding the Narrative in Numbers: Making the Most of Metrics  [UX Immersion 2...Finding the Narrative in Numbers: Making the Most of Metrics  [UX Immersion 2...
Finding the Narrative in Numbers: Making the Most of Metrics [UX Immersion 2...
Ā 
Dropbox startup lessons learned 2011
Dropbox   startup lessons learned 2011Dropbox   startup lessons learned 2011
Dropbox startup lessons learned 2011
Ā 
The Other C Word: What makes great content marketing great
The Other C Word: What makes great content marketing greatThe Other C Word: What makes great content marketing great
The Other C Word: What makes great content marketing great
Ā 
Finding the one growth metric that matters
Finding the one growth metric that mattersFinding the one growth metric that matters
Finding the one growth metric that matters
Ā 
A/B testing at Spotify
A/B testing at SpotifyA/B testing at Spotify
A/B testing at Spotify
Ā 
Dropbox Startup Lessons Learned
Dropbox Startup Lessons LearnedDropbox Startup Lessons Learned
Dropbox Startup Lessons Learned
Ā 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics
Ā 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Ā 
UI-UX Services | Web Designing Services
UI-UX Services | Web Designing ServicesUI-UX Services | Web Designing Services
UI-UX Services | Web Designing Services
Ā 
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)
Startup Metrics 4 Pirates (DogPatch Labs, Boston, March 2010)
Ā 
Startup Metrics for Pirates (SeedCamp 2008)
Startup Metrics for Pirates (SeedCamp 2008)Startup Metrics for Pirates (SeedCamp 2008)
Startup Metrics for Pirates (SeedCamp 2008)
Ā 
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)
Zapmeals: Sample Startup Pitch Deck (from SuperNova 2007)
Ā 
30-Day Facebook PM Interview Study Guide
30-Day Facebook PM Interview Study Guide30-Day Facebook PM Interview Study Guide
30-Day Facebook PM Interview Study Guide
Ā 
Design System & Atomic Design
Design System & Atomic DesignDesign System & Atomic Design
Design System & Atomic Design
Ā 
Data-Driven @ Netflix
Data-Driven @ NetflixData-Driven @ Netflix
Data-Driven @ Netflix
Ā 
UX Lesson 1: User Centered Design
UX Lesson 1: User Centered DesignUX Lesson 1: User Centered Design
UX Lesson 1: User Centered Design
Ā 
Report on Beautiful Evidence by E. Tufte
Report on Beautiful Evidence by E. TufteReport on Beautiful Evidence by E. Tufte
Report on Beautiful Evidence by E. Tufte
Ā 
A Content Strategy Roadmap
A Content Strategy RoadmapA Content Strategy Roadmap
A Content Strategy Roadmap
Ā 

Similar to Metrics Driven Design by Joshua Porter

How small changes big
How small changes bigHow small changes big
How small changes bigAdam Hassan
Ā 
Metrics drivendesign
Metrics drivendesignMetrics drivendesign
Metrics drivendesignRoman Zykov
Ā 
Scrum, it's all about the people
Scrum, it's all about the peopleScrum, it's all about the people
Scrum, it's all about the peopleDennis Vroegop
Ā 
Adaptive Experiences
Adaptive ExperiencesAdaptive Experiences
Adaptive ExperiencesWINTR
Ā 
Promise notes
Promise notesPromise notes
Promise notesCS, NcState
Ā 
SEF for students 2012. Product Management vs. Project Management
SEF for students 2012. Product Management vs. Project ManagementSEF for students 2012. Product Management vs. Project Management
SEF for students 2012. Product Management vs. Project ManagementAnthony Marchenko
Ā 
What User-Centered Design is Good For
What User-Centered Design is Good ForWhat User-Centered Design is Good For
What User-Centered Design is Good ForDan Saffer
Ā 
Midwest UX Mobile Workshop 2012
Midwest UX Mobile Workshop 2012Midwest UX Mobile Workshop 2012
Midwest UX Mobile Workshop 2012Haig Armen
Ā 
LISA 2011 Keynote: The DevOps Transformation
LISA 2011 Keynote: The DevOps TransformationLISA 2011 Keynote: The DevOps Transformation
LISA 2011 Keynote: The DevOps Transformationbenrockwood
Ā 
Critical Thinking forUX Designers (Workshop)
Critical Thinking forUX Designers (Workshop)Critical Thinking forUX Designers (Workshop)
Critical Thinking forUX Designers (Workshop)Stephen Anderson
Ā 
Ux paper prototyping
Ux paper prototypingUx paper prototyping
Ux paper prototypingGrace Ng
Ā 
Sustainable Theming with Fusion - DCCO 2011
Sustainable Theming with Fusion - DCCO 2011Sustainable Theming with Fusion - DCCO 2011
Sustainable Theming with Fusion - DCCO 2011sheenadonnelly
Ā 
Designing for Smart Spaces and Objects
Designing for Smart Spaces and ObjectsDesigning for Smart Spaces and Objects
Designing for Smart Spaces and Objectsfrog
Ā 
Insider's Technology Guide to Measuring and Improving Your ROI
Insider's Technology Guide to Measuring and Improving Your ROIInsider's Technology Guide to Measuring and Improving Your ROI
Insider's Technology Guide to Measuring and Improving Your ROIDan Leatherman
Ā 
Design for Marketing Technical Services
Design for Marketing Technical ServicesDesign for Marketing Technical Services
Design for Marketing Technical ServicesJon Sandruck
Ā 
Design thinking notes with explanation i
Design thinking notes with explanation iDesign thinking notes with explanation i
Design thinking notes with explanation iSRINURAJU3
Ā 
Design Thinking is Killing Creativity
Design Thinking is Killing CreativityDesign Thinking is Killing Creativity
Design Thinking is Killing Creativitydesignsojourn
Ā 
S.P.A.C.E. Exploration for Software Engineering
 S.P.A.C.E. Exploration for Software Engineering S.P.A.C.E. Exploration for Software Engineering
S.P.A.C.E. Exploration for Software EngineeringCS, NcState
Ā 
Class 2: Setting the foundation for a successful website design
Class 2: Setting the foundation for a successful website designClass 2: Setting the foundation for a successful website design
Class 2: Setting the foundation for a successful website designDon Stanley
Ā 

Similar to Metrics Driven Design by Joshua Porter (20)

How small changes big
How small changes bigHow small changes big
How small changes big
Ā 
Metrics drivendesign
Metrics drivendesignMetrics drivendesign
Metrics drivendesign
Ā 
Scrum, it's all about the people
Scrum, it's all about the peopleScrum, it's all about the people
Scrum, it's all about the people
Ā 
Adaptive Experiences
Adaptive ExperiencesAdaptive Experiences
Adaptive Experiences
Ā 
Promise notes
Promise notesPromise notes
Promise notes
Ā 
SEF for students 2012. Product Management vs. Project Management
SEF for students 2012. Product Management vs. Project ManagementSEF for students 2012. Product Management vs. Project Management
SEF for students 2012. Product Management vs. Project Management
Ā 
What User-Centered Design is Good For
What User-Centered Design is Good ForWhat User-Centered Design is Good For
What User-Centered Design is Good For
Ā 
Midwest UX Mobile Workshop 2012
Midwest UX Mobile Workshop 2012Midwest UX Mobile Workshop 2012
Midwest UX Mobile Workshop 2012
Ā 
LISA 2011 Keynote: The DevOps Transformation
LISA 2011 Keynote: The DevOps TransformationLISA 2011 Keynote: The DevOps Transformation
LISA 2011 Keynote: The DevOps Transformation
Ā 
Critical Thinking forUX Designers (Workshop)
Critical Thinking forUX Designers (Workshop)Critical Thinking forUX Designers (Workshop)
Critical Thinking forUX Designers (Workshop)
Ā 
Ux paper prototyping
Ux paper prototypingUx paper prototyping
Ux paper prototyping
Ā 
Sustainable Theming with Fusion - DCCO 2011
Sustainable Theming with Fusion - DCCO 2011Sustainable Theming with Fusion - DCCO 2011
Sustainable Theming with Fusion - DCCO 2011
Ā 
Designing for Smart Spaces and Objects
Designing for Smart Spaces and ObjectsDesigning for Smart Spaces and Objects
Designing for Smart Spaces and Objects
Ā 
Insider's Technology Guide to Measuring and Improving Your ROI
Insider's Technology Guide to Measuring and Improving Your ROIInsider's Technology Guide to Measuring and Improving Your ROI
Insider's Technology Guide to Measuring and Improving Your ROI
Ā 
Design for Marketing Technical Services
Design for Marketing Technical ServicesDesign for Marketing Technical Services
Design for Marketing Technical Services
Ā 
13 0212 toccon - carpenter altmetrics 2
13 0212 toccon - carpenter altmetrics 213 0212 toccon - carpenter altmetrics 2
13 0212 toccon - carpenter altmetrics 2
Ā 
Design thinking notes with explanation i
Design thinking notes with explanation iDesign thinking notes with explanation i
Design thinking notes with explanation i
Ā 
Design Thinking is Killing Creativity
Design Thinking is Killing CreativityDesign Thinking is Killing Creativity
Design Thinking is Killing Creativity
Ā 
S.P.A.C.E. Exploration for Software Engineering
 S.P.A.C.E. Exploration for Software Engineering S.P.A.C.E. Exploration for Software Engineering
S.P.A.C.E. Exploration for Software Engineering
Ā 
Class 2: Setting the foundation for a successful website design
Class 2: Setting the foundation for a successful website designClass 2: Setting the foundation for a successful website design
Class 2: Setting the foundation for a successful website design
Ā 

More from Andrew Chen

What's Next in Growth? 2016
What's Next in Growth? 2016What's Next in Growth? 2016
What's Next in Growth? 2016Andrew Chen
Ā 
Zero to traction
Zero to tractionZero to traction
Zero to tractionAndrew Chen
Ā 
SaaS products aren't viral
SaaS products aren't viralSaaS products aren't viral
SaaS products aren't viralAndrew Chen
Ā 
Minimum Desirable Product
Minimum Desirable ProductMinimum Desirable Product
Minimum Desirable ProductAndrew Chen
Ā 
Growing Renewable Audiences (presented at OATV)
Growing Renewable Audiences (presented at OATV)Growing Renewable Audiences (presented at OATV)
Growing Renewable Audiences (presented at OATV)Andrew Chen
Ā 
CommunityNext: People not Pages
CommunityNext: People not PagesCommunityNext: People not Pages
CommunityNext: People not PagesAndrew Chen
Ā 
Jeremy Liew - Stanford Facebook Class Presentation
Jeremy Liew - Stanford Facebook Class PresentationJeremy Liew - Stanford Facebook Class Presentation
Jeremy Liew - Stanford Facebook Class PresentationAndrew Chen
Ā 

More from Andrew Chen (7)

What's Next in Growth? 2016
What's Next in Growth? 2016What's Next in Growth? 2016
What's Next in Growth? 2016
Ā 
Zero to traction
Zero to tractionZero to traction
Zero to traction
Ā 
SaaS products aren't viral
SaaS products aren't viralSaaS products aren't viral
SaaS products aren't viral
Ā 
Minimum Desirable Product
Minimum Desirable ProductMinimum Desirable Product
Minimum Desirable Product
Ā 
Growing Renewable Audiences (presented at OATV)
Growing Renewable Audiences (presented at OATV)Growing Renewable Audiences (presented at OATV)
Growing Renewable Audiences (presented at OATV)
Ā 
CommunityNext: People not Pages
CommunityNext: People not PagesCommunityNext: People not Pages
CommunityNext: People not Pages
Ā 
Jeremy Liew - Stanford Facebook Class Presentation
Jeremy Liew - Stanford Facebook Class PresentationJeremy Liew - Stanford Facebook Class Presentation
Jeremy Liew - Stanford Facebook Class Presentation
Ā 

Recently uploaded

Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
Ā 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
Ā 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
Ā 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
Ā 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
Ā 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
Ā 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
Ā 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
Ā 
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better StrongerModern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Strongerpanagenda
Ā 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
Ā 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
Ā 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
Ā 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
Ā 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
Ā 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
Ā 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
Ā 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
Ā 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
Ā 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
Ā 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
Ā 

Recently uploaded (20)

Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
Ā 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Ā 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
Ā 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
Ā 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
Ā 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
Ā 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Ā 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
Ā 
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better StrongerModern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad ā€“ Cheaper Faster Better Stronger
Ā 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Ā 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
Ā 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Ā 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
Ā 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
Ā 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Ā 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
Ā 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Ā 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Ā 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
Ā 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Ā 

Metrics Driven Design by Joshua Porter

  • 1. Metrics-Driven Design by Joshua Porter Twitter: bokardo bokardo 52weeksofUX performable microcopy designing for the social web make them care! by Joshua Porter Saturday, March 12, 2011
  • 3. ā€œUnfortunately for me, there was one small problem I didnā€™t see back then.ā€ Saturday, March 12, 2011
  • 4. Doug Bowman on Design at Google ā€œ Yes, itā€™s true that a team at Google couldnā€™t decide between two blues, so theyā€™re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I canā€™t operate in an environment like that. Metrics-Driven Design Daring. Decisive. Conviction. SXSW Saturday, March 12, 2011
  • 5. 41 Shades of Blue Test link color Gmail Google Search 41 Bucket Split Test: ~2.5% of users each got a shade for 2 weeks. Saturday, March 12, 2011
  • 7. #2200C1 #0044CC Google Bing Saturday, March 12, 2011
  • 8. ā€œ That blue was worth at least $80 million. Paul Ray, UX Manager, Bing Metrics-Driven Design Microcopy SXSW Saturday, March 12, 2011
  • 9. Doug Bowman on Design at Google ā€œ When a company is ļ¬lled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative eļ¬€ects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions. Metrics-Driven Design Daring. Decisive. Conviction. SXSW Saturday, March 12, 2011
  • 10. Spectrum of Design Intuition-Driven Data-Driven Make best-guesses Every design choice is tested Rely on previous experience Takes others experience with a grain of salt Study what others are doing Design is a logic problem Use best practices, principles & patterns Rely on data for decision-making Aesthetics are integral Aesthetics are secondary Rely on our gut Never trust your gut Creative, visionary Cold, calculating Inherently risky Risk-averse Dougā€™s words: Assumed: instinctive, subjective, daring deliberate, objective, safe Saturday, March 12, 2011
  • 11. Imagine that your design is a mountain... Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 12. Imagine that your design is a mountain... EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 13. Imagine that your design is a mountain... Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 14. Imagine that your design is a mountain... Optimized (local maxima) Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 15. Imagine that your design is a mountain... Potential Improvement Optimized (local maxima) Now Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 16. Imagine that your design is a mountain... Goal BETTER DESIGN Current Design EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 17. Imagine that your design is a mountain... Goal Potential Improvement BETTER DESIGN Now EXISTING DESIGN Metrics-Driven Design The grass is always greener on other mountainsides. SXSW Saturday, March 12, 2011
  • 18. Optimization asks: What works best in the current model? Design innovation asks: What is the best possible model? Saturday, March 12, 2011
  • 19. What are Metrics? Saturday, March 12, 2011
  • 20. DEFINITION Metrics are simply numbers that measure the eļ¬€ectiveness of your business. Saturday, March 12, 2011
  • 21. 5 Reasons why Metrics are a Designerā€™s Best Friend Saturday, March 12, 2011
  • 22. 1. Metrics reduce arguments based on opinion. Saturday, March 12, 2011
  • 23. 2. Metrics give you answers about what really works. Saturday, March 12, 2011
  • 24. 3. Metrics show you where youā€™re strong as a designer. Saturday, March 12, 2011
  • 25. 4. Metrics allow you to test anything you want. Saturday, March 12, 2011
  • 26. 5. Clients love metrics. Saturday, March 12, 2011
  • 27. PRINCIPLE Your metrics will be as unique as your business. Saturday, March 12, 2011
  • 29. Vanity Metrics Metrics-Driven Design Ego SXSW Saturday, March 12, 2011
  • 31. Interested Trial/beta User Customer Passionate Customer Saturday, March 12, 2011
  • 32. Acquisition Conversion Engagement Satisfaction Interested Trial/beta User Customer Passionate Customer Saturday, March 12, 2011
  • 33. With metrics you are essentially just measuring how well you move people along the lifecycle! Saturday, March 12, 2011
  • 35. CPA - Cost per Acquisition If your CPA is higher than your LTV then youā€™re in trouble. Saturday, March 12, 2011
  • 36. Acquisition vs. Referral: Dropbox Ran Google Adwords campaigns to drive traļ¬ƒc to dropbox.com The traļ¬ƒc that completed the lifecycle: 1) Searched on a keyword 2) Visited their site 3) Signed-up for service 4) Became a customer cost them $233-$388 per person! (for a $99 product) Metrics-Driven Design LTV = Lifetime value SXSW Saturday, March 12, 2011
  • 37. Dropbox Lifecycle 30 days prior to April 2010, Dropbox users sent 2.8 million direct referral invites. Dropboxā€™s referral program with 2-sided incentive increased sign-ups by 60% permanently. Metrics-Driven Design Referral = Word of Mouth SXSW Saturday, March 12, 2011
  • 38. Performable Acquisition Metrics Comparative Metrics Revenue by Channel Revenue by Keyword Metrics-Driven Design Microcopy SXSW Saturday, March 12, 2011
  • 40. Trials / Conversions = Conversion Rate Saturday, March 12, 2011
  • 41. Conversion Funnel Analysis A A 100% B B 60% C C 20% Metrics-Driven Design SXSW Saturday, March 12, 2011
  • 42. Sign-up Conversion Funnel Original Flow 1 2 3 4 conļ¬rm personal info add your friends invite others getting started Original Conversion Funnel 1 100% of the 100% of people who started the sign-up process, only 2 63% 14% made it to the getting started screen. 3 26% 4 14% Metrics-Driven Design SXSW Saturday, March 12, 2011
  • 43. Sign-up Conversion Funnel New Flow 2 4 add your friends getting started New Conversion Funnel 2 100% of the 100% of people who started the sign-up process, 86% made it to the getting started screen. 4 86% Metrics-Driven Design Removing stuļ¬€ is a quick way to improve your funnel SXSW Saturday, March 12, 2011
  • 44. Engagement Matters: Twitter http://www.lukew.com/ff/entry.asp?1039 Metrics-Driven Design Speed without context is wasted. SXSW Saturday, March 12, 2011
  • 46. ā€¢ Hits ā€¢ Page views ā€¢ Visits ā€¢ Unique Visitors ā€¢ Returning Visitors ā€¢ Registered Users ā€¢ Customers ā€¢ Frequency ā€¢ Time on Site ā€¢ Daily Active Users Saturday, March 12, 2011
  • 47. Cohort Analysis Engagement over time Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 (Joined in) 100% 20% 19% 13% 13% 10% 12% 11% 7% 7% 7% ? January 100% 21% 16% 13% 11% 9% 9% 7% 7% 7% ? February 100% 24% 20% 17% 15% 13% 11% 10% 10% ? March 100% 31% 27% 24% 19% 15% 12% 12% ? April 100% 31% 27% 25% 21% 18% 16% ? May 100% 39% 28% 24% 20% 19% ? June 100% 40% 33% 27% 23% ? July 100% 47% 41% 32% ? August 100% 52% 43% ? September In this case engagement is improving nicely. 100% 53% ? Of the January cohort, only 20% were engaged in month 2. October Of the October cohort, 53% were engaged in month 2. 100% ? November ? December http://redeye.ļ¬rstround.com/2008/01/after-the-techc.html Saturday, March 12, 2011
  • 48. Prevention: Facebook Deactivation Design changes to the deactivation page accounted for 1 million members not leaving the service. Julie Zhou, Facebook http://www.readwriteweb.com/archives/what_happens_when_you_deactivate_your_facebook_acc.php Metrics-Driven Design I will miss you after my talk today. SXSW Saturday, March 12, 2011
  • 50. Referral: Net Promoter Score How likely is it that you would recommend our company to a friend or colleague? 0 1 2 3 4 5 6 7 8 9 10 Metrics-Driven Design Do you know what your net promoter score is? SXSW Saturday, March 12, 2011
  • 51. Net Promoter Score How likely is it that you would recommend our company to a friend or colleague? 0 1 2 3 4 5 6 7 8 9 10 Detractors Passives Promoters Score = % Promoters - % Detractors Metrics-Driven Design Do you know what your net promoter score is? SXSW Saturday, March 12, 2011
  • 52. Mint.com & Net Promoter Score ā€œ Maybe we didnā€™t have a high viral coeļ¬ƒcient but we had a great net promoter score. Jason Putorti, Lead Designer, Mint.com Metrics-Driven Design SXSW Saturday, March 12, 2011
  • 54. Emergent Metrics: 5 Friends The magic number is 5. Once a FriendFeed user found ļ¬ve friends, they became active users. Bret Taylor, Friendfeed Metrics-Driven Design SXSW Saturday, March 12, 2011
  • 55. Is there one metric that drives others? ā€œ At Blogger, we determined that our most critical metric was number of posts. An increase in posts meant that people were not just creating blogs, but updating them, and more posts would drive more readership, which would drive more users, which would drive more posts. Ev Williams founder of Blogger & Twitter Metrics-Driven Design Once youā€™re lucky, twice youā€™re good. SXSW Saturday, March 12, 2011
  • 56. Principles of Metrics-Driven Design 1. Optimize in small steps; innovate with daring leaps. 2. No design survives contact with the user. 3. Small improvements, taken together, yield amazing results.Ā  4. Testing is empowering, reversion is cleansing. 5. Metrics are not creative: human beings are. 6. All team members are responsible for the user experience. 7. If metrics arenā€™t actionable, they arenā€™t useful. 8. Design is never done. Metrics-Driven Design Reminder: Principal = Person & Principle = Thing SXSW Saturday, March 12, 2011
  • 58. This Talk. http://bokardo.com/talks/metrics-driven-design/ Saturday, March 12, 2011
  • 59. My Life. http://performable.com http://oneļ¬‚ightbooks.com Saturday, March 12, 2011