Successfully reported this slideshow.
Your SlideShare is downloading.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

15

Share

Measure Right The First Time

The prevailing culture of “Ship the MVP!” – where sites and apps are built in the cheapest, fastest way possible without regard for the future – needs to end. In this session, Vuurr will show you how to build web properties right the first time, specifically emphasising analytics and data collection. Learn how to properly configure many analytics tools, track offline interactions like phone calls and tangible marketing efforts, and structure onsite code and content. In addition, you will learn what to do after data has been collected and it’s time to iterate. We’ll show you how these strategies can lead to better decisions by you and your clients.

Measure Right The First Time

  1. 1. MEASURE RIGHTTHE FIRST TIME Scott Yacko & Jonathan Kressaty BOLO 2012 vuurr.com 1
  2. 2. WHAT WE’LL COVER_ What you need_ How to track_ How to design experiments_ What to experiment on_ Putting it all together 2
  3. 3. INTERNET MARKETING IS THE INTERSECTION OF COPYWRITING AND MATHEMATICS. 3
  4. 4. YOUR HIRING DECISIONSSHOULD REFLECT THIS. 4
  5. 5. TOOLS 5
  6. 6. TOOLS YOU MUST HAVE_ Google Analytics (or SiteCatalyst, or similar)_ Optimizely (or similar, but you need killer statistics)_ Call tracking => analytics platform of choice_ Link builder appropriate for your analytics platform of choice_ AdWords/AdCenter/Facebook properly configured (if advertising) 6
  7. 7. DETERMINE YOUR MODEL CPA ROI MARGIN Minimize the Maximize Minimize ad cost of each revenue for spend as a customer every dollar percent of acquisition spent revenue 7
  8. 8. DETERMINE YOUR MODEL CPA ROI MARGIN Ad Spend ($) Revenue - Spend Revenue - Spend Conversion (qty) Spend Revenueanswer is in dollars ($) per conversion all in dollars ($), answer is % all in dollars ($), answer is % 8
  9. 9. DETERMINE YOUR MODEL -FINDING ROI & MARGIN MARGIN ROI 9
  10. 10. DETERMINE YOUR MODEL - FINDING CPA (MANUALLY) CompletedAdvertising => AdWords => Campaigns => Site Usage Conversions Advertising => AdWords => Campaigns => Clicks Total Cost 10
  11. 11. DETERMINE YOUR MODEL -FINDING CPA (AUTOMATICALLY) _ Use the Google Analytics API to query for total number of conversion by source & medium, and then properly divide them into the total marketing expense for each source & medium. _ Use a third party tool 11
  12. 12. TRACKING 12
  13. 13. INSTALLING ANALYTICS_ Follow your vendor’s instructions_ Don’t just dump the tracking code in and “go”_ Setup proper funnels, events, goals, and filters_ Know the difference between accounts, properties, and profiles_ Attach GWT and AdWords (if applicable)_ Track ALL OF THE THINGS 13
  14. 14. GOALS & ECOMMERCE_ Assign relevant goal values - video view != purchase_ Track eCommerce as accurately as possible_ Explicit funnels are only useful if the supporting steps are relevant and provide information - use sparingly 14
  15. 15. TRACKING CAMPAIGNS _ Every click is valuable Name/ Content _ Every traffic source matters _ Proper campaign tracking is Campaign crucial to adequately gaugingSource Medium traffic value 15
  16. 16. PROPER URL TAGGING 16
  17. 17. TRACKING RICH MEDIA:VIDEOS_ Instantiate video with YouTube/Vendor JavaScript API_ Track state changes, namely Plays and Completes_ On these changes, push GA events 17
  18. 18. TRACKING RICH MEDIA:SOCIAL BUTTONS_gaq.push([_trackSocial, network, socialAction, opt_target,opt_pagePath]); Instantiate by subscribing to edge.create and message.send Google Analytics handles for you Instantiate by subscribing to the ‘tweet’ intent event 18
  19. 19. TRACKING RICH MEDIA:FORM ABANDONMENT _ Track full and empty exits as events _ Determine fields causing abandonment 19
  20. 20. TRACKING OFFLINE:PHONE CALLS _ Build GA gif request and HTTP GET on phone call _ Push calls as pageviews ( /call/+14805551234/abcdefg ) _ Use Twilio & Twimlbin or another 3rd party service _ Consider filtering out those pageviews from main profile _ Setup goals in Google Analytics to fire on calls _ Give each ad, landing page, social source, different number 20
  21. 21. TRACKING PAID CLICKS _ Properly configure AdWords to push to Analytics _ Verify all cost data is applied to Analytics account _ Put MSN AdCenter {AdId} & {Keyword} in destination URL, then use AdCenter API to pull the cost and associate with relevant clicks from Analytics API 21
  22. 22. EXPERIMENT DESIGN 22
  23. 23. INTERNET MARKETING ISABOUT EXPERIMENTATIONy = f( x1, x2, x3, . . . . xn )y= CTR x1 = Page or Ad Copy CR x2 = Page Layout Goals x3 = Button Color & Size xn = Other Factors... 23
  24. 24. DESIGNING EXPERIMENTS - ONE FACTOR AT A TIMELet’s run an experiment where we test two different button colors. Run Button Color ( x1 ) CTR ( y ) 1 Green 5.2% 2 Orange 7.7%Factors are tested in series, one at a time. This is extremely timeintensive, requiring many tests for very gradual improvement. 24
  25. 25. DESIGNING EXPERIMENTS - FULL FACTORIAL DESIGNLet’s run an experiment where we test two different button colors,button positions, and button labels simultaneously. 3 FACTORS WITH 2 VARIATIONS EACH Number of Unique Tests =23=8This tests multiple changes in parallel to find optimal combination offactors (x variables) to optimize results (y variable). 25
  26. 26. DESIGNING EXPERIMENTS -FULL FACTORIAL DESIGN Run Color (x1) Position (x2) Text (x3) CTR (y) 1 Green Left Try Now 5.2% 2 Orange Left Try Now 7.7% 3 Green Right Try Now 9.3% 4 Orange Right Try Now 12.7% 5 Green Left Buy Today 21.2% 6 Orange Left Buy Today 19.3% 7 Green Right Buy Today 14.9% 8 Orange Right Buy Today 17.7% 26
  27. 27. DESIGNING EXPERIMENTS -OFAT VS. FULL FACTORIALONE FACTOR AT A TIME FULL FACTORIAL Iterative More Complex to Setup Simple to Setup VS. Requires more visitor dataEasy to Measure Results to test all combinations SLOW & GRADUAL QUICK & DRASTIC IMPROVEMENT IMPROVEMENT 27
  28. 28. DESIGNING EXPERIMENTS - STATISTICAL SIGNIFICANCEPreliminary results - is orange better?Button Color ( x1 ) Visitors Clicks CTR ( y ) Green 38 2 5.2% Orange 39 3 7.7%Too small of a sample size leads to incorrect conclusions. 28
  29. 29. DESIGNING EXPERIMENTS - STATISTICAL SIGNIFICANCEIt turns out our original conclusion of CTR was incorrect.Button Color ( x1 ) Visitors Clicks CTR ( y ) Green 8,238 486 5.9% Orange 7,893 734 9.3% Too large of a sample size wastes time, effort, & money. 29
  30. 30. SO WHAT QUANTITY OF VISITORS IS ENOUGH? 30
  31. 31. DETERMINING MINIMUM SAMPLE SIZEx = Sample meanµ = Population mean E=x-µZ = Number of standarddeviations above meanα = Confidence distance σfrom 100%σ = Standard deviation E = Z α/2 √nn = Minimum sample size 31
  32. 32. DETERMINING MINIMUMSAMPLE SIZE - α/2 Z α/2 32
  33. 33. DETERMINING MINIMUMSAMPLE SIZE (CONTINUED) σ Standard E = Z α/2 √n Deviation Minimum Sample Size Z σ n= [ α/2 E ] 33
  34. 34. A/B TESTING SOFTWARE 34
  35. 35. TESTING 35
  36. 36. EVERY SINGLE CLICK HAS ACOST ASSOCIATED WITH IT 36
  37. 37. KNOW YOUR LEVERS _ Product Pricing _ Organic Rankings _ Ad bids, placement, content _ Email subject lines & content _ Social content _ Site Content 37
  38. 38. FUNNEL ANALYSIS &OPTIMIZATION IMPRESSIONS Traffic Sources drive impressions & clicks CTR _ SEM, SEO, Social, Email, & More Optimizing those sources get more clicks CLICKS _ Improve Quality Score (Site Visits) _ Split Test Ad Copy _ Adjust Keyword Bids _ Increase Organic Rankings _ Increase Social Reach CR Optimized landing pages & site content increase conversions _ Split Test Landing Page Copy _ Improve Call to Action _ Experiment with Deep Linking CONVERSIONS _ Look at Bounce Rates _ Look at Cart Abandonment (Sales/Leads) 38
  39. 39. FUNNEL ANALYSIS &OPTIMIZATION ORGANIC OFFLINE EMAIL SEARCH MARKETING AFFILIATES PPC SOCIAL PHONE CALLS ALL OF THESE SOURCES HAVE LEVERS THAT MANIPULATE THEIR QUALITY & LIKELIHOOD TO BECOME BUYERS 39
  40. 40. EMAIL OPTIMIZATION LEVERSEMAILS SENT LEVER 1: List Size Increase List Size _ Completely _ Optimize CR on dependent on subscription page conversion rate of _ Increase sales to lure OPENED subscription form repeat buyers EMAILS LEVER 2: Open Rate Test Subject Line LEVER 3: CTR Test Email Copy LINKS CLICKED 40
  41. 41. EMAIL OPTIMIZATION LEVERSEMAILS SENT LEVER 1: List Size Increase List Size LEVER 2: Open Rate Test Subject Line Quantity Opened _ Use an email application OPEN = OPENED with A/B testing RATE Quantity Sent _ Call to action is EMAILS important LEVER 3: CTR Test Email Copy LINKS CLICKED 41
  42. 42. EMAIL OPTIMIZATION LEVERSEMAILS SENT LEVER 1: List Size Increase List Size LEVER 2: Open Rate Test Subject Line LEVER 3: CTR Test Email Copy OPENED Clicks from Email _ Treat as a landing page EMAILS CTR = Quantity Opened _ Test layout, buttons, content, etc. LINKS CLICKED 42
  43. 43. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget _ Directly drive _ Optimize for maximum conversions factoring in conversions at current CPC conversion rate & CPC LEVER 2: Bids Optimize for KPI AD CLICKS LEVER 3: Quality Score Maximize QS LEVER 4: CTR Test Ad Copy 43
  44. 44. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI Ranking Score of Position Below Actual Cost = Quality Score of Your Ad + $0.01 AD CLICKS LEVER 3: Quality Score Maximize QS LEVER 4: CTR Test Ad Copy 44
  45. 45. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI LEVER 3: Quality Score Maximize QS _ Function of ad content, _ Relevant ad AD CLICKS landing page content, _ Relevant landing page historical CTR, & more _ Display URL LEVER 4: CTR Test Ad Copy 45
  46. 46. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI LEVER 3: Quality Score Maximize QS AD CLICKS LEVER 4: CTR Test Ad Copy Clicks from Ad _ Continuously A/B test ad CTR = Ad Impressions _ Keep highest CTR ad _ Delete lower CTR ads _ Repeat until returns diminish 46
  47. 47. SOCIAL CONTENTOPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks _ Be there in the first place _ Pinterest _ Reddit _ OthersSOCIAL CONTENT CLICKS LEVER 2: Content Quantity/Quality LEVER 3: Interactions Followers/Frequency 47
  48. 48. SOCIAL CONTENTOPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks LEVER 2: Content Quantity/Quality _ Links to deep content _ Frequency _ Polite, relevant posts _ Better content drivesSOCIAL CONTENT more shares/retweets CLICKS LEVER 3: Interactions Followers/Frequency 48
  49. 49. SOCIAL CONTENTOPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks LEVER 2: Content Quantity/Quality LEVER 3: Interactions Followers/FrequencySOCIAL CONTENT _ More interactions drives _ Increase quantity and CLICKS more clicks value of followers _ Don’t “buy” followers 49
  50. 50. WEBSITE LAYOUT & CONTENT VISITS LEVER 1: Layout Forms, Phone #s LEVER 2: Content Quantity, Quality LEVER 3: Click Areas Size, Color, PositionCONVERSIONS LEVER X: etc..... 50
  51. 51. EXECUTION 51
  52. 52. THE WEB IS A SYSTEM_ Your marketing strategy is a system of equations_ Some levers are independent of others_ Some levers manipulate others_ Some levers have a larger impact on KPI than othersAd Bid SEO Email Social Spend Subject Effort 52
  53. 53. THE WEB IS A SYSTEM_ Think about moving several levers at once_ Always consider what a single and multiple-lever move will do tothe system, and accordingly, your end result_ Design a good experiment, continue to measure, and repeaty = f( x1, x2, x3, . . . . xn ) 53
  54. 54. EXAMPLE SCENARIO_ Current Budget: $7,500 / month ( $250 / day )_ Conversions: 48 / month ( 1.6 / day )_ Average CPA = $158.17_ Average CPC = $2.29_ Landing Page CR = 1.45%New Target: 10 Conversions / Day AND Lower CPA 54
  55. 55. SCENARIO ONE:ADJUSTING BUDGET_ Average CPA at $158.17 means 10 conversions per day is $1581.70_ $47,000 per month_ Easy, but too expensive 55
  56. 56. SCENARIO TWO:ADJUSTING CPC_ Budget stays at $250 / day, conversion rate steady at 1.45%_ Need 689 clicks to get 10 conversions_ Average CPC must be $0.36_ Improbable to bring $2.29 down to $0.36 in this space whilemaintaining volume 56
  57. 57. SCENARIO THREE:ADJUST CONVERSION RATE_ Budget stays at $250 / day, CPC at $2.29_ Can purchase 110 clicks for $250_ To get 10 conversions, landing page needs to convert at 9.1%_ Improbable to quickly jump from 1.45% to 9.1% 57
  58. 58. FINAL SCENARIO:MOVE ALL THREE LEVERS_ Double the budget to $15,000_ Reduce average CPC by 50% ( $2.29 to $1.15 ) via QS & Keywords_ Improve landing page CR from 1.45% to 2.3% via A/B testing_ ( $15k / $1.15 per click ) at 2.3% drives 300 conversions monthly ata CPA of $50_ Used 3 levers simultaneously to improve conversions6x and reduce average CPA by 68% 58
  59. 59. ITERATING 59
  60. 60. CREATE A CULTURE OFCONTINUOUS IMPROVEMENT_ Define success_ Measure current performance and compare to targets_ Determine biggest levers and how they contribute to KPIs_ Design an experiment_ Test repeatedly_ Finalize improvements based on adequate data_ Repeat 60
  61. 61. QUESTIONS? Scott Yacko - scott@vuurr.com - @scottmyacko Jonathan Kressaty - jonathan@vuurr.com - @kressaty Code & Presentation at vuurr.com/bolo 61

    Be the first to comment

    Login to see the comments

  • cindykingbiz

    Oct. 13, 2012
  • inirs

    Oct. 17, 2012
  • jcolman

    May. 25, 2013
  • thenemesis

    Nov. 7, 2013
  • NicoRoddz

    Jan. 3, 2014
  • lcostafm

    Jun. 9, 2014
  • andrebf

    Jun. 9, 2014
  • felipefcb

    Jun. 9, 2014
  • KrishnaDelToso

    Jun. 10, 2014
  • vauxhall1979

    Jun. 10, 2014
  • hennerfehr

    Jun. 23, 2014
  • jcvangent

    Jul. 4, 2014
  • Nodesktophero

    Jul. 7, 2014
  • pmartella

    Apr. 3, 2015
  • slacabos

    May. 5, 2015

The prevailing culture of “Ship the MVP!” – where sites and apps are built in the cheapest, fastest way possible without regard for the future – needs to end. In this session, Vuurr will show you how to build web properties right the first time, specifically emphasising analytics and data collection. Learn how to properly configure many analytics tools, track offline interactions like phone calls and tangible marketing efforts, and structure onsite code and content. In addition, you will learn what to do after data has been collected and it’s time to iterate. We’ll show you how these strategies can lead to better decisions by you and your clients.

Views

Total views

10,291

On Slideshare

0

From embeds

0

Number of embeds

6,933

Actions

Downloads

0

Shares

0

Comments

0

Likes

15

×