Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Combining Methods: Web Analytics and User Testing

Combining Methods: Web Analytics and User Testing. Presentation delivered at the UPA2010 in Munich

  • Login to see the comments

Combining Methods: Web Analytics and User Testing

  1. 1. Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010
  2. 2. Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010 Who are we..? Web Analytics User Research Why you should combine methods Practical examples Why you should try it too
  3. 3. About User Intelligence
  4. 4. Martijn Klompenhouwer UX researcher 11 years UX experience 8 years at User Intelligence
  5. 5. Adam Cox Web Analyst 7 years UX / Analyst experience 3 years at User Intelligence
  6. 6. User Intelligence <ul><li>It’s our goal to design interactive products that provide great User Experiences, based on knowledge of both the users and the business. </li></ul><ul><li>Services we offer: </li></ul><ul><li>Research </li></ul><ul><li>Design </li></ul><ul><li>Optimize </li></ul>
  7. 7. <ul><li>How to conduct Global User Research written by partners of the UXalliance and other experts </li></ul><ul><li>Available since November 2009 published by Morgan Kaufman </li></ul>Handbook of Global User Research
  8. 8. About Web Analytics
  9. 9. <ul><li>&quot;Web Analytics is the measurement , collection , analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage.“ - The Official WAA Definition of Web Analytics </li></ul>Web Analytics Definition Measurement Data Collection Analysis Reporting
  10. 10. What does the definition mean? Where are your users coming from? What are they doing? Where and when are they leaving?
  11. 11. Some examples of Web Analytic measurements Referrals Visitors, visits and page views Popular pages Bounce rates Funnel analysis Path Analysis
  12. 12. Issues with Web Analytics  Data often perceived as boring statistics  Implementation of WA tool not always correct  Reports: just the facts, no insights, no actions
  13. 13. Web Analytics is quantitative in nature To get insights, interpretation of the numbers is needed
  14. 14. About User Research
  15. 15. Some examples of User Research methods Field observations Personas Diary studies Card sorting Expert Review Interviews Focus groups User Testing
  16. 16. Issues with User Research methods  Data usually from small numbers  Most methods take a snapshot in time  Difficult to capture some behavior  Setting sometimes artificial (e.g. lab test)
  17. 17. User Research is qualitative in nature You get the ‘why’, but not the ‘big numbers’…
  18. 18. So, why should you combine these two methods?  User Research findings can help interpret web data  Web data can help focus the User Research  More certainty of findings (based on two sources)
  19. 19.  The methods complement each other Quantitative vs. Qualitative
  20. 20. Let’s clarify with some examples…
  21. 21. Example 1: Recruiting the right participants Visitor Analysis Analyzing current traffic on website gives clues about audience Target audience Who to invite to participate in a test? Better insights More knowledge on users helped in defining recruitment profiles
  22. 22. Example 2: Test scenarios User journeys Homepage was not the main entry point Writing test script Creating realistic test scenarios: how is site used now..? Google scenario Testing outcomes of a common scenario
  23. 23. Example 3: Explaining abandonment rates Funnel analysis Sales funnel analysis revealed in which steps users left the flow Target test tasks Knowledge enabled us to concentrate usability test on those steps More focus Finding reasons for abandonment and looking for solutions
  24. 24. Example 4: Use of Advanced features Not measured Use of feature was known, but no data on the use of settings Popular settings? Which options are used and are there any patterns or issues? Collect data Decided not to use too much test time: wait for proper measurements
  25. 25. Example 5: Validating findings Impact analysis Quantified issue using data of thousands of visitors Big issue? Only 2 out of 10 test participants had this problem End of discussion Discussion changed from: “Is that an issue?” to: “Let’s solve it!”
  26. 26. Some additional examples… Effectiveness of segmentation Verifying user feedback Unintended user-flows Interpreting bounce rate data
  27. 27. Why you should try it too  Combining methods works! You can tell a story backed up with data  One report: Clear actions and no conflicting recommendations  Useful in different stages of projects (Research, Design, Optimize)  The combination works both ways! Web Analyst and User Researcher benefit from each other  Web Analytics can be used with many User Research methods
  28. 28. Just remember…  Fruitful results can already be gained from basic analysis You don’t need to be a Web Analytics expert, but it helps ;-)  Web Analytics tools do not magically provide the insights...  Measure the impact of changes (Optimize)  The UX team should make more use of Web Analytics! It shouldn’t just belong to the IT or Marketing departments
  29. 29. What we want you to take away…  Find out what web data is available  Use it!..... Just try it! No more excuses ;-)  Web data will help you in different stages of a project  If important data is missing, try to get it measured in the future  Integrate Web Analytics into your process and methods
  30. 30. user intelligence Amsterdam office Thank you! Martijn Klompenhouwer [email_address] Adam Cox [email_address]