This document discusses combining web analytics and user testing methods to provide more comprehensive insights. It outlines how each method has limitations on its own but provides complementary quantitative and qualitative data when used together. Specific examples show how web analytics can help focus user research activities like participant recruitment and test scenarios, while qualitative findings from user testing help interpret web analytics metrics. The overall message is that combining these methods allows telling a stronger story backed by both data and insights.

1. Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010

2. Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010 Who are we..? Web Analytics User Research Why you should combine methods Practical examples Why you should try it too

3. About User Intelligence

4. Martijn Klompenhouwer UX researcher 11 years UX experience 8 years at User Intelligence

5. Adam Cox Web Analyst 7 years UX / Analyst experience 3 years at User Intelligence

8. About Web Analytics

10. What does the definition mean? Where are your users coming from? What are they doing? Where and when are they leaving?

11. Some examples of Web Analytic measurements Referrals Visitors, visits and page views Popular pages Bounce rates Funnel analysis Path Analysis

12. Issues with Web Analytics  Data often perceived as boring statistics  Implementation of WA tool not always correct  Reports: just the facts, no insights, no actions

13. Web Analytics is quantitative in nature To get insights, interpretation of the numbers is needed

14. About User Research

15. Some examples of User Research methods Field observations Personas Diary studies Card sorting Expert Review Interviews Focus groups User Testing

16. Issues with User Research methods  Data usually from small numbers  Most methods take a snapshot in time  Difficult to capture some behavior  Setting sometimes artificial (e.g. lab test)

17. User Research is qualitative in nature You get the ‘why’, but not the ‘big numbers’…

18. So, why should you combine these two methods?  User Research findings can help interpret web data  Web data can help focus the User Research  More certainty of findings (based on two sources)

19.  The methods complement each other Quantitative vs. Qualitative

20. Let’s clarify with some examples…

21. Example 1: Recruiting the right participants Visitor Analysis Analyzing current traffic on website gives clues about audience Target audience Who to invite to participate in a test? Better insights More knowledge on users helped in defining recruitment profiles

22. Example 2: Test scenarios User journeys Homepage was not the main entry point Writing test script Creating realistic test scenarios: how is site used now..? Google scenario Testing outcomes of a common scenario

23. Example 3: Explaining abandonment rates Funnel analysis Sales funnel analysis revealed in which steps users left the flow Target test tasks Knowledge enabled us to concentrate usability test on those steps More focus Finding reasons for abandonment and looking for solutions

24. Example 4: Use of Advanced features Not measured Use of feature was known, but no data on the use of settings Popular settings? Which options are used and are there any patterns or issues? Collect data Decided not to use too much test time: wait for proper measurements

25. Example 5: Validating findings Impact analysis Quantified issue using data of thousands of visitors Big issue? Only 2 out of 10 test participants had this problem End of discussion Discussion changed from: “Is that an issue?” to: “Let’s solve it!”

26. Some additional examples… Effectiveness of segmentation Verifying user feedback Unintended user-flows Interpreting bounce rate data

27. Why you should try it too  Combining methods works! You can tell a story backed up with data  One report: Clear actions and no conflicting recommendations  Useful in different stages of projects (Research, Design, Optimize)  The combination works both ways! Web Analyst and User Researcher benefit from each other  Web Analytics can be used with many User Research methods

28. Just remember…  Fruitful results can already be gained from basic analysis You don’t need to be a Web Analytics expert, but it helps ;-)  Web Analytics tools do not magically provide the insights...  Measure the impact of changes (Optimize)  The UX team should make more use of Web Analytics! It shouldn’t just belong to the IT or Marketing departments

29. What we want you to take away…  Find out what web data is available  Use it!..... Just try it! No more excuses ;-)  Web data will help you in different stages of a project  If important data is missing, try to get it measured in the future  Integrate Web Analytics into your process and methods

30. user intelligence Amsterdam office www.userintelligence.com Thank you! Martijn Klompenhouwer [email_address] Adam Cox [email_address]