Get Going with Users sessions held on 30th June 2015 in London.

1. #users1st
Get Going with Users
30 JUNE 2015

2. #users1st
Objectives
By the end of this workshop you will be able to…
 get buy-in for remote testing
 design your own tests
 launch them
 analyse them
 and present results internally.

3. #users1st
Agenda
 Why do you need to do remote UX Testing? [10’]
 Test Design and Launch [30’]
 Other Test Types & Number of Users [20’]
Getting buy-in [20’]
 Live analysis [30’]
 Presenting results [10’]
 Q&A + UX Clinic […]

4. #users1st
1. What is Remote UX Testing?

5. #users1st
WhatUsersDo
Founded 6 years ago
N. London office
20+ Full-time staff
20+ remote analysts
11,000+ panelists
1,500+ clients
250,000+ user videos
WhatUsersDo.com

6. #users1st

7. #users1st

8. #users1st
In essence…
 Capturing in the moment experiences
 With spoken thoughts
 From profiled panel members

9. #users1st
What are the Pros & Cons of this
approach?

10. #users1st
2. Let’s design and launch a live test
 What’s involved in designing a test?
 7 tips for test design
 2 example tests
 Each table writes one test
 We launch the test

11. #users1st
What’s involved in designing a test?
 Setting objectives
 Deciding on what type of test to run
 Participants: Profile, Volume and Devices
 Write tasks for users to follow
 Exit question

12. #users1st
7 Tips for test design
1. Set a scenario
2. Focus on tasks NOT opinions
3. Number each task
4. You don’t need to start at your Home Page
5. Aim for 20 minutes(ish)
6. Use fake personal data, stop at the point of payment
7. Run a pilot to check your tasks

13. #users1st
Meta-tasks
Start the screen recorder and click on the displayed link.
Please make sure you follow every instruction and talk through your
experience while you provide us with your opinions.
Remember, we are testing the site, not you.
[…]
Thank you – you can stop the screen recorder.

14. #users1st
1. Pretend you are moving to Reading and for your new home will
need internet, TV and phone service.
2. Using as many websites as you like find a plan (or plans) that
genuinely suit your needs.
3. At the end, please say which plan you would choose and why.

15. #users1st
1. Pretend you’re looking to re-mortgage and you land on this
website
2. Without thinking too much, say what three words come to mind
about [the brand]?
3. Would you give your personal details if requested by this site?
4. Imagine that you were searching for a mortgage to buy a new
home. Where would you click first on this page?
5. Explore the mortgage offers, as if you would be genuinely
interested in applying for a mortgage.
6. Apply for a mortgage, and proceed as far as you can. You can use
fake details, if you want.
7. If you had a magic wand how would you improve our website?

16. #users1st
Your mission [20 m]
1. Decide on a website to test: general consumer in the UK is best
2. What do you want to find out?
3. Write a few tasks for users (desktop only) + one person types
them
4. Launch them on the platform!
u: missing
p: missing
5 x UK users (keep the demographics broad)

17. #users1st
Top 9 Test Types
1. Live site key journey to identify points of friction
2. Pre-live design assets e.g. Axure prototype to de-risk change
3. Cross device RWD
4. How Users Search starting at Google
5. Specific pages/journeys to inform AB Testing*
6. Email campaign -> online journey testing
7. Competitor benchmarking
8. Scaling Lab insights
9. International local language
We’ll send you 10 with Tasks & a list by business question

18. #users1st
How many participants?
 Usually 10 per segment for key journeys
 5 if it’s very specific e.g. a specific area of a prototype
 Usually multiplied by device types
 20+ for Card Sorting/Tree Testing
 50+ for user research or marketing type tests
 100+ full scale Abandonment Audit, Digital Index

19. #users1st
Choosing participants
 Online panel: Demographics + screener
 or/and your customers: Private Panel
 or/and Special Recruit: anywhere
We’ll send you the Panel Book

20. #users1st
Activity (5 minutes)
 Share your burning UX issues or projects
 How we would design a test

21. #users1st
4. Getting buy-in
 Benefits and Objections
 3 x Case Studies you can use
 Role play: sell it to me!

22. #users1st
Benefits & Objections
 Discuss (we’ll write these up)

23. #users1st
Case Study
 3 x Case Studies:
 Fashion retailer = to show how UX testing makes sense of analytics
 AO.com = to show how it can improve conversions
 Shearings = to show how it can improve marketing

24. #users1st
Fashion retailer Case Study

25. #users1st

26. #users1st
AO.com Case Study
 Like you they had a development list/product roadmap
 Based upon hunches and opinion
 Prioritised by HiPPOs

27. #users1st
User eXposure
 Matthew Lawson insisted everyone watch users buy white goods
On their site, on competitors’ sites, starting at Google
 Listed the top issues and re-wrote the development list
 In a few months made changes that:
 Increased online sales by 9.5%
 Reduced contact by 33%
 Increased engagement (product reviews) 12%
 Read the full Case Study

28. #users1st

29. #users1st

30. #users1st
Pizza + User Videos = Buy-in

31. #users1st
Shearings Case Study
 2012 TV advert very successful
 New 2013 TV advert performed poorly

32. #users1st
Shearings Case Study
 25 users watching both versions
 Images changed too rapidly
"Using WUD to A/B test our TV campaigns has been an extremely quick
an efficient way to radically enhance the performance our campaign.”
Chris Barker, Head of Marketing, Shearings

33. #users1st
Getting buy-in
 Rehearse the benefits and objections
 Use 3 x Case Studies: Analytics, Conversions & Marketing
 Use your free WhatUsersDo video
 Bonus: call it Digital Customer Experience Testing

34. #users1st
5. Live Analysis
 Tag types
 Live Tagging

35. #users1st
Tag Types: issues
High: conversion killers, that block users or can prevent people from
converting in real life.
Medium: may become conversion killers when they occur more than
once.
Low: small niggles that slow down users and decrease trust in the site.

36. #users1st
Tag Types
Positive: highlights in the experience, moments where users go “wow”.
We don’t tag when the interface is just doing what it’s supposed to do.
Suggestions: when users suggest an improvement.
Other: everything else worth noting.

37. #users1st
Now let’s give it a try
Show your hand when
you see something interesting
http://app.whatusersdo.com/videoplayer/test/71b23dd3-90ca-
23bc?html5=1

38. #users1st
6. Compelling Presentations
 6 Steps:
Extracting insight from the platform
Creating a presentation

39. #users1st
6 Steps to Build Presentations
1. Read the WUD platform synopsis [MS clients]
2. Watch playlist + star clips
3. PPT: Replay the test objectives, tasks & participants
4. PPT: Count of issues by severity and type
5. PPT: Exec Summary of 5 main findings
6. PPT: Each issue: problem & recommendation
With a screen shot
With a video clip (from those you starred)
“User quotes”
Written verbosely so non-attendees can understand it

40. #users1st
1. Read the WUD platform synopsis
(Managed Service clients only)
• The written synopsis includes:
• Overall user experience, including trends
• Top 5 positives
• Top 5 issues
• Any user suggestions

41. #users1st
Use the synopsis as a springboard to explore the videos in the
WhatUsersDo platform.

42. #users1st
2. Watch playlist + star clips
Watch a playlist of all the video tags from your project, prioritised by high,
medium and low severity issues, positives and user suggestions.
Favourite the “killer clips” which support the main issues you need to
address.
Download your favourites in a CSV file, or download individual clips, to
embed them into your presentations.
http://app.whatusersdo.com/client/tests/10668

43. #users1st
3. PPT: Replay the test objectives, tasks & participants
Replaying the test objectives and scope
of test is great for reminding those on
the project, as well as informing
stakeholders who might review many
different supplier reports.
It helps “set the scene”.

44. #users1st
4. PPT: Count of issues by severity and type
Screengrab the coloured bars in the
WUD platform, and embed into your
presentation, to illustrate number of
issues and issue weighting (high,
medium and low severity issues, and
positives).
Use icons to visually illustrate what type
of issues caused the most problems for
users.

45. #users1st
5. PPT: Exec Summary of 5 main findings
Briefly summarise the top 4 or 5 issues
from the project which users
experienced, and visually illustrate them
with icons.

46. #users1st
6. PPT: Each issue: problem & recommendation
Expand upon each issue in detail.
Illustrate the issue with:
- Screengrabs
- User quotes
- “Killer” clips
Follow up with recommended Action to
take, and what the expected outcome
would be if that action was taken.
This will give you a really compelling
deck of slides, to overcome stakeholder
objections, and achieve buy-in for
necessary change or further testing.

47. #users1st

48. #users1st
We’ll send you the template

49. #users1st
Sum-up
We’ve now covered…
 get buy-in for remote testing
 know what you can test
 design your own tests
 launch them
 analyse them
 and present results internally.

50. #users1st
You can do this…
Self Serve
barend@whatusersdo.com
= DIY, occasional.
Managed Service
nathan@whatusersdo.com
= Expert Team, BAU.
or a blend.

51. #users1st
Q&A

52. #users1st
What next?
 We’ll send
accounts with a free credit coming your way Tuesday &
this Deck &
 Handbook
 Results PPT template
 Panel Book
 10 test types with tasks.
If you please complete the post event survey!

53. #users1st
“Shut up & listen to users.”
-- Dr Jakob Nielsen