The slides from my General Assembly workshop on January 13th, 2013 (https://generalassemb.ly/education/introduction-to-usability-testing-the-diy-approach)
ABOUT THIS WORKSHOP
Usability testing can quickly uncover areas of an interface that frustrate users and hurt business goals but many teams put it off due to budget, time, or training concerns.
This workshops will take you through a do-it-yourself approach to usability testing. We'll cover the basics (benefits, recruiting, and how to plan a test), learn how to facilitate a test to get reliable results, and how to use the testing results to move usability improvements forward. You'll walk away with the tools to hold a complete usability testing right away.
TAKEAWAYS
Learn why and when to hold usability testing
Learn practical tools and methods to overcome time, budget or training concerns that block user testing from happening
Shift the conversation from opinions and hunches to proven usability problems that your team can solve together
2. AGENDA
BY THE END OF THIS SESSION,
YOU WILL BE ABLE TO...
‣
‣
‣
Plan a usability test
Facilitate a test and identify usability problems
Decide which usability problems to tackle first
!2
3. INTRODUCTION
EVGENIA (JENNY) GRINBLO
UX SPECIALIST, FUTURE WORKSHOPS
Trained in ethnographic research
‣ Work with 12 devs, multiple projects
‣ Introduced usability testing inside our process
‣
!3
6. !6
AGENDA
WHAT WE’LL COVER
!
✓
✓
✓
✓
!
!
How to run a £3000 fool-proof test
The easiest, quickest, cheapest methods
Tips on facilitating, note-taking, & prioritising
How to run a usability test tomorrow
Resources & tools to keep learning
7. AGENDA
HOUSEKEEPING
‣
‣
‣
‣
‣
You don’t need laptops yet
These slides will be on SlideShare.net/JennyEvgenia
Questions always welcome
Short break after this part
If you need to leave early, please tell me now
!7
8. USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!8
9. USABILITY TESTING: WHAT & WHY
USER EXPERIENCE BUILDING BLOCKS
http://www.flickr.com/photos/7855449@N02/2779597129
Diagram by David Armano
10. USABILITY TESTING: WHAT & WHY
USABILITY IS A MAJOR BUILDING BLOCK OF A
GOOD EXPERIENCE
http://www.flickr.com/photos/7855449@N02/2779597129
Diagram by David Armano
11. USABILITY TESTING: WHAT & WHY
1. INTUITIVE (n.)
Using what one feels to be true without conscious reasoning
www.youtube.com
12. USABILITY TESTING: WHAT & WHY
NEEDING INSTRUCTIONS MEANS
SOMETHING COULD BE MORE SIMPLE.
!12
13. USABILITY TESTING: WHAT & WHY
2. EASY TO USE
Doesn’t make people feel stupid or inferior to the tool they are using.
Helps them feel in control.
www.errorwallofshame.tumblr.com
!13
16. !16
USABILITY TESTING: WHAT & WHY
67.45% = AVERAGE E-COMMERCE CART
ABANDONMENT RATE
July 28, 2013
http://baymard.com/lists/cart-abandonment-rate
17. USABILITY TESTING: WHAT & WHY
http://www.shopify.com/blog/8484093-why-online-retailers-are-losing-67-45-of-sales-and-what-to-do-about-it
!17
18. USABILITY TESTING: WHAT & WHY
http://www.shopify.com/blog/8484093-why-online-retailers-are-losing-67-45-of-sales-and-what-to-do-about-it
!18
19. !19
USABILITY TESTING: WHAT & WHY
DIFFERENT WAYS TO FIND PROBLEMS
Analytics
(Impersonal)
Survey
(Tricky)
Usability test
(Immediate, easy)
20. !20
USABILITY TESTING: WHAT & WHY
USABILITY TESTING BENEFITS
‣ Behavioral vs. Self-reported
‣ Opportunity to ask “why?”
‣ Motivates teammates
‣ Removes “designer bias”
‣ Can be done on prototypes
‣ Can be cheap & quick
21. USABILITY TESTING: WHAT & WHY
WHAT HAPPENS IN A USABILITY TEST?
Video: http://www.youtube.com/watch?v=QckIzHC99Xc
!21
22. !22
USABILITY TESTING: WHAT & WHY
USABILITY TESTING SETUP
‣
Who is there
‣
Equipment and setup
‣
Space
‣
Context
23. USABILITY TESTING: WHAT & WHY
MOST IMPORTANT:
TEST YOUR IDEAS REGULARLY WITH
REAL PEOPLE.
!23
24. !24
USABILITY TESTING: WHAT & WHY
USABILITY TESTING: STEPS
Recruit
Participants
Prepare a
test plan
Test
Analyse +
iterate
25. USABILITY TESTING: WHAT & WHY
HOW TO CHOOSE PARTICIPANTS
‣
Anyone who hasn’t seen the app
‣
Similar demographics to real users (whenever
possible)
!25
26. USABILITY TESTING: WHAT & WHY
CRITERIA TO CONSIDER
‣
Demographics: age, gender, tech ability
‣
Behaviour: new/ experienced users, habits,
loyalty
‣
Mindset: angry customers, feel strongly
about social media, insecure about
technology, philosophy about saving
!26
27. USABILITY TESTING: WHAT & WHY
WHERE DO WE FIND PARTICIPANTS?
‣
Friends, family, colleagues
‣
Social media followers
‣
Existing customers
‣
Streets and coffee shops
‣
Recruitment agencies (£££)
!27
28. !28
USABILITY TESTING: WHAT & WHY
WHEN DO WE TEST?
AS EARLY AS POSSIBLE
Idea stage
Definition stage
Exploring
Many ideas
High-level
“Dream world”
Focusing
Best ideas
Refined
Realistic
30. USABILITY TESTING: WHAT & WHY
The number of team members exposed to users
trumps the number of users you test with
http://www.uie.com/articles/user_exposure_hours/
!30
31. USABILITY TESTING: WHAT & WHY
SOMETIMES IT’S HARD TO ACCEPT YOUR
PRODUCT NEEDS HELP
https://twitter.com/adrianhungate/status/391954628184915970
!31
32. USABILITY TESTING: WHAT & WHY
FACE-TO-FACE “MAGIC:” WHEN YOU SEE SOMEONE IN
PAIN, YOU WANT TO HELP THEM
http://www.flickr.com/photos/martin_mcdonald/371884786/
!32
33. !33
USABILITY TESTING: WHAT & WHY
“
!
I was completely surprised at how
cumbersome the users experience with the app
was. Basic tasks were which was so obvious
to me were suddenly a huge barrier.
— DANIEL
Lead Developer, Future Workshops
34. !34
USABILITY TESTING: WHAT & WHY
!
“
... What the usability testing did for me was
really prove how much tech people can
lose sight of who they are building for.
It about going back to basics in some cases.
— DANIEL
Lead Developer, Future Workshops
35. USABILITY TESTING: WHAT & WHY
IN SUMMARY:
USABILITY TESTING BENEFITS
1. Highlights where our product isn’t intuitive / easy to use
2. Puts team in the shoes of users
3. Helps sell product improvements to stakeholders/ clients
4. Helps prioritise feature backlog
!
!35
37. USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!37
38. PREPARING A TEST PLAN
To run a test, you need
‣
Some representation of your product
‣
Preferably interactive
‣
Doesn’t have to be finished
‣
Or: competitor product!
‣
Idea of what you want to test
!38
39. !39
PREPARING A TEST PLAN
What do we test?
‣
General impressions
‣
Ease of use of critical features
‣
Features we’re not sure about (e.g., is the
help clear?)
‣
Response to design, iconography
Tip: start from high-level
when testing prototypes,
test the details with later
builds
40. !40
PREPARING A TEST PLAN
SET THE SCOPE
Wide
(Generative)
Narrow
(Validating)
WHY will people use this
product?
HOW do these people
use this product?
41. !41
PREPARING A TEST PLAN
SET THE SCOPE
Wide
(Generative)
Explore
How is the value
proposition of this tool fit
in with people’s lives
Confirm/iterate
Observing what people
do with this tool, in
their own environment
Narrow
(Validating)
Validate
How do people perform
specific task with this tool?
42. !42
PREPARING A TEST PLAN
SET THE SCOPE
First...
Wide
(Generative)
Explore
How is the value
proposition of this tool fit
in with people’s lives
Confirm/iterate
Observing what people
do with this tool, in
their own environment
Narrow
(Validating)
Validate
How do people perform
specific task with this tool?
45. !45
PREPARING A TEST PLAN
5 SECOND TEST QUESTIONS
• What is the purpose of the website/app?
• Name 3 things you can do with this
website
• Who does this site or app belong to?
• How do you feel about this organisation?
46. !46
PREPARING A TEST PLAN
SET THE SCOPE
Wide
(Generative)
Explore
How is the value
proposition of this tool fit
in with people’s lives
Now...
Confirm/iterate
Observing what people
do with this tool, in
their own environment
Narrow
(Validating)
Validate
How do people perform
specific task with this tool?
47. PREPARING A TEST PLAN
THE MORE DETAILED THE ARTIFACT, THE
MORE DETAILED THE FEEDBACK.
!47
48. !48
PREPARING A TEST PLAN
PREPARING A DETAILED USABILITY TEST
Recruit
Participants
Prepare a
test plan
Test
Analyse +
iterate
49. EXERCISE 1: CHOOSE A PRODUCT TO TEST
TIME
KEY OBJECTIVE(S)
STEPS
Choose an app or website to
focus on today
10 mins
1. As a group, choose a website or app you will
usability test
2. It can be your app/website but I must be able to
interact with it
3. Avoid sites that require logging in (unless you have
a login to share)
DELIVERABLE
A group decision about an app/website
!49
50. PREPARING A TEST PLAN
Test plan
!
‣
Tasks: What are we testing? Why?
‣
Success Paths: What do we expect to
happen?
‣
Scenarios: How can we find out if our
expectation is true?
!50
51. // TEST PLAN FOR:
// GROUP:
// GENERAL SCENARIO
!
TASK
TASK
// SCENARIO
!
// SUCCESS PATH
!
!
!
// INFO (passwords, etc)
// SUCCESS PATH
!
!
@ Template by @futureworkshops
// SCENARIO
// INFO (passwords, etc)
// PART:
52. // TEST PLAN FOR:
// GROUP:
// GENERAL SCENARIO
// PART:
!
Set up the motivation
What we are testing
TASK
TASK
// SCENARIO
// SCENARIO
Motivation to do the !
task + instructions
// INFO (passwords, etc)
!
!
// INFO (passwords, etc)
!
Information necessary for the task
// SUCCESS PATH
!
What we want/expect the user to do
// SUCCESS PATH
!
53. // TEST PLAN FOR:
// GROUP:
// GENERAL SCENARIO
// PART:
!
You need to cook a new recipe for the company potluck. You found a recipe online
and now you need to buy the ingredients and cook it.
TASK
Text note
TASK
// SCENARIO
Share a note via e-mail
// SCENARIO
You need to remember what to buy. Use
the app to make a shopping list you can
!
take to the supermarket.
You really liked the new recipe and you
think your friend Jenny will like it too.
!
Send the recipe to Jenny from within
the app.
// INFORMATION
// INFORMATION
!
Print-out of a recipe from the internet
// SUCCESS PATH
!
Jenny’s email:
jenny@futureworkshops.com
Don’t show this to the
// SUCCESS PATH
participant!
!
Create a text note and make a bulleted
list of ingredients
!
E-mail a note to a friend using swipe to
share
54. !54
PREPARING A TEST PLAN
Scenario Structure
!
‣
// SCENARIO
!
Your character
‣
Your motivation
‣
What you need to do
‣
Any details like login information, etc
55. PREPARING A TEST PLAN
!55
Scenario Tips
‣
Avoid using terminology from the
interface (e.g., search, tag, filter)
‣
Scenarios can have steps (e.g., find a
recipe and add it to your collection)
‣
Escalate the difficulty as the test
progresses
Tip: make the scenario
specific enough so
someone can relate to it,
but not specific enough to
exclude people
56. EXERCISE 2: CREATE A TEST PLAN
TIME
20 mins
KEY OBJECTIVE(S)
STEPS
Learn to write a non-biased test
that covers the features you
need tested
1. Using the Test Plan Worksheet, define 3 key tasks
that must be intuitive and easy to use in the
application
2. Identify the success paths for each task (what we
expect users to do)
3. Write specific tasks, then a general scenario
4. Run through the complete test once
DELIVERABLE
Filled out test plan worksheet, ready to be used in a test with a participant
!56
57. USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!57
58. !58
RUNNING A USABILITY TEST
USABILITY TEST STEPS
Recruit
Participants
Prepare a
test plan
Test
Analyse +
iterate
59. !59
RUNNING A USABILITY TEST
As a facilitator, your jobs are:
‣
“Tour guide”
‣
“Therapist”
‣
Burst the thought bubbles
From: Rocket Surgery Made Easy by Steve Krug
http://books.google.co.uk/books?id=9Q3OQVyX_-QC
60. RUNNING A USABILITY TEST
Facilitating
‣
Practice staying quiet
‣
Remind the participant to speak out loud
‣
Don’t give anything away
‣
Be reassuring, say thank you
‣
If they get stuck, wait (a little)
!60
61. RUNNING A USABILITY TEST
Facilitator Cheat-Sheet
If they forget to think aloud
‣
"What are you thinking right now?"
‣
"What are you finding confusing on this screen?"
‣
"I know it's an unusual thing to do but could I please ask you to
keep thinking out loud and talk as you go along? It really helps
me understand your experience better."
!61
62. RUNNING A USABILITY TEST
Facilitator Cheat-Sheet
If they get upset
‣
"This is very helpful for us, we are
uncovering a lot of problems in the design that
will help us make the app better.”
‣
You’re struggling with the same areas that
other participants found confusing - you’re
totally normal
!62
63. RUNNING A USABILITY TEST
Facilitator Cheat-Sheet
If they have a question
‣
"Please continue in the way that makes
sense to you. We will learn a lot from seeing
how you proceed with this without any help."
!63
64. RUNNING A USABILITY TEST
Facilitator Cheat-Sheet
The golden go-to
‣
"We expected to uncover problems in the
design. You are not doing anything
wrong - we knew there would be issues and
you are helping us find them."
!64
65. RUNNING A USABILITY TEST
PEOPLE WILL FEEL STUPID.
MAKE THEM FEEL AT EASE.
!65
66. RUNNING A USABILITY TEST
Facilitator checklist
Introduce yourself and anyone else present
Thank the participant for coming
Explain why they’re here & what will happen
“We are not testing you”
Introduce thinking aloud
Explain questions, breaks, and ask if they have any questions
Optional: consent forms, recording, interview
!66
67. RUNNING A USABILITY TEST
Ways to record our findings
From easy to complicated
‣
Pair up with a note-taker
‣
DIY recording equipment
‣
Professional recording equipment
‣
Live-stream to a group of stakeholders
!67
68. RUNNING A USABILITY TEST
NOTE-TAKING BRINGS DOWN 2 BARRIERS
TO TESTING: TIME & EQUIPMENT
(But I’ll give you some tips on recording later!)
!68
69. RUNNING A USABILITY TEST
Note-taking tips
Only record what’s important
‣
If someone is really happy
‣
If someone is really frustrated
‣
If someone fails a task
‣
Write down short quotations
!69
70. !70
RUNNING A USABILITY TEST
Notes
Example
•
!
•
!
•
Taps Map on merchant profile but this opens the system
Maps app — confused
Goes to Search and types “coffee shops” but there are no
results — “why can’t I see what’s around me?”
"If I had to think about something around me, I’d start
the merchant profile and expect to see other things
around me"
!
•
The icons aren’t telling me that Cafe Nero is a coffee shop
!
!
71. !71
RUNNING A USABILITY TEST
What to bring to the test
Printed scenarios & tasks for participant
Printed scenarios & tasks with success paths for facilitator
Facilitator script
Consent form (if needed)
Pen + paper / recording tools
Test setup checklist
Optional: List of pre-test and debrief questions
Tip: Prepare a printed copy
of the scenarios & steps for
the participant, they will
use it
72. RUNNING A USABILITY TEST
Setting up the test
Reset the mobile app / site to the default settings
Clear the test area of any open documents, written notes, etc.
Prepare participant forms: scenarios, intro script, consent
form
Optional: Verify the recording equipment
Optional: Make sure the brightness on the screen is reduced
!72
73. EXERCISE 3: USABILITY TEST
TIME
40 mins
KEY OBJECTIVE(S)
STEPS
1. Gain experience facilitating
1. Every round, swap one member of your group in
exchange for a member from another group
2. Collect notes containing
usability problems
3. Identify weaknesses in your
test plan
2. The ‘foreign’ member is the participant
3. Start running through the test you’ve planned
4. Everyone in the group should try to be a facilitator
at least once
DELIVERABLE
A collection of notes detailing usability problems found in the product
!73
76. USABILITY TESTING: THE DIY APPROACH
0 / USABILITY TESTING: WHAT & WHY
1 / PREPARING A TEST PLAN
2 / RUNNING A USABILITY TEST
3 / TURNING DATA INTO ACTIONS
!76
77. TURNING DATA INTO ACTIONS
!77
USABILITY TEST STEPS
Recruit
Participants
Prepare a
test plan
Test
Analyse +
iterate
78. TURNING DATA INTO ACTIONS
What to do with all the data?
‣
Identify changes with the biggest impact &
smallest effort as a team
‣
Find the “low-hanging fruit”
‣
What has impact on the bottom line?
‣
Put everything else in the ice-box
!78
79. TURNING DATA INTO ACTIONS
Identifying key findings
High-priority: people fail, or abandon the
task
Medium: people succeed but are frustrated
and unhappy
Low: people would enjoy the app more if...
!79
Tip: You can’t fix everything.
Pair usability priority with
business needs and product
roadmap to find the fixes
with the biggest impact.
81. TURNING DATA INTO ACTIONS
!81
Presenting findings
Task
What we expected
to happen
Interface screenshot
here
What really
happened
What we can fix
right now
82. EXERCISE 4: IDENTIFYING KEY FINDINGS
TIME
15 mins
KEY OBJECTIVE(S)
STEPS
1. Consider how to prioritise
usability problems
1. Individually, write down 2 high-level and 2 mediumlevel usability problems on post-its
2. Discuss and agree on
priorities as a team
2. As a group, arrange your post-its from high to low
priority.
3. Jot down a quick summary of key findings to
present to the larger group
DELIVERABLE
A group decision about an app/website
!82
84. SUMMARY & CONCLUSION
Summary
‣
Match your test artifact to your scope
‣
Plan realistic scenarios that don’t give
away the test purpose
‣
Be encouraging and open to feedback
‣
Record only what’s needed
‣
Prioritise findings for maximum impact
!84
85. SUMMARY & CONCLUSION
See “Resources and Furthe
r
Reading” printout for more
DIVE IN MORE DEEPLY
DIY Usability
Testing
Steve Krug
Lean UX
Jeff Gothelf
Remote Research
Nate Bolt & Tony Tulathimutte
!
!85
87. SUMMARY & CONCLUSION
A FINAL THOUGHT
“
Any intelligent fool can make things bigger
and more complex. It takes courage, and a
touch of genius, to move in the opposite
direction.
– Albert Einstein
http://www.humanfactors.com/ROIposter.asp
!87