4. @kaniasty | #uxpabos17
4
Bias is everywhere!
• Inter-evaluator bias
• Loaded or leading questions
• Cognitive biases (e.g. primacy, recency)
• Group dynamics (i.e. groupthink)
• Prejudgements
• Selection bias
• Respondent bias (e.g. positive, social)
“Soylent Green…
Bias…
it’s made of
people!”
5. @kaniasty | #uxpabos17
5
• Plan
• Structure
• Focus
• Automate
• Don’t rely on memory
Reduce Your Cognitive Load
“An ounce of prevention is worth a
pound of cure.” - Ben Franklin
8. @kaniasty | #uxpabos17
8
Recruiting
Recruiting Brief
Screening Survey
Scheduling
- Research Recruiter
- UserInterviews.com
- Stakeholders
Participant Data
- Contact Info
- Time Zones
- Screening Data
“Keep it simple…”
9. @kaniasty | #uxpabos17
9
Customer Super Screener (i.e. survey)
Better screening
- Larger pool of potential
participants
- Foot in the door
- Comparison sample
- Bias fighter
Measure twice, cut once
Collect data as you screen
- Demographics
- Feature usage
- Card sorts
- Desirability (product reaction)
- Technical
- Quotes (collected judiciously)
- Opt-in for future research
(online & moderated)*Don’t forget legal review
10. @kaniasty | #uxpabos17
10
Recruiting Logistics
Written Consent
•Context
•Logistics
•Recording
•Compensation
•(Optional) Nondisclosure
*Don’t forget legal review
“80 percent of success is just showing up.”
Reducing No-Shows
•Written consent
•Email confirmation
•Remote tool testing/setup
•Phone/Texting
Session Status Tracking
Compensation
16. @kaniasty | #uxpabos17
16
Recording & Photography
• Personal room
• Test meeting OR
Tech call
• Mac & Windows
compatible
Mobile Screen
Mobile Device
• Sleds
18. @kaniasty | #uxpabos17
18
The Session Summary
This participant had major issues with X and Y features
because she expected [X] and [Y] to happen. She followed an
interesting workflow that was different from what we
expected. She also did [this interesting or surprising thing].
She was hesitant to explore the UI and fit the ‘Ms. X’ persona
pretty well…
“Take 5.”
19. @kaniasty | #uxpabos17
19
Avoiding the Slow TV Syndrome
• Re-watching videos
• Relying on notes alone
• Human transcription
• Automatic transcription
VLC player
25. @kaniasty | #uxpabos17
25
Analysis & Reporting Strategy
Findings Report
- Narrative
- Includes detailed study
methodology & background
- Includes raw data
- Includes participant detail
- Includes session summaries
- Focused on detailed findings
Strategic Brief Presentation
- Focused on the big picture
- Focused on priorities
- Focused on critical findings
- Includes recommendations
- Visually impactful
- Includes quotes
- Synthesizes data from multiple
sources
- Summary deliverables:
(e.g. profiles, personas, user journeys)
“Use the right tool
for the job.”
27. @kaniasty | #uxpabos17
Participant Data
27
♀ 5 Female ♂ 5 Male
4 on Laptops 4 on Smartphones
(4 iPhone, 3 Android, 1 Windows Phone)
Customers
Company 1, Company 2,
Company 3
Customers RepresentedProfile 1 | Profile 2
Session Summaries
as part of detailed
findings / or with
recordings
32. @kaniasty | #uxpabos17
32
Infogr.am
PROS:
Range of Graphs
Fun viz options
Full control over colors
Option to show values (not %)
CONS:
Limited font control
No option to show confidence intervals
35. @kaniasty | #uxpabos17
35
Final Presentation
Strategic Brief
- Focused on the big picture
- Focused on priorities
- Focused on critical findings
- Includes recommendations
- Visually impactful
- Includes quotes
- Synthesizes data from multiple
sources
- Summary deliverables:
(e.g. profiles, personas, user journeys)
“Looks matter”