1. Making sense of the data
Collaborative techniques for
analyzing observations
Dana Chisnell
@danachis
#makingsense
1
2. What I’m talking about
In-time reporting, collaboratively
Shortcut to persona attributes
Ending the opinion wars through team analysis
Democratic, fast prioritizing
2
16. Dennis
Building contractor
Cell phone is for calling home
Very tuned in to current events
Sees no usefulness in Facebook
Recently more absent-minded
Difficulty doing standard calculations
30. Ask the right questions
• persistence with tech
• tolerance for risk & experimentation
• how patient, how easily frustrated
• tech savviness, expertise
• strength of tech vocabulary
• physical or cognitive abilities
31. • Attitude
motivation, emotion, risk tolerance,
persistence, optimism or pessimism
• Aptitude
current knowledge, ability to make
inferences, expertise
• Ability
physical and cognitive attributes
37. Ask the right questions
• Who is the most persistent when it comes to working
with technology?
• Who is the most likely to experiment and create
workarounds when something doesn’t work the way
they expect?
• Who do you think will give up when they encounter
frustrations out of impatience?
38. Ask the right questions
• Which one has the most tech expertise? Which one
strikes you as the least tech savvy?
• Which one is the most likely to call tech support to help
them get out of some tech pickle?
• Who will have the most advanced tech vocabulary
when they do call for support?
39. Matthew
Attorney
House in Tel Aviv, married
Assistant books reservations
Loves his Blackberry
Hates email
Reads NYTimes on iPad
Addicted to Words With Friends
Doesn’t spend a lot of time on the Web
Avid hiker and birder
Bad knees
Needs Rx eyepiece for scope
43. Dennis
Building contractor
Married to a nurse
They have 2 kids
8-year-old cell phone
Gets current events on the Web
Sees no usefulness in Facebook
Trouble sleeping, easily distracted
Served in the military:
Blunt Force Brain Trauma
45. Jane
1,000-acre farm
12 different systems every day
Tweets soil chemical readings
Smartphone alerts from exchanges
Tablet instrumented to aggregate data
Minimized manual input
Arthritic thumbs
Needs stronger progressive lenses
49. Subtlety of
affordances
Size of targets
Directness of the
happy path
Feedback modalities
Labeling & trigger
words
Amount of copy
Wording of
instructions &
messages
50. If you had this tool, what would you do
next?
What would be different for your
design?
51. A3 Persona modeler
• Designing for attitude, aptitude, & ability
• Accounting for what the user brings to the design
• Checking that personas cover the right things
52. Big ideas
Persona modeler: fast way to visualize users by
asking important questions
framework for talking about who users are
can tell which users might be missing
works with any amount of data
middle ground between demographics and
research-based personas
53. What obstacles do teams face
in delivering the best possible
user experiences?
62. Observations
Sources:
What you saw usability testing
What you heard user research
sales feedback
support calls and emails
training
63. - Participants are drawn to
open areas when they are
trying to communicate with
other attendees
- Participants are drawn to
the first open area they see
Inferences
64. Inferences
Judgements
Conclusions
Guesses
Intuition
+ Weight of evidence
65. Inferences
Review the inferences
What are the causes?
How likely is this inference to be the
cause?
How often did the observation happen?
Are there any patterns in what kinds of
users had issues?
66. - Make the response area smaller until it has content
Direction
67. Direction
What’s the evidence for a design change?
What does the strength of the cause
suggest about a solution?
Test theories
68.
69.
70. Big ideas: Making sense of the data
Observation Inference Direction
What happened Why there’s a A theory about
gap between what to do
behavior & UI
74. 1. Focus
question
What needs to be fixed in
Product X to improve the user
experience?
(observations, data)
What obstacles do teams face in
implementing UX practices?
(opinion)
75. 2. Organize
the group
Call together everyone
concerned
For user research, only those
who observed
Typically takes an hour
76. 3. Put opinions
or data on
For a usability test, ask for
observations
(not inferences, not opinion)
No discussion
77. 4. Put notes
on a wall
Random
Read others’
Add items
No discussion
78. 5. Group
similar items
In another part of the room
Start with 2 items that seem like
they belong together
Place them together, one below
the other
Move all stickies
Review and move among groups
Every item in a group
No discussion
79. 6. Name each
group
Use a different color
Each person gives each group a
name
Names must be noun clusters
Split groups
Join groups
Everyone must give every group
a name
No discussion
80. 7. Vote for the
most important
Democratic sharing of opinions
Independent of influence or
coercion
Each person writes their top 3
Rank the choices from most to
least important
Record votes on the group sticky
No discussion
81. 8. Rank the
groups
Pull notes with votes
Order by the number of votes
Read off the groups
Discuss combining groups
Agreement must be unanimous
Add combined groups’ votes
together
Stop at 3-5 clear top priorities
82. Big ideas, so far
Goal: share the most important observations
Focus: identify priorities
democratic
quick
83. Big idea: Collaborative analysis
build consensus in real time: rolling issues
observers contribute to identifying issues
you learn constraints
instant reporting
model users: A3 persona modeler
attitude
aptitude
ability
make sense of the data, together
observation: what you heard, saw
inference: why the gap between the UI and the behavior
direction: a theory about what to do about it
85. Where to learn more
Dana’s blog: http://
usabilitytesting.wordpress.com
Download templates, examples, and
links to other resources from
www.wiley.com/go/usabilitytesting
