Doing user research before and during development helps inform your choices about strategy (what to build) as well as tactics (how to build it)-- and it doesn't have to slow down your development process . In fact some rapidly executed research can speed up your time to market by reducing the need to refactor late in a project.
This presentation includes practical information to help product owners and developers quickly get inside the heads of their users, validate product ideas and improve the usability of their software at warp speed. The talk included tips and techniques for recruiting research participants, shadowing and interviewing users effectively, getting valuable feedback on product concepts and information architecture, and rapidly iterating on the user interface to improve usability. They discussed remote testing tools that help teams evaluate if users can successfully achieve their goals with their designs, and reviewed best practices collecting feedback from users after launch.
Customers don’t always share our knowledge,
values, assumptions or interests.
User Research helps bridge the gaps
between developers and users
You might need user research
if you hear or see these phrases:
“… I think they want to do this….”
“I’d want it to work like this…”
“"ey asked for this feature...”
“I assume they want…”
HOW DO YOU DO
RAPID USER RESEARCH?
Methods to use depends on where
you are in your product lifecycle
And on what answers you need.
Understanding actual user behavior:
What are users doing? When? Where?
Understanding reasons for behavior
Why are users doing that?
Understanding user attitudes and opinions
How do your users feel about doing it?
Diﬀerent methods provide
diﬀerent insights: mix ‘em up
A few proven rapid methods
• Interviewing & Shadowing Users
• Concept Tests
What do we build?
• Card Sorts, Tree Tests & Click Tests
• RITE studies
How do we build it?
• Compare key metrics pre and post
• Product Experience Feedback
How did we do?
Rapid research requirements
1. Executive and team buy-in
2. Feedback from the right people
3. Eﬃcient data collection & analysis
4. Actionable, understandable insights
5. In-time reporting
• What roles do they play in relation
to the product?User Roles
• How would you describe them?
• Any relevant skills & knowledge?
• What’s special about their
• What are they trying to achieve?
• How do they feel about these
• What do they need to accomplish
• What needs aren’t being met?
• What are they doing now?
• What can you improve?Pain Points
Empathizing with users’
pain and frustration
Pain scale (adapted from Hyperbole and a Half)
Finding the right people:
push the right lever
– Explore needs,
– Obtain recollections
– Gather feedback
asked you to
• Image copyright Wil Wheaton CC BY-NC-SA 2.0
Behavior: Observing vs. Asking
What people say,
what people do, and
what they say they do
are entirely diﬀerent
Task Card: [task description]
Performed by Role: [role name]
Context of Use:
q Where and when is it performed?
q In what environment?
q What corporate culture?
q Where in development process?
q Direction of information ﬂow?
q Device constraints/ media channels?
q Needs for
q Accuracy & Credibility
q Intensity of use
q Timeframe to act
q Who controls the process?
q Legal/regulatory restrictions
q Operational/safety risks
q Other roles involved:
ROLE: Business Owner
TASK: Approve visual design direction
CONTEXT: Waterfall dev process. Supervises
multiple product managers. Frequently mobile; uses
CHARACTERISTICS: Short attention span.
Under significant time pressure.
Focuses on visuals and metrics.
CONTENT CRITERIA: Brief, clear presentation in
common formats consumable on mobile devices
TASK:Approve visual design direction
Communicating insights from a
and design ideas
How do they feel about our concept(s)?
Do they think our ideas make sense?
Will our concept work for them?
What features do users value?
with a narrative,
storyboard, UI concept,
prototype, demo, video
Explore needs, rationales,
Gather feedback on ideas
Are these tables
the right ones?
Designcomics.org by Martin Hardee / Sun
Ques+on P1 P2 P3
How to organize the site architecture
What labels to use on the navigation
What kind of navigation do we use?
What visual design approach to use
Do users understand how to use the
site to accomplish their goals?
Does this product meet our quality
standards (prior to launch)?
How do we make
Tools for Rapid Remote Testing
Open online card sort interface
Darker clusters, more associations
Stands for Rapid Iterative
Testing and Evaluation.
USE IT TO:
• Identify and resolve
usability issues in an
levels of ﬁdelity through
• Improve and validate
ease of use.
1) Obvious Cause
& Solution With
Time to Design
3) Problem With
4) Issues Related To
Test Script Or
Include participant #,
Include # of
participant, issue, fix
Include # of
participant and issue
Include # of
participant and issue
scroll down to
P#2 expected to
P#2 didn’t know
that link was
clickable – add
P#3 unable to
• Participants must represent and/or share key characteristics with target users
• Conduct session in person or remotely using screen sharing applications.
• Decision-makers must attend all sessions because decisions are made after each one.
• Prototypes and task scripts may change during the study, so don’t collect success metrics
that depend on experimental rigor (i.e. time on task, error rate.)
• Use either concurrent or stimulated retrospective think aloud technique to understand users’
expectations for and understanding of design elements.
• Number of participants may vary depending on number of iterations needed.
• Leave some time between sessions to allow for debrieﬁng and making design changes. Try
scheduling a day between every three or four sessions to allow for design changes that
require additional thought or time to implement.
1 or 2 participants per session, 1 facilitator, stakeholder observers.
MORE PRACTICAL INFORMATION
Using the RITE Method to improve products: a deﬁnition and a case study. .Medlock, M. C., et al.
(2002). Usability Professionals Association, Orlando FL July 2002
Do users understand how to accomplish
Are users satisﬁed with it?
Did we increase conversion/sales?
Are we keeping users engaged?...
• Interchangeable design
• Modules within a grid
• Complete features or
• Self-contained feature
within an existing platform
• I am so, so sorry.
Eﬃcient Cumulative Consistent
Sharable Retainable Contextual
product experience feedback
Product Experience Feedback
q Provides actionable insights
q Easy to share information with team members
q Knowledge retained in the company
q Doesn’t pollute the user experience
q Easy to implement, uses resources eﬃciently
q Contextual to speciﬁc feature of interest
q Consistent across product lines
q Measurable, trackable progress
q Construct validity, resistance to bias
Open ended responses
ü Problem reports
ü Other comments
Product Usability Scale
measuring perceptions of
Example template flow:
Collect bugs ﬁrst; then
group feedback by type
1.Have you experienced any problems or errors when using
[NAME OF FEATURE] in [PRODUCT NAME]?
2. Please describe any problems or errors you've noticed
while using [NAME OF FEATURE]
3. What, if anything, do you like most about [NAME OF
4. Do you have any ideas or suggestions for improving
[NAME OF FEATURE]?
5. If there is anything else you'd like us to know about the
[NAME OF FEATURE] in [PRODUCT NAME], tell us here.
Product Experience Ratings:
subjective experience metrics
6. Please rate how strongly you agree or disagree with
each of the following statements:
1) Strongly Disagree 2) Disagree 3) Neither agree nor disagree 4) Agree 5) Strongly Agree)
Ease of Use
Rapid + Valuable = 6 steps
1. Engage stakeholders early and often
2. Keep plan focused
3. Get a representative sample
4. Ask questions and listen without bias
5. Collect data eﬃciently (but follow up
hunches and surprises)
6. Share actionable ﬁndings
Send your ﬂames, ideas, and
comments to @avivaux or