In this Webinar, Stephen Fleming-Prot, Principal UX Researcher, provides techniques to guide you through the sometimes rough waters of customer experience research in 2019. With executives demanding that their teams connect with customers and build empathy for their users, this webinar gives you actionable tactics to help you expand your cross-functional teams’ methods and approaches for research. You’ll learn: Guidance on “mapping” out a plan for 2019 Considerations for the “gear” and tools you need for the journey, including balancing quantitative and qualitative approaches to research New techniques to help you “navigate” your research needs Research considerations for dealing with new tech Tips on ensuring everyone is moving in the same direction - towards a better understanding of, and more empathy for customers

1. CX Survival Guide for
2019
UserTesting
January 23, 2019

2. 2
Introduction
Steve Fleming-Prot, Principal UX Researcher
UserTesting Professional Services
18 years of UX Research, Design, and Training Experience

3. 3
Introductions
Mapping out a plan
FULLSERVICERESEARCH
Agenda
Tools for your journey
Hone your techniques
Final tips as you start your journey

4. 4
UserTesting’s on-demand
Human Insight Platform
enables every organization to
make accurate customer-first
decisions at every level, at the
speed business demands

5. 5

6. 6
Gather human insights at every stage
What do users need?
What are they already using?
Are we on the right track? How are we
performing? How can
we improve?
Can they use it?
Interviews...
Competitive eval...
Needs
assessment...
Card sort...
Wireframe eval..
Tree test...
Benchmark study...
Preference test...
Longitudinal study...
Prototype study...
Feature validation...
Summative eval...
Discovery Design Monitor &
Optimize
Development

7. 7
You should care about customer feedback

8. 8
Mapping Out a Plan
USERTESTING

9. 9
Being proactive is better than being reactive
Formulate research plans that drive the
most important outcomes
Map out your business initiatives by
prioritizing them into:
● Now (next 2-4 weeks)
● Next (next 1-3 months)
● Later (3+ months)

10. If you don’t know what you are trying to do, trying
harder won’t help!
10
Know what you want to achieve
For each business initiative…
● Identify the user insights needed to inform each initiative
● Define the methodology that will get you those insights, given the time and resources
you have available
Prioritize the research that the project team is willing (and able) to act on.

11. 11
...So have a good plan

12. 12
Make it fancy...

13. 13
Or keep it simple
...just do it!

14. 14
Survival checklist - planning!
Gain agreement on the priorities for what’s “Now” and
what’s “Next”
Identify and define the insights needed to inform each
initiative - not research for research’s sake
Document your research plan (in as little or as much
detail as necessary)

15. 15
Tools For Your Journey
USERTESTING

16. 16
For any good journey, you need the right gear

17. 17
The right tool for the right job
Qualitative tools
● In-person capabilities
● Remote moderated
● Remote unmoderated
Quantitative tools
● Survey
● Instrumentation / analytics
● A/B testing capabilities
Recruiting tools
● Panel services
● Bodies to do guerilla recruiting
(Facebook, LinkedIn, conferences)

18. 18
Qualitative versus quantitative research
Qualitative research Quantitative research
Number of participants 6-8 per user group 30+ per user group
Statistical confidence Lower Higher
Deep detail per participant Yes No
Good for discovery Yes Yes
Good for design and build Yes No
Good for optimization Yes Yes
Good resource for quantifying the user experience: https://measuringu.com

19. 19
But let’s stop saying “versus”
Which approach is best?
A particular research effort may lend itself to one
or the other, but the best approach is a
combination:
● Understand actual behaviors through
quantitative methods
● Understand the “why” behind the numbers
with qualitative methods
Together, quantitative and qualitative data help
tell a comprehensive story about the user
experience.

20. 20
“Qual” and “Quant” support each other
When should you use qualitative research to support your quantitative
data?
● Before you collect quantitative data (e.g., piloting a survey)
● During the collection of quantitative data (parallel research activities)
● After you have collected quantitative data (insight into the results)

21. 21
Invest in qualitative tools as much as you invest
in quantitative tools
In UserTesting’s 2018 CX Industry survey, we found that respondents said their company puts
too much emphasis on quantitative evidence for business decisions.
Percent of companies reporting the
indicated success rate for their A/B tests
More than three-fourths of
respondents said it’s common for
them to have trouble explaining
customer attitudes or behaviors
found in surveys and analytics.
More than half of A/B tests fail to produce
a statistically significant result. The
failure rate was especially high in high
tech and retail.

22. 22
Survival checklist - tools!
Consolidate your resources and methods of finding
participants
Invest in tools for conducting moderated and
unmoderated research
Invest in tools that let you run qualitative and quantitative
research

23. 23
New Techniques
USERTESTING

24. 24
Techniques to navigate your research
Once you have the right tools, you need to
use them well...and creatively!
● Multiple touchpoints
● Omni-channel research
● Free exploration
● Leveraging learnings
● Research in a world full of new
technologies

25. 25
Using multiple touchpoint methodologies
Sometimes you need to collect data from
the same participant over time:
● Collect feedback from the same
participants as you iterate through your
design
● Collect diary feedback as your users
interact with your design over weeks

26. Day 1: Unboxing and first use
Day 2-6: Daily diary entries
Day 7: One-week check-in
Day 14: Two-week check-in
Day 15: Exit interview
26
It is a complex method
Caveats:
● Participants with great feedback might
not be able to commit to the time
requirement
● Expect some drop-off - even when
people do commit, the unexpected
happens
● Increased compensation may help
encourage participation - but it costs
more!
● You’ll have a lot of data to go through

27. 27
Within the UserTesting platform...
You can “Reuse” the same participants
for simple multiple touchpoint studies
● Override the Fresh Eyes feature to
have the same participants take
another study from you
● In addition to formal longitudinal
studies, provides ability to ask brief
follow-up questions or use an interview
to augment screening

28. Ensure you research each channel
Determine your baseline by researching the core experience on
a:
● Desktop/laptop
● Smartphone
● Tablet
● Watch
● TV
● And …..
And also:
● Same participant across different devices
● Different participants for each device
● Mobile app vs. mobile website 28

29. 29
Multiple touchpoints may include a mix of
moderated and unmoderated studies
For example:
● Participants come into the lab for
unboxing, then complete unmoderated
diary entries
● Participants complete a series of
unmoderated studies, but then a final
moderated exit interview to review the
individual insights from the unmod
sessions
● Or be completely unmoderated:
Participants complete an unmoderated
interview up front and then a series of
unmoderated “usage” sessions
Recording Mobile Device
Usability Testing Sessions -
Guerilla Style

30. 30
Sometimes we need field research
Field research isn’t new…
But budget and timeline can make it
prohibitive, even when the team needs the
data
Develop your techniques for capturing good
data remotely

31. 31

32. 32
Field research allows us to study Voice UIs
Voice UIs are built to be used “in the
wild”, so be sure to test them there
● Ensure you have the capability to
capture the participant’s environment to
understand what impacts their voice
interactions
● Conduct discovery work to understand
the current experience, but also
evaluate potential updates
Future of Retail report from Walker Sands,
2017

33. 33
Sometimes we don’t know all the questions
Conduct free exploration studies to gain
insight into what the users naturally care
about
● Ask the participant to describe an
activity they would do given the asset
you want feedback on
● Then have them accomplish that
activity
Caveat:
● They may not explore or use the exact
feature you want feedback on, but that
is good data itself!

34. 34
Studying some technologies can be hard
Evaluating your AI experiences is crucial,
but can be challenging
● Set the stage with the right introduction
● Validate the perception of success
● Use the opportunity to gather emotional
feedback

35. Hi <customer name>,
We want to inform you of important
changes to your account. In order to serve
you better, we’ve made the following
updates:
● Custom seismographing
● PX synchronization
These updates will be made unless you opt
out before January 31, 2018.
35
Leverage learnings across studies
Execute consistent research on each new
property or topic so that each new round
of research provides data that builds on
the last round of data
● Gain a baseline for how the type of
design performs
● See trends over time
● Build a “quantity” of data on some
topics that are consistently covered in
each study
● Be able to refer back to existing insights
from previous studies
Making these learnings accessible to all
stakeholders ensures research is providing
compounding value and may save time and
money on future research

36. 36
Create a playbook that highlights key findings
Reinforce your design principles with research-based guidance
Or create your own documentation that leverages research insights to avoid re-inventing the
wheel and also to highlight to your stakeholders the actionable value of research

37. 37
Create a database to house individual learnings
Avoid creating shelf-ware
● Break your findings down into discrete
pieces of insight
● Include the data supporting those
insights, such as metrics or video clips

38. 38
Survival checklist - techniques!
Document your method for being able to conduct multi-
touchpoint studies
Even if you are mobile-first, include activities in your
roadmap for evaluating the experience across devices -
and beyond devices
Hone your techniques for research in the field
Build your repository of research insights

39. 39
Final Tips as You Start Your Journey
USERTESTING

40. 40
Build empathy for your customers
- every day!
1

41. 41
Invest in tools that provide the
most flexibility in the research you
can run
2

42. 42
Expand beyond single touchpoint
studies
3

43. 43
Use free exploration studies to
inform what you should be
working on
4

44. 44
Keep findings alive by leveraging
them for future work
5

45. 45
Questions?
For more information or questions please visit us at: usertesting.com/about-us/contact-us or email us at webinars@usertesting.com

46. 46
Thank you!
For more information or questions please visit us at: usertesting.com/about-us/contact-us or email us at webinars@usertesting.com