This document discusses 7 cognitive biases that researchers should be aware of in their work: selection bias, observer-expectancy effect, framing bias, confirmation bias, groupthink and bandwagon effect, anchoring bias, and clustering illusion. It provides examples of each bias and recommends ways to prepare for, conduct, and analyze research to mitigate the influence of these biases. This includes using multiple recruitment channels, keeping an open mindset, considering alternative hypotheses, and collaborating with other researchers.

1. The researcher’s blind spot:
7 cognitive biases we shouldn’t ignore
in research
UX Singapore 2016
Ruth Ellison, Principal User Researcher, PwC’s Experience Centre
@RuthEllison

2. @RuthEllison from PwC’s Digital Services
Hi
:)
2

3. @RuthEllison
Free!
$10
$7
$20

4. @RuthEllison
$1
$10
$8
$20

5. 5
Kiasu
Mr. Kiasu by Johnny Lau (Source: http://www.nlb.gov.sg/sure/wp-content/uploads/2013/10/Mr-Kiasu.jpg)

6. @RuthEllison from PwC’s Digital Services
The Zero Price Effect is a Cognitive Bias

7. @RuthEllison
What is a cognitive bias?
Mental shortcut

8. @RuthEllison 8
We are all human
Photo available under a CC by 2.0 licence https://www.flickr.com/photos/yellowbookltd/14907364551/

9. @RuthEllison Image source: http://www.businessinsider.com.au/what-new-coke-looked-like-in-1985-2013-3

10. @RuthEllison
Applying it to design research
Preparing
Running
Analysing

11. @RuthEllison from PwC’s Digital Services
Preparing for research

12. @RuthEllison

13. @RuthEllison from PwC’s Digital Services
Selection bias

14. @RuthEllison 14

15. @RuthEllison from PwC’s Digital Services
Multiple channels - mix of
recruitment companies, social
media, trusted networks
Avoid professional respondents
Careful screeners
Behavioural based recruitment
Dealing
with
it….

16. @RuthEllison from PwC’s Digital Services
Running research sessions
Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153

17. @RuthEllison from PwC’s Digital Services
Observer-Expectancy effect

18. @RuthEllison 18
Source: https://medium.com/@userfocus/the-7-deadly-sins-of-user-research-22857c5a971b#.xb38ukdtr

19. @RuthEllison
The way we frame our questions matter

20. @RuthEllison
Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and
Verbal Behavior, 1974, 13, 585-589.
McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html
About how fast were the cars going
when they smashed into each
other?

21. @RuthEllison
About how fast were the cars going when they
collided into each other?
About how fast were the cars going when they
bumped into each other?
About how fast were the cars going when they
contacted into each other?
About how fast were the cars going when they
smashed into each other?

22. @RuthEllison from PwC’s Digital Services
Triangulate research
Use observational methods
Keeping positive-neutral body
language, watch the tone of your
voice
Avoid leading questions
Dealing
with
it….

23. @RuthEllison 23
I need a volunteer please

24. 24
2
4
8

25. PwC’s Digital Services
What’s going on?

26. @RuthEllison from PwC’s Digital Services
Confirmation bias

27. @RuthEllison 27

28. 28
Take the opposing view

29. 29
Don’t just confirm your hypothesis, see if you can prove it wrong
Analysis of Competing Hypotheses (ACH)

30. 30
Use open ended questions
Some people think that soft drinks are bad for you.
What do you think?
What’s your opinion about soft drinks?

31. @RuthEllison from PwC’s Digital Services
List assumptions
Be skeptical, especially if everyone
agrees with you
Remain open
Consider all evidence equally
Multiple user researchers
Leave your ego by the door.
Dealing
with
it….
@RuthEllison from PwC’s Digital Services

32. @RuthEllison from PwC’s Digital Services
Groupthink & the Bandwagon effect

33. @RuthEllison 33

34. @RuthEllison from PwC’s Digital Services
Triangulate with other research
methods (e.g. observational)
Use a mixture of individual and
group exercises
Avoid stating preferences and
expectations at start
Give someone the devil’s advocate
role to question assumptions
Dealing
with
it….

35. @RuthEllison from PwC’s Digital Services
Anchoring bias

36. @RuthEllison 36

37. 37https://www.flickr.com/photos/naotakem/17105160150/

38. @RuthEllison from PwC’s Digital Services
Consider the order of questions and
designs carefully
Open ended questions
Alternate order in which participants
are shown concept or design versions
Show version A first
1 3 5
2 4 6 Show version B first
Dealing
with
it….

39. @RuthEllison from PwC’s Digital Services
Analysing

40. @RuthEllison 40

41. @RuthEllison
Apophenia
41Source: https://au.pinterest.com/pin/249738741807276910/

42. @RuthEllison from PwC’s Digital Services
Clustering illusion & reporting bias

43. @RuthEllison from PwC’s Digital Services
Sample sizes – it’s about the WHY
Consider evidence equally – not just
the ones that confirm your belief/
assumption
Collaborative analysis sessions
Strive for objectivity
Dealing
with
it….

44. @RuthEllison 44
How vulnerable are you to
these biases?

45. @RuthEllison from PwC’s Digital Services
What percentage of the
population will be more biased
than the average person?

46. @RuthEllison from PwC’s Digital Services
Are you more biased than the
average person?

47. @RuthEllison from PwC’s Digital Services
Blind spot bias

48. @RuthEllison from PwC’s Digital Services
Listen with an open
mind
Become more rational, but less
rationalising
Continuous learning
48
Always assess your method,
your analysis and yourself for
bias

49. @RuthEllison 49
We are all human

50. @RuthEllison Image source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions_1954.JPG

51. Thank you
:)
Say hello @RuthEllison

52. @RuthEllison
Further reading
List of cognitive biases
You are not so smart: a celebration
of self delusion
9 Biases In Usability Testing
52