The Role of Eye Tracking in User Experience Research
(Download at: http://www.usercentric.com)
Webinar presented on behalf of the Human Factors and Ergonomics Society (HFES)
December 11, 2009
By Gavin Lew, Managing Director of User Centric
Back to 2009 Publications Index
Gavin Lew, Managing Director of User Centric, presented a Webinar on behalf of the Human Factors and Ergonomics Society (HFES) on December 11, 2009. During this Webinar, The Role of Eye Tracking in User Experience Research, Gavin described eye tracking measures, methods, and applications to user experience research. User Centric is well known for its eye tracking research through the widely-published works of both Gavin Lew and Associate Director, Aga Bojko, a leading expert in the field.
The HFES launched a series of Webinars in 2009; in 2010 there will be webinars on a bi-monthly basis.
Webinar summary:
The ability to track eye movements for scientific research has existed for generations. Recent technological advances in this area have made both data collection and analysis easy. Thus, what was once an extremely cumbersome and time consuming research endeavor, now is a very intriguing research technique—with the visual outputs sometimes being perceived as having more value than the data itself. With the growing number of eye tracking system installations in both academic and commercial research centers, a discussion of the role of eye tracking in user experience research is critical.
Download the Full Presentation
The presentation is available for free download in PDF format to the right (8 MB, Adobe Reader required).
About the Human Factors and Ergonomics Society
The Human Factors and Ergonomics Society was founded in 1957. Its mission is to promote the discovery and exchange of knowledge concerning the characteristics of human beings that are applicable to the design of systems and devices. The Society's 5,000+ members advocate systematic use of human factors knowledge to achieve compatibility in the design of interactive systems of people, machines, and environments to ensure their effectiveness, safety, and ease of performance. Through its 21 technical groups, 35 local chapters and 31 student chapters, the Society encourages education and training for those entering the human factors and ergonomics profession and for those who conceive, design, develop, manufacture, test, manage, and participate in systems.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Eye Tracking in User Experience Research - Webinar
1. The Role of Eye Tracking in
User Experience Research
HFES Webinar Series
Prepared by User Centric, Inc.
Gavin Lew
Managing Director
glew@usercentric.com
Presentation Delivered
11 December 2009
2. Outline
The Basics
Eye Tracking and UX Research
The Case for Quantitative Analysis
Ceilings and Floors
Method case study on packages
Q&A
2
4. Eye Tracking: The Basics
Basics of Eye Movements
Eye tracking is a
research technique
that captures eye
behavior in response
to a visual stimulus
“Eye-mind
hypothesis”: where
people look is where
they focus their
attention
Saccadic eye
movements (most Saccade (line)
common) consist of: Fixation (circle)
– Fixations
– Saccades
4
5. Eye Tracking: The Basics
Sample Eye Measures
Quantitative Measures Meaning
Informativeness of an area / user interest in the
# fixations on an area
area
Info clarity / info density / info processing
Fixation length
demands
# fixations before target
Time to 1st target fixation Layout effectiveness / search demands
Scanpath complexity
% users fixating on an area
Order of 1st fixation Prominence / perceived importance of an area
# visits to area
Cognitive processing demands / user mental
Pupil diameter
workload / emotion
5
6. Eye Tracking: The Basics
Eye Trackers
Eye tracker determines the position of one or both eyes multiple
times (30 – 1000+) per second
Commonly used eye trackers differ in physical form, setup
procedures, and tracking methodology:
6
7. Eye Tracking: The Basics
The Tobii Eye Tracker
Our eye tracker: Tobii 1750
– Integrated into a 17”
computer monitor
– No restraints, freedom of
head movement
– Binocular tracking
– Sampling rate 50 Hz
– Quick and automatic
calibration
7
8. Eye Tracking: The Basics
Lab Setup
Moderator’s station to view Face Tobii 1750 remote eye-tracking
eye gaze in real time and camera system integrated into a 17”
control eye tracking software monitor (set to 1024 x 768 px)
One-way
mirror
Moderator Participant
8
14. Eye Tracking: Eye Tracking and UX Research
Eye Tracking in UX Research
14
15. Eye Tracking: Eye Tracking and UX Research
How Can Eye Tracking Help Usability Testing?
TASK: Find a branch near you
Page with no
gaze. With
something hard
to find.
That’s
impossible.
15
16. Eye Tracking: Eye Tracking and UX Research
How Can Eye Tracking Help Usability Testing?
ET FINDINGS:
(1) Average # fixations before the target
link was found: 112
(2) The top right corner of the page
attracted initial & the most fixations.
Ah, quant data…
16
17. Eye Tracking: Eye Tracking and UX Research
How Can Eye Tracking Help Usability Testing?
TASK: Find a list of indoor climbing walls.
So… she only
considered the
first two results?
17
18. Eye Tracking: Eye Tracking and UX Research
How Can Eye Tracking Help Usability Testing?
ET FINDING: She looked at (considered) more
results than just the top 2.
18
19. Eye Tracking: Eye Tracking and UX Research
How Can Eye Tracking Help Usability Testing?
Typically in usability testing, we
collect:
– Behavioral measures (e.g.,
clicks, time)
– User self-report (e.g., RTA)
Eye tracking can:
– Support and illustrate UT
findings
– Help determine user
expectations
– Augment usability findings
by filling in the gaps
between
• Observable events
• User comments
19
20. Eye Tracking: Eye Tracking and UX Research
How Can Usability Testing Help Eye Tracking?
Different eye movement patterns
produced by the same person
looking at the same picture…
Why are they
all so different?
Yarbus, A. L. Eye Movements and Vision. Plenum. New York. 1967
20
21. Eye Tracking: Eye Tracking and UX Research
How Can Usability Testing Help Eye Tracking?
TASK: Estimate
people’s ages.
Different eye movement patterns
produced by the same person looking
at the same picture…
but in a different context!
TASK: Estimate the
family’s material
circumstances.
Yarbus, A. L. Eye Movements and Vision. Plenum. New York. 1967
21
22. Eye Tracking: Eye Tracking and UX Research
How Can Usability Testing Help Eye Tracking?
TASK: Find the museum hours.
22
23. Eye Tracking: Eye Tracking and UX Research
How Can Usability Testing Help Eye Tracking?
ET FINDINGS:
(1) All users looked at the image.
(2) Time spent looking at the image:
30% of all time spent on page.
Great!
Image got lots
of attention!
23
24. Eye Tracking: Eye Tracking and UX Research
How Can Usability Testing Help Eye Tracking?
Actual target link
So this is
bad…
24
25. Eye Tracking: Eye Tracking and UX Research
Usability Testing and Eye Tracking
Usability Eye Tracking
Provides context for…
Testing
reveals factors
reveals (mostly) that contribute
outcomes of Illustrates and helps understand to the outcomes
interaction
ANSWERS PROVIDES
PRACTICAL MORE
QUESTIONS DETAILED
ANALYSIS
25
26. Eye Tracking: Eye Tracking and UX Research
Eye Tracking In Isolation Yields Uncertainty
Eye tracking has limited
applicability when used in
isolation
– A fixation on a face may
indicate recognition, liking,
dislike, or confusion
– More fixations may indicate
interest or inefficient search
Instead combine with
attitudinal and behavioral
probes
– Learn the “why”
– Just knowing where people
look is often insufficient
26
44. Eye Tracking: Ceilings and Floors
What Did Participant #7 See?
Key words
Watched pig
Even followed the mouse!
In testing, we asked about the commercial…
– Love it
– I want the deal
So, you go and get it!
– Literally. Participant asked to open a browser and off they went
– They even entered the URL…
44
45. Eye Tracking: Ceilings and Floors
TV Commercial to Web “Handshake”
Empirical weblog data:
– 20% clicked the pig
45
46. Eye Tracking: Ceilings and Floors
Results
Attitude
– Viewers liked the ad
– Some even “loved it” and wanted to get the “deal”
Behavior
– In the study, 23% clicked (empirically, 20% click)
Note how Eye
Attention Tracking was not
used in isolation…
– 70% looked
• But, the pig spins around, so some glances will occur
– Setting a 150ms threshold (look at pig, read, etc.)
• 40% looked
• Relative to ad spaces on site
46
47. Eye Tracking: Ceilings and Floors
Ceilings and Floors
Not everything you design gets attention (there is a “ceiling”)
Designers and marketers tend to assume that if you build it, 100%
will see it
What would you say about these results?
We said:
– Not bad. Your ceiling was 40% and you are getting 20%.
– To improve click-thru, improve attention.
• If you do not attend, you will not click.
Eye Tracking is useful answer: Do user NOT CLICK because
– They looked, but the value proposition <> click (marketing).
– They never looked (design).
47
49. Eye Tracking: Bing vs. Google
Comparing Two Search Results Pages
Research objective:
– Compare the distribution of attention on equivalent areas of Bing
and Google
– Assess how much attention is captured by elements that are
unique to Bing
Participants (N=21) completed search tasks for each engine
– Two informational (e.g., "Learn about eating healthy")
– Two transactional (e.g., "Book a last minute vacation“)
49
51. Eye Tracking: Bing vs. Google
Results: Similarities
Google and Bing did not differ in
amount of attention on the organic
search results
– In each search, all participants
looked at the organic search
results, spending an average of 7
seconds in that area
Attention on the sponsored links
located above the organic results was
similarly high for both Bing and
Google
– Over 90% of participants looked in
that area during each search.
51
52. Eye Tracking: Bing vs. Google
Results: Differences
Sponsored links on the right attracted
more attention
– Bing (~42% of participants per
search) than they did on Google
(~25% of participants per search)
– Participants who fixated on these
links spent approximately 2.5
seconds looking at the area during
transactional searches and 2
seconds during informational
searches
52
54. Eye Tracking: Method
Method Case Study
Study objectives:
– Determine how the client’s packaging
compares to competitors’ packaging in
Marketing
terms of noticeability and visual
engagement / interest
– Assess the findability and clarity of the
information on the client’s package as
compared to the information on the Usability
competitors’ packages
Wearable eye trackers vs. remote eye trackers
54
55. Eye Tracking: Method
A Walk Down The Aisle…
Macro-level:
– Participants saw pictures of
aisles
– Five picture frames per
store (2 s per frame)
– Each showed the entire
prepaid phones display
– However, the display was in
a different position in each
frame to simulate the
changing view of a moving
customer (in a “freeze
frame” way) and minimize
the effect of product position
in the picture
55
56. Eye Tracking: Method
Product View
Micro-level: Front Back
– Participants saw pictures of
individual packages
– Given tasks to answer to find
information on the package
itself
– Participants were able to
“flip” package around by
clicking
Q3: Can you
Q2: What is
The answer is: browse the The answer is:
the brand of
______ Web on this ______
this phone?
phone?
56
57. Eye Tracking: Method
Actual Package Experience
Constructed shelves
Participants were different from
the macro- and micro-level
study
– Far view
– Up close
– Manipulation
– Selection decision
– Discussion
57
58. Eye Tracking: Method
Measures
Memory
– Free recall of products and brands
– Recognition exercises of products and brands
Preference
– First impressions from afar
– Likes and dislikes
Performance tasks
– Accuracy of finding answers to tasks
– Efficiency in time to find answers (excluded incorrect answers)
Eye tracking (computed based on package area)
– % who looked (noticeability)
– # of fixations (visual engagement)
58
60. Q &A
Final Takeaways
Eye Tracking should be combined with other UX techniques
– Avoid using ET in isolation
Context matters
– One can change a heatmap with just four words!
When looking at a heatmap, ask:
– What was the task or context?
– What is the heatmap showing?
• % who fixated, gaze duration
• # of fixations, etc.
• Each measure reveals a different story…
With so much data, where is the quantitative analysis?
60
61. Q &A
Q & A?
Questions?
Additional references:
– Me: glew@usercentric.com
– Peer-reviewed journal articles, presentations,
white papers on www.usercentric.com
– ET case study of package labels by Aga Bojko
in Tullis and Albert’s book, Measuring The
User Experience
– ET global case study in Bob Schumacher’s
book, The Handbook of Global Research
• www.globaluserresearch.com
• www.elsevierdirect.com
61