The 7 Things I Know About Cyber Security After 25 Years | April 2024
Et sync uxpa2013
1. Putting the Pieces Together:
Synchronization, Analysis, and Presentation of Eye
Tracking and Complimentary Data Streams
Wilkey Wong
Tobii Technology
2. External devices and applications
Basic issues in timing
General synchronization concept
Approaches to synchronization
Data analysis/presentation
Questions
2Copyright Tobii Technology AB
Synchronization of eye trackers with external devices
4. Why would you want to do this?
1. Triangulate
2. Disambiguate
3. Richer interpretation
Attention, processing,
engagement…
5. For monitoring online/on task state
AOI triggering and gaze contingent experiments
Artifact rejection
Selection of the data of interest for analysis
Synchronization with user actions/reactions
5Copyright Tobii Technology AB
1. EXTERNAL DEVICES
APPLICATIONS
What are the applications?
8. 8Copyright Tobii Technology AB
Latency – Inside the Eye Tracker
When is latency important?
When synchronizing eye tracking data with other data (latency) during a recording.
Any applications where gaze data is needed to trigger responses (Gaze contingency)
10 – 40 ms
2. TIMING
Eye Trackers
9. Communication Chain – Up to Stimulus Presentation
9Copyright Tobii Technology AB
The more you are interested in short events the more you need to care about timing
2. TIMING
System
12. 12Copyright Tobii Technology AB
3. APPROACHES TO SYNCHRONIZE
TOTAL EVENT LATENCY CHAIN
Device
Event sender
latency
Software
Hardware
Transmission Latency
Software
Event receiver
latency
Hardware
Event
Total Latency – END-TO-END
Event sender Event receiver
Device
Each system should be characterized at the component level to understand the system
13. 13Copyright Tobii Technology AB
3. APPROACHES TO SYNCHRONIZE
COMMON SYNCHRONIZATION EVENTS
Trigger
Event
External
device
14. 14Copyright Tobii Technology AB
3. APPROACHES TO SYNCHRONIZE
SYNCHRONIZATION EVENT STRATEGY
One common event
Stimulus
Event
Evenly spaced events
Stimulus
Event
Events based on Stimulus onset
Stimulus
Event
Manually triggered events
Stimulus Event
Accuracy and drift
16. 16Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
0/5V low latency
17. 17Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
LAN
Signaling flexibility
18. 18Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
Single master clock
19. 19Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
User Events
(clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
Direct user input
20. 20Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
Direct measurement
21. 21Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
EEG
Eye Tracking
Rich possibilities
22. 22Copyright Tobii Technology AB
4. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
Extremely simple
23. 23Copyright Tobii Technology AB
2. HOW TO SYNCHRONIZE
TYPES OF EVENT TRIGGERS
TTL pulses (Parallel/Serial Port)
TCP/IP
NTP
Events (clicks, keystrokes)
Light sensors
Physiological events (blinks)
Button presses
Video
EEG
Effort-intensive
24. Types of event triggers - Summary
1. Choose base on flexibility, accuracy, action
2. Options range from simple to complex and
time intensive
3. Consider data integration needs
26. 5. SYNCHRONIZATION EXAMPLES
OVERVIEW
Video TTL & TCP/IP Light Sensor
Live screen recording
Devices that support
external video recording
Offline synchronization
Stimulus presentation
software records Eye
Tracking data
Stimulus presentation
software and Tobii Studio
records Eye Tracking data
StimTracker for TX300
Copyright Tobii Technology AB 26
28. 28Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
LIVE SCREEN RECORDING
NON-CRITICAL TIMING
Tobii Studio :
Records Eye Tracking data
External device software:
Records the external data.
TX300 Eye Tracking data
LAN
VGA/DVI
Tobii Studio
&
External device software
Computer screen:
Shows live both recordings.
VGA/DVI
External Device
Computer screenEye Tracker
29. EXTERNAL VIDEO RECORDING
5. SYNCHRONIZATION EXAMPLES
29Copyright Tobii Technology AB
Live video
Eye Gaze data
Stimulus presentation
Stimulus presentation +
gaze
Tobii Studio:
Presents the stimuli
Records Eye Tracking data
Has the Live Viewer feature enabled
External device software:
Records the physiological data synchronized
with the video
Captures a video in real time from Tobii Studio
Live Viewer feature
Eye Tracker Tobii Studio computer External device computer
Video
Capture
device
External
device
30. 30Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
VIDEO – EXAMPLE BIOPAC
Example: ET video recording using Acqknowledge software and Tobii Studio Live viewer
TX300 Eye Tracking data
Capture
device
LAN
VGA/DVI
VGA/DVI
(Live Viewer)
Eye Tracking Video
Tobii Studio Computer Recording computer with
Acqknowledge 4
Biopac System:
Amplifier MP150 +
Isolation Module STP100C +
GSR module GSR100C
32. 32Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
TTL: E-PRIME / PRESENTATION / MATLAB / OTHER
Stimulus Software presentation:
Collects eye tracking data from ET
Timestamps in the ET data the stimulus onset
Sends through the parallel/serial port TTL signals
at the stimulus onset
E-Prime / Presentation:
Tobii Extensions needed
Matlab / Other:
Tobii SDK
Stimulus presentation software records Eye Tracking data
TET packages
External
device
DVI / VGA
LAN
Parallel/Serial Port
External Device Computer
Stimulus Presentation Computer
33. 33Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
E-PRIME EXTENSIONS FOR TOBII
E- Prime controls Eye Tracker with TET packages
E- Prime send TTL pulses to the external device
E-Prime collects raw data from the Eye Tracker
• Fixation Filters (Fixation / Saccades)
• Visualizations (Heat maps / Gaze plots)
• AOI
• Necessary a third software to analyze Eye Tracking
data
Example: TX300 and Brain Products EEG recording using E-Prime and TET Packages
DVI
LAN
Parallel Cable
BrainVision Recorder
Computer
USB Adapter EEG BrainAmp
34. 34Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
E-PRIME EXTENSIONS FOR TOBII
Example: TX300 and Brain Products EEG recording using E-Prime and TET Packages
DVI
LAN
Parallel Cable
BrainVision Recorder
Computer
USB Adapter EEG BrainAmp
E-Prime
Events
37. 37Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
EXTERNAL TRIGGERING/SIGNALING
Two data types:
- EventMarkerValue: 0=off, 1 = on
- StudioEvents: EventMarkerOn, EventMarkerOff
Studio
38. 38Copyright Tobii Technology AB
5. SYNCHRONIZATION EXAMPLES
EXTERNAL TRIGGERING/SIGNALING
Light sensor
TX30
0
StimTracker
Coaxial
Parallel Cable DB-25
DVI
LAN
Tobii Studio Computer
Example: TX300 and ANT EEG Recording using StimTracker for TX300
ASA-Lab Computer
ANT EEG Amplifier
Tobii Studio Software ASA-Lab Software
StimTracker
Events
42. 42Copyright Tobii Technology AB
6. DATA ANALYSIS
SEPARATE ANALYSIS BY DATASTREAM
Analyze each data stream independently
External Device Data Eye Tracking Data
. . .
Eye Tracking
Results
External Device
Results
Stimulus
Stimulus
44. 44Copyright Tobii Technology AB
6. DATA ANALYSIS
COMBINED ANALYSIS
Merge data and analyze it together
2. Match sampling rates!!
External Device Data e.g.:
Sampling rate: 250 Hz
250 samples / sec
Eye Tracking Data e.g.:
Sampling rate: 120 Hz
120 samples / sec
After interpolate data:
250 samples / Sec
1. Export raw data
3. Merge data
Eye Tracking &
External Device
Results
Interpolate
. . .
Analysis or display
45. 45Copyright Tobii Technology AB
6. DATA ANALYSIS
COMBINED ANALYSIS
Merge data and analyze it together
2. Match sampling rates!!
1. Export raw data
3. Merge data
Eye Tracking &
External Device
Results
Interpolate
. . .
EEG data
Eye Tracking data
47. 47Copyright Tobii Technology AB
6. DATA ANALYSIS
JOINT ANALYSIS: EXAMPLES OF USE
Eye Tracking triggers - EEG, ECG, GSR,…
EEG, ECG, GSR,…
Eye Tracking
Analyze data based on the eye tracking data validity
48. 48Copyright Tobii Technology AB
6. DATA ANALYSIS
JOINT ANALYSIS: EXAMPLES OF USE
Analyze data based on electrophysiological signals
Physiological measure triggers - EEG, ECG, GSR,…
Eye Tracking
EEG, ECG, GSR,…
49. 49Copyright Tobii Technology AB
6. DATA ANALYSIS
JOINT ANALYSIS: EXAMPLES OF USE
Artifact removal – Blinks:
Localize blinks in EEG data
EEG
Eye Tracking
50. 50Copyright Tobii Technology AB
6. DATA ANALYSIS
JOINT ANALYSIS: EXAMPLES OF USE
Areas of Interest
Analyze the External data based on the AOIs of the Eye Tracking data
BALL
BOX
External Data
AOI Ball
AOI Box
Today we are here to talk about how to synchronize Tobii eye trackers with external devices and what are the current possibilities to do it.
First I would give you a brief introduction of what devices are commonly synchronize with Eye Trackers and what are the applications.Then I will talk about the importance of timing and what things are important to bear in mind when synchronizing devices. Then I’m going to talk about the general concept of synchronization We will focus on synchronization possibilities and go through the most important setups, explaining the characteristics of each one and giving examples. Finally, I’ll talk a bit about the analysis data, the existing options to analyze the data collected and some examples of use. then questions.
What kind of devices are we referring to, When we talk about synchronizing an eye tracker with an external device?Well, if we are conducting a study with eye tracking, it’s because we are interested somehow in understanding better the cognitive processes that are behind some actions or decisions that are being testing. Sometimes it’s useful to combine the information that the eye tracking provides with other sources of information.For example, many times it’s useful to collect ET data together with EEG data, so that we don’t have only the reactions that takes place in the brain, but also we can know where the participant was focusing the attention during these reactions. The same as with EEG, we can combine ET with other electrophysiological measurements, such as Electrodermal Activity, Electrocardiography, Electromyography , etc. We may also be interested in synchronizing the ET data with some external camera source, for example a camera that is recording the scene or the participant’s face.Or we want to know how the participant was moving while paying attention to some stimulus, so we would want to synchronize the ET data with some motion capture device.And like these examples, there are other devices that for some reason we are interesting in synchronizing with Tobii Eye trackers.
And if we talk briefly about applications, what are the main applications where combining ET data with other data is useful??Here we have some of the main applications, For monitoring, We can use the ET data to make sure that the participant is looking at the stimulus at the moment of recording other source of data. One example would be when recording EEG on children, they get distracted really easily and the ET data could be use to select just the stimulus where the child was looking, making the results more robust. ET data can be also used to control the stimulus presentation of an experiment using gaze contingency, for example waiting for the participant to fixate in one Area on Interest for certain time before continuing with the next stimulus. Another application is using the ET data to detect blinks and saccades in order to remove artifacts from the EEG data. The eye blinks creates big artifacts in the EEG data, and with the ET data, it can be known which artifacts are due to eye blinks, making things easier to select valid data from the EEG. The combination of two synchronized sources of data can be used to analyze one of the data sources depending on the selected data in the other source. One example would be to analyze the ET data only during the period of time when the participant was below certain level of heart rate. Finally, depending on the application and outcome, we need to think if we want a separate analysis of the data, so even though we want to synchronize the data, we don’t need to analyze it together, we just need to complement it, or, the other possibility would be to merge the data and analyze it in the same software.
Let’s introduce now some general concepts about timing and latency and what things are important to bear in mind when trying to synchronize two devices.
First I’m going to introduce general concepts about computer timing. This is important because usually are computers the ones that are responsible for timestamp and communicate between devices. When we look at the environment of the computer and the different components of the computer there are several challenges when it comes to obtain accurate timing. One is that the processor speed will change depending on different aspects, like the temperature of the processor and since the clocks are related to the processor speed you will also have a change in the clock rate. This means that different hardware systems may have different drifts.The other challenge is that you have different applications trying to access this clock which may cause delays. The other issue is that nowadays computers have more than one processor and normally each of these processors have a hardware clock and programs may access each one of these clocks independently which means that if these two clocks have a certain drift from each other then the timing would be inaccurate.Being said this, the biggest challenge in terms of computer timing is that operations do not necessarily perform in a predictable time lapse and makes it hard to produce or register an event at any precise instant. This is true for different operating systems such as Windows, Mac OS or Linux.
Another variable that is relevant when we talk about synchronization of Eye Trackers and other devices, is the ET latency. The total latency of an eye tracker goes from the moment the image is acquired by the sensor until the gaze data is delivered to an application. Most of these time is spent processing the image data and transforming it into gaze data. This latency is dependent on the ET, being up to 10 ms for the TX300 Tobii ET and up to 67 ms in the 30 Hz X2 Tobii ET.It is important to highlight that the gaze data is not affected by the Eye tracker latency, because the timestamp always will reflect the mid-exposure of the image of the eye on the sensor.Latency is mostly important when you want to synchronize live ET data with other device. If you try to merge different sources of data during the recording, it’s likely that each data has a different latency and arrives to the computer at different times. So if you try just to merge the data as it comes from the devices, this will end up creating an inaccurate synchronization of data, as the latencies of the different devices are not totally known. That is why the most common scenario to synchronize data, as we will see in a moment, is to record both data sources independently and merge it after the recording has finished.Another application where latency is important is when we are using gaze contingency paradigms, that is when we want to access the gaze data in order to trigger events during the stimulus presentation.
For example, if we talk about the communication delays that can happen when a stimulus presentation computer wants to access to the screen to present a new stimulus. The stimulus presentation software needs to prepare the image for the presentation and send the image to the graphics card. The graphics card will render the image and transmit it to the screen of the eye tracker and then the actual time when the image will be displayed will depend on the refresh rate and the response time of the screen. All this process takes time and the real challenge for a stimulus presentation software is to know not only the timestamp when the image was sent to be displayed, but to know the timestamp when the image was actually displayed on the screen, which is the timestamp that we want to record to be accurate. This is just a real example to understand better the challenge that exist when communicating between devices, but the general conclusion about this brief introduction of timing is that the more you are interested in short events, the more you need to care about timing, and the more you need to know your systems and how they work and perform operations.*************Multitask environment - Different hardware may have different drifts – it’s hard to produce or register an event at any precise instantWhen using separate devices simultaneously, there will be internal and external delaysInternal: sending the triggersExternal: time lag for sending and receiving the triggerThe delays are not constant, but they vary between different ..HardwareStimuli presentation softwareMonitors
Taking into account the general concepts that we have introduced about timing, now we are going to talk about how to synchronize Eye tracking data with other devices.
let’s talk a bit more about timing, this time in relation to the latency that we might be introducing since we say to the Event sender that we want to create and event until the event is received and timestamped in the device data.There are three parts that can increase latency, the first is the event sender. If the event sender is software running in a computer, we have already learnt that computer may introduce a delay in the execution of a command. The event sender could also be a hardware device designed specifically for the purpose of sending events, which would reduced the event sender latency.Then we have the transmission latency, which normally can be ignored because we can consider transmission through cables almost instantaneous. And we also have to take into account the delay that can be introduced in the event receiver, since it receives the event until it actually includes it in the data with a certain timestamp. Again, the event receiver can be the recording software running on a computer or the actual device that is recording the data.This doesn’t mean that we are always introducing a big delay when sending events, the most common setups are already tested and are used because they have a really good latency numbers, but it’s important to keep in mind that the use of different strategies can lead to undesirable results that we won’t notice unless we study them.
The recommendation when we want to synchronize two devices is that each recording software run in a different computer for performance and efficiency reasonsThis means that each computer will have its own clock, so according with the timing concepts we talked about before, computer clock will have slightly different operation and systems might drift with time. Also, it has to be taken into account the latency that each device may introduce since the data is recorded until it is available to store and use it by the recording software. For that reason, to synchronize two sources of data, the most accurate way is create at least one common event on both data recordings that allows to align the Eye tracking data and the external device data. Normally, this alignment is done after the recording. This events should be sent at the same time to both devices and should be included in the data at the time the event was created and sent.
There are different strategies when synchronizing two sources of data. The simplest one would be to send one common event at the beginning of the recording and use this as the basis for synchronization. If we want to avoid system drifts and improve accuracy in synchronization we can send more than one event during the recording. One approach is to send equally spaced, repeated events during the recording to synchronize the data afterwards. Another possibility is to send events manually with keystrokes or response boxes. Finally, the most common approach is to send events at the time of stimulus onset. This means that each time a stimulus is presented on the screen, an event will be sent to both the eye tracking data and the external device data.
Let’s talk a bit about the types of events that can be used to synchronize eye tracking data and external devices.
The most common way to send events to different devices to synchronize them is using TTL technology, which stands for Transistor-Transistor Logic Pulses. It´s a digital signal that has two discrete states, 0V or 5V and the changes between these two those states are used to mark events in the devices. As it consists in an basic electrical signal, the delay in the signal transmission is almost zero and it is normally connected directly to the hardware part to the devices, so there is no computer involved causing delays. These reasons make this technology very convenient synchronize when timing is critical . It’s an old technology but still can be found in almost every computer in the parallel and serial ports.
Events can also be created via TCP/IP Protocol, the main difference between this option and the TTL synchronization is that the events are sent between computers through a LAN cable and this allows to send a lot of different types of events.
The NTP protocol is based on the idea of having one clock that Is used as the master and synchronize all the other computer or devices clocks based on this one.
We can create events manually by pressing keystrokes, using the mouse, or with response boxes for example.
Another option is to use light sensors attached to the stimulus presentation screen and trigger events based on the stimulus presented on the screen.
We can synchronize data with common physiological events for example, blinking, which can be easily detected in EEG and eye tracking data.
In some cases, we might want also to synchronize eye tracking videos with other sources of data.
We are going to focus now in the main possibilities that already exist to synchronize Tobii Eye Trackers with external devices
Speciffically, I will talk about three main different ways to synchronize devices. I will explain the setups needed and the main characteristics of each one and I will also give examples for each one of the setups for you to understand better what are the outcomes and results of each setup.First, I will introduce the video synchronization possibilities, with a simple synchronization based on live screen recording and with another a bit more advanced setup suitable for external devices that support external video recording. After, I will introduce some set ups based on TTL and some of them include also TCP/IP protocols for communicating between softwares. We well go through two main setups, one of them using TET packages to communicate with the ET and the other one using ClearViewAPI to communicate and record in studio. This options always communicate with external devices using TTL pulses, parallel or serial port.The last way of synchronizing Tobii eye trackers with external devices that I will present will be using a light sensor to generate events and send them at the same time to Tobii Studio as well as to the external device. This option is possible when we include in our setup a third device called StimTracker and only using Tobii TX300 Eye Tracker.
This is the most basic setup that we can have to record together eye tracking data and data coming from other device. In this case we just need one computer which runs Tobii Studio as well the software needed to record the external device data. Tobii Studio will record the Eye tracking data, and using the live viewer feature, a video with the information of where the person is looking can be shown in real time. This video running in the Tobii Studio application can be combined in the same screen with the live video of the data recorded from the other device. This setup is useful when we don’t care much about timing, because as we have explained during this presentation, showing a live video of the data that is being recorded will introduce certain latency, as well as recording both data in the same computer may create computer timing problems. Nevertheless, it’s a very easy and useful setup when we don’t need too much precision in synchronizing events in our experiment. On example can be synchronizing some kind of wireless device to record electrophysiological data.
There are software that support external video recording. Which means that a video can be synchronized and recorded together with the device data. In this case we have a two computer setup. Tobii Studio records the eye tracking data and has the live viewing feature enabled. In order to record the video coming from the TS computer, the external device computer must have a video capture device. This computer will record the data from the external device as well as synchronizing the live view video coming from the TS computer.It is also possible to run an external device from the TS computer but now you’d have to consider additional internal latencies and lags.
In this example, our external device is a Biopac MP150 amplifier, recording the galvanic skin response and electrocardiogram from the participant while the gaze is also being recording. The Biopac recording software Acqknowledge, allows synchronization of a live video source with electrophysiological data. To synchronize the video gaze live with the set up the systems as I just explained, having a computer running tobii studio recording the eye tracking data and a computer running acknowledge recording GSR and ECG data and synchronizing it live with the live viewer video coming from Tobii Studio.
Now, we are going to talk about the second method of synchronization, which is the TTL synchronizationTS cannot send TTL pulses so for this section, we will be referring to other software for stimulus presentation.
The first option TTL is stimpressf that can record ET data, can communicate with ET via TET packEprime, presentation, matlabComp runs stimpres, collecting data, and timestamp. 2 comps are synched so onset/offset are accurately stamped. TTL sent to ext device stim on/off.
Here is an example of output from this setup
I now will talk about the last option for synchronizing tobii Eye Trackers with external devices that I will present during this webinar. This option uses a third device called StimTracker for TX300 designed specifically to send synchronized events to Tobii Eye trackers and other external devices. The stim tracker will send events based on the status of a light sensor attached to the stimulus presentation screen.
First, I will give a brief introduction about the working concept and the advantages of synchronizing using optionThe StimTracker is a device manufactured by Cedrus that is capable of sending very accurate events both to the Tobii Eye tracker TX300 and to most of the leading EEG and biometric data collection systems, as well as a PC parallel port.The StimTracker will send a synchronized event based on the status of a light sensor that must be attached to one corner of the stimulus presentation Screen. As soon as the light sensor detects a color change in the area where it is attached, it will change its status and the stimtracker will send a simultanous event to both devices. The light sensor can have two values: a value of 0 or 0Vwhen it detects a black color on the screen and a value of 1 or 5V when it detects a with color on the screen. Changing the color from black to white and white to black each time a new stimulus is displayed on the screen, will allow to have a very accurate onset stimulus Event.The main advantage of using this system for synchronizing, Is that it is independent of delays in operating systems or graphic cards, as the light sensor will change its status only when detects a pixel change. This gives a really good timing accuracy recommended in studies that require high temporal resolution. Thestimtracker for TX300 is compatible with Tobii Eye Tracker TX300 and most of the EEG and biometric data collector systems, as well as with the PC parallel’s port of computers.
Tobii Studio will integrate the binary triggered events from the light sensor of the StimTracker TX300 in the tobii Studio recording and the eye tracking data. These events will be accessible in two locations in tobii Studio: The replay tab as events on the timeline as well as in the Events list and data available for export in the Data Export Tab.In the Replay tab, the events appearing in the timeline and in the Event list indicate when the signal from the external device changes. This will generate two Studio Events: EventMarkerOn and EventMarkerOff. The event markers can be used to automatically generate segments that can be used to create scenes for data analysis and visualizationsThe second data type that the EventMarkerValue, which provides the signal registered by the eye tracker. The registered signal will be continuous, each sample received from the eye tracker will be associated with a value of the current status of the sensor.Besides from Tobii Studio, when using the Tobii SDK 3.0 to control Tobii TX300 hardware features, it is possible to obtain the EventMarkerValue variable from the Sync Port of the TX300, where the StimTracker is connected, which means that any stimulus presentation software built on the SDK 3.0 is able to record the light sensor values coming from the StimTracker for TX300.
Once we have the data from the eye tracker and the external device recorded, it has to be decided if the we want to analyze the data separately or use the events to merge both data sources and analyze them together.
Let’s talk first about the separate analysis
Sometimes, we want to combine eye tracking with other data but we are not interested in analyzing both signals together, we want to obtain results and conclusions from each source of data separately. However, if we want to be accurate, it is useful to synchronize the Eye tracking data and the external device data by sending events at the onset of each stimulus. Then, both results will be consistent with the stimulus timingIn the case of the Eye tracking data, once it is recorded, it will be analyzed with Tobii Studio or other analysis software and we will obtain Eye tracking results. In the same way, the external device data will be analyzed with the analysis software selected and we will obtain the results just taking into account the information obtained with this source of data. Once we have both results, they can be studied together and complemented to reach more conclusions.
We need to have both data sets together in one file in order to start analyzing the data.The first step will be to export the raw Eye tracking data. If the recording software was tobii Studio, it can be included in the exported data variables as Areas of Interest, fixations or Event markers as mouse clicks or keystrokes. The dataset must include one variable that corresponds with the events created to synchronize.If the analysis software is not the same as the one used to record the external device data, the recorded data must be exported including one variable that also corresponds with the events create to synchronize with the eye tracking data.One concept that it hasn’t been introduced yet, it the sampling rate of the recordings. If we want to merge two data sources, we need to have the same number of samples in each dataset. Normally, the sampling rate of the Eye tracking data will not be the same as the sampling rate of the other external device. For example, if we recorded an eye tracking study with the Tobii Eye tracker T120 using a sampling rate of 120Hz, means that we collected 120 gaze data points per seconds. On the other hand, if we have recorded synchronized with the eye tracking data the ECG of the participant with a sampling rate of 250Hz, means that we collected 250 data points per second. We cant merge directly both data sets, as one them, in this case the ECG, will have more points and the points wont correspond in time. Here is where the concept of interpolation is introduced, the shortest data set, in this case, the eye tracking data, will be modified by matching the points per seconds of the other dataset. Once the data has been interpolated, both data set will have the same sampling rate. In this example, the eye tracking data will have 250 points per second. Then, both the eye tracking data and the external device data can be merge using as reference points the events created, and then analyze with the analysis software chosen.At the end, the results will reflect the combination of the eye tracking data and the external device data.
Here we can see an example of a merged file containing a recording of EEG synchronized with Eye tracking data.
In the case of biometric data, like EEG, ECG or GSR, we are interested only in analyzing the data if the participant was looking at the stimulus shown. To analyze only valid biometric data, the eye tracking data can be used to select the parts of the biometric recording where the stimulus caused a reaction because the participant was looking at it.
We may also be interested in the opposite, analyze Eye tracking data based on the levels of the biometric data recorded.For example, analyze eye tracking data only when the signal of the biometric data recorded is above certain level.
In EEG, blinks cause artifacts, easily detected in ET, ET can be used to remove artifacts
The last example of use is to analyze the external data based on certain areas of interest created in the Eye tracking data.The picture on the right has two areas of interest, the ball and the box. If we create a variable with two values, a value of 0 when the participant was not looking at the area of interest, and a value of 1 when the participant was looking at the area of interest, we can combine this information and analyze the external data For example analyze first the external data when the participant was looking at the ball, and separately when the participant was looking to the box.
