Two hours in Hollywood: A manually annotated ground truth data set of eye movements during movie clip watching

In this short article we present our manual annotation of the eye movement events in a subset of the large-scale eye tracking data set Hollywood2. Our labels include fixations, saccades, and smooth pursuits, as well as a noise event type (the latter representing either blinks, loss of tracking, or physically implausible signals). In order to achieve more consistent annotations, the gaze samples were labelled by a novice rater based on rudimentary algorithmic suggestions, and subsequently corrected by an expert rater. Overall, we annotated eye movement events in the recordings corresponding to 50 randomly selected test set clips and 6 training set clips from Hollywood2, which were viewed by 16 observers and amount to a total of approximately 130 minutes of gaze data. In these labels, 62.4% of the samples were attributed to fixations, 9.1% – to saccades, and, notably, 24.2% – to pursuit (the remainder marked as noise). After evaluation of 15 published eye movement classification algorithms on our newly collected annotated data set, we found that the most recent algorithms perform very well on average, and even reach human-level labelling quality for fixations and saccades, but all have a much larger room for improvement when it comes to smooth pursuit classification. The data set is made available at https://gin.g- node.org/ioannis.agtzidis/hollywood2_em

Performance Validity Assessment Of Bona Fide And Malingered Traumatic Brain Injury Using Novel Eye-Tracking Systems

Purposeful presentation of neurocognitive impairment (i.e., dissimulation) in assessment of brain injury is a primary pitfall to accurate psychological assessment, especially among individuals seeking compensation. Current methods used to evaluate effort test failure (EFT; Webb et al., 2012) and dissimulation in brain injury assessment has advanced over the past few decades, but remains unacceptably inaccurate. In diagnostic decision-making, current methods identify obvious cases of purposefully poor performance, but they are considerably less accurate in subtle cases typically seen clinically; more important, they are vulnerable to coaching. Oculomotor behavior during visual tasks may be a promising avenue in the assessment of performance validity. Oculomotor patterns observed after brain injury have been well documented, and patterns characteristic of normal decision-making have been studied in healthy adults, but findings from these endeavors have not been applied to performance validity assessment. Accordingly, this study evaluated contributions of oculomotor patterns to detection of purposeful poor performance using state-of-the-science eye-tracking equipment by studying the predictive ability of a gold-standard performance validity test: The Test of Memory Malingering (TOMM). The study examined 39 adults with moderate to severe traumatic brain injury (TBI), 42 healthy adults coached to simulate memory impairment (SIM), and 50 healthy adults providing full effort (HC). The results supported the main hypothesis: One index derived using oculomotor patterns of performance provided a reliable increase to the predicative accuracy of the TOMM in differentiating bona fide TBI from simulated TBI. Numerous other oculomotor indexes showed promise, both in their relationships to key cognitive constructs and in their ability to differentiate dissimulation from healthy adults and bona fide TBI. The predicative ability of these measures was insignificant, however, due to an underpowered sample size and violations of the assumptions of pivotal statistical models. As such, future research is needed to replicate these findings and should strive to increase sample sizes to more accurately assess those visual patterns that showed predictive potential

Oculomotor Behavior as a Biomarker for Differentiating Pediatric Patients With Mild Traumatic Brain Injury and Age Matched Controls.

IMPORTANCE: Children have the highest incidence of mild traumatic brain injury (mTBI) in the United States. However, mTBI, specifically pediatric patients with mTBI, are notoriously difficult to detect, and with a reliance on traditional, subjective measurements of eye movements, the subtle but key oculomotor deficits are often missed. OBJECTIVE: The purpose of this project is to determine if the combined measurement of saccades, smooth pursuit, fixations and reaction time represent a biomarker for differentiating pediatric patients with mild traumatic brain injury compared to age matched controls. DESIGN: This study used cross-sectional design. Each participant took part in a suite of tests collectively labeled the "Brain Health EyeQ" to measure saccades, smooth pursuit, fixations and reaction time. PARTICIPANTS: The present study recruited 231 participants - 91 clinically diagnosed with a single incident mTBI in the last 2 days as assessed by both the Glasgow Coma Scale (GCS) and Graded Symptoms Checklist (GSC), and 140 age and gender-matched controls (n = 165 male, n = 66 female, M age = 14.20, SD = 2.78). RESULTS: One-way univariate analyses of variance examined the differences in performance on the tests between participants with mTBI and controls. ROC curve analysis examined the sensitivity and specificity of the tests. Results indicated that together, the "Brain Health EyeQ" tests were successfully able to identify participants with mTBI 75.3% of the time, providing further validation to a growing body of literature supporting the use of eye tracking technology for mTBI identification and diagnosis

Considerations for the Use of Remote Gaze Tracking to Assess Behavior in Flight Simulators

Complex user interfaces (such as those found in an aircraft cockpit) may be designed from first principles, but inevitably must be evaluated with real users. User gaze data can provide valuable information that can help to interpret other actions that change the state of the system. However, care must be taken to ensure that any conclusions drawn from gaze data are well supported. Through a combination of empirical and simulated data, we identify several considerations and potential pitfalls when measuring gaze behavior in high-fidelity simulators. We show that physical layout, behavioral differences, and noise levels can all substantially alter the quality of fit for algorithms that segment gaze measurements into individual fixations. We provide guidelines to help investigators ensure that conclusions drawn from gaze tracking data are not artifactual consequences of data quality or analysis techniques

MAD saccade: statistically robust saccade threshold estimation via the median absolute deviation

Saccade detection is a critical step in the analysis of gaze data. A common method for saccade detection is to use a simple threshold for velocity or acceleration values, which can be estimated from the data using the mean and standard deviation. However, this method has the downside of being influenced by the very signal it is trying to detect, the outlying velocities or accelerations that occur during saccades. We propose instead to use the median absolute deviation (MAD), a robust estimator of dispersion that is not influenced by outliers. We modify an algorithm proposed by Nyström and colleagues, and quantify saccade detection performance in both simulated and human data. Our modified algorithm shows a significant and marked improvement in saccade detection - showing both more true positives and less false negatives – especially under higher noise levels. We conclude that robust estimators can be widely adopted in other common, automatic gaze classification algorithms due to their ease of implementation

GraFIX: a semiautomatic approach for parsing low- and high-quality eye-tracking data

Fixation durations (FD) have been used widely as a measurement of information processing and attention. However, issues like data quality can seriously influence the accuracy of the fixation detection methods and, thus, affect the validity of our results (Holmqvist, Nyström, & Mulvey, 2012). This is crucial when studying special populations such as infants, where common issues with testing (e.g., high degree of movement, unreliable eye detection, low spatial precision) result in highly variable data quality and render existing FD detection approaches highly time consuming (hand-coding) or imprecise (automatic detection). To address this problem, we present GraFIX, a novel semiautomatic method consisting of a two-step process in which eye-tracking data is initially parsed by using velocity-based algorithms whose input parameters are adapted by the user and then manipulated using the graphical interface, allowing accurate and rapid adjustments of the algorithms’ outcome. The present algorithms (1) smooth the raw data, (2) interpolate missing data points, and (3) apply a number of criteria to automatically evaluate and remove artifactual fixations. The input parameters (e.g., velocity threshold, interpolation latency) can be easily manually adapted to fit each participant. Furthermore, the present application includes visualization tools that facilitate the manual coding of fixations. We assessed this method by performing an intercoder reliability analysis in two groups of infants presenting low- and high-quality data and compared it with previous methods. Results revealed that our two-step approach with adaptable FD detection criteria gives rise to more reliable and stable measures in low- and high-quality data

Investigation of vision strategies used in a dynamic visual acuity task

Purpose: Dynamic visual acuity (DVA), the ability to resolve fine details of a moving target, requires spatial resolution and accurate oculomotor control. Individuals who engage in activities in highly dynamic visual environments are thought to have superior dynamic visual acuity and utilize different gaze behaviours (fixations, smooth pursuits, and saccades). This study was designed to test the hypothesis that athletes and video game players (VGPs) have superior DVA to controls. Furthermore, the study was designed to investigate why DVA may be different between groups. Methods: A pre-registered, cross-sectional study examined static visual acuity (SVA), DVA, smooth pursuit gains, and gaze behaviours (fixations, smooth pursuits, and saccades) in 46 emmetropic participants (15 athletes, 11 VGPs, and 20 controls). Athletes were members of varsity teams (or equivalent) who played dynamic sports (such as hockey, soccer, and baseball) for more than 1 year with a current participation of more than 6 hours per week. VGPs played action video games four times per week for a minimum of one hour per day. Controls did not play sports or video games. SVA (LogMAR) was tested with an Early Treatment Diabetic Retinopathy Study (ETDRS) chart. DVA (LogMAR; mov&, V&mp Vision Suite) was tested with Tumbling E optotypes that moved either horizontally (left to right) or randomly (Brownian motion) at 5°/s, 10°/s, 20°/s, or 30°/s. Task response time was measured by averaging the amount of time it took to respond to each letter per trial (i.e random 30°/s, horizontal 10°/s, etc.) which indicated the time it took for a motor response to occur. Smooth pursuit gains were tested with El-Mar eye tracker while participants completed a step-ramp task with the same respective velocities as the DVA task. A one-way independent measures ANOVA was used to analyze smooth pursuits. Relative duration of gaze behaviours were measured with the Arrington eye tracker while participants performed the DVA task. A one-way independent measures ANOVA was used to test for group differences in SVA. A one-way ANOVA was used to test for group and speed differences in DVA. A repeated-measures two-way ANOVA was used to compare gaze behaviours of the first five and last five letters of 30°/s velocity. Results: SVA was not significantly different between groups (p=0.595). Random motion DVA at 30°/s was significantly different between groups (p=0.039), specifically between athletes and controls (p=0.030). Thus, athletes were better than controls at random 30°/s. Horizontal motion DVA at 30°/s was also significantly between groups (p=0.031). Post-hoc analysis revealed a significant difference between athletes and VGPs (p=0.046). This suggests that athletes were better than VGPs at horizontal 30°/s. DVA task response time per letter was not significantly different between groups for horizontal motion at 30°/s (p=0.707) or random motion at 30°/s (p=0.723). Therefore, the motor response times were similar between groups at both motion types. Smooth pursuit gains were not significantly different between group at 30°/s (p=0.100) which indicates similar physiological eye movements. Eye movement gaze behaviours of horizontal motion at 30°/s were not significant between each groups for fixations (p=0.598), smooth pursuits (p=0.226), and saccades (p=0.523). Similarly, there was no significant difference in gaze behaviours for random motion at 30°/s between groups, for fixation (p=0.503), smooth pursuits (p=0.481), and saccades (p=0.507). Thus, gaze behaviours for horizontal and random motion were similar for all groups. Conclusion: Athletes exhibited superior DVA for randomly moving targets compared to controls, and superior DVA for horizontally moving targets compared to VGPs. The task response times, gaze behaviours and smooth pursuit gains of each group were not significantly different. Therefore task response times, smooth pursuit gains and gaze behaviours cannot explain the superior DVA displayed by the athletes. Further research is required in order to determine why DVA in athletes is superior at 30°/s