Eye Movement and Pupil Measures: A Review

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first describe the main oculomotor events studied in the literature, and their characteristics exploited by different measures. Next, we review various eye movement and pupil measures from prior literature. Finally, we discuss our observations based on applications of these measures, the benefits and practical challenges involving these measures, and our recommendations on future eye-tracking research directions

Digital Oculomotor Biomarkers in Dementia

Dementia is an umbrella term that covers a number of neurodegenerative syndromes featuring gradual disturbance of various cognitive functions that are severe enough to interfere with tasks of daily life. The diagnosis of dementia occurs frequently when pathological changes have been developing for years, symptoms of cognitive impairment are evident and the quality of life of the patients has already been deteriorated significantly. Although brain imaging and fluid biomarkers allow the monitoring of disease progression in vivo, they are expensive, invasive and not necessarily diagnostic in isolation. Recent studies suggest that eye-tracking technology is an innovative tool that holds promise for accelerating early detection of the disease, as well as, supporting the development of strategies that minimise impairment during every day activities. However, the optimal methods for quantitative evaluation of oculomotor behaviour during complex and naturalistic tasks in dementia have yet to be determined. This thesis investigates the development of computational tools and techniques to analyse eye movements of dementia patients and healthy controls under naturalistic and less constrained scenarios to identify novel digital oculomotor biomarkers. Three key contributions are made. First, the evaluation of the role of environment during navigation in patients with typical Alzheimer disease and Posterior Cortical Atrophy compared to a control group using a combination of eye movement and egocentric video analysis. Secondly, the development of a novel method of extracting salient features directly from the raw eye-tracking data of a mixed sample of dementia patients during a novel instruction-less cognitive test to detect oculomotor biomarkers of dementia-related cognitive dysfunction. Third, the application of unsupervised anomaly detection techniques for visualisation of oculomotor anomalies during various cognitive tasks. The work presented in this thesis furthers our understanding of dementia-related oculomotor dysfunction and gives future research direction for the development of computerised cognitive tests and ecological interventions

The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study

Carminati MN, Knoeferle P. The Processing of Emotional Sentences by Young and Older Adults: A Visual World Eye-movement Study. Presented at the Architectures and Mechanisms of Language and Processing (AMLaP), Riva del Garda, Italy

The role of phonology in visual word recognition: evidence from Chinese

Posters - Letter/Word Processing V: abstract no. 5024The hypothesis of bidirectional coupling of orthography and phonology predicts that phonology plays a role in visual word recognition, as observed in the effects of feedforward and feedback spelling to sound consistency on lexical decision. However, because orthography and phonology are closely related in alphabetic languages (homophones in alphabetic languages are usually orthographically similar), it is difficult to exclude an influence of orthography on phonological effects in visual word recognition. Chinese languages contain many written homophones that are orthographically dissimilar, allowing a test of the claim that phonological effects can be independent of orthographic similarity. We report a study of visual word recognition in Chinese based on a mega-analysis of lexical decision performance with 500 characters. The results from multiple regression analyses, after controlling for orthographic frequency, stroke number, and radical frequency, showed main effects of feedforward and feedback consistency, as well as interactions between these variables and phonological frequency and number of homophones. Implications of these results for resonance models of visual word recognition are discussed.postprin

Interactive effects of orthography and semantics in Chinese picture naming

Posters - Language Production/Writing: abstract no. 4035Picture-naming performance in English and Dutch is enhanced by presentation of a word that is similar in form to the picture name. However, it is unclear whether facilitation has an orthographic or a phonological locus. We investigated the loci of the facilitation effect in Cantonese Chinese speakers by manipulating—at three SOAs (2100, 0, and 1100 msec)—semantic, orthographic, and phonological similarity. We identified an effect of orthographic facilitation that was independent of and larger than phonological facilitation across all SOAs. Semantic interference was also found at SOAs of 2100 and 0 msec. Critically, an interaction of semantics and orthography was observed at an SOA of 1100 msec. This interaction suggests that independent effects of orthographic facilitation on picture naming are located either at the level of semantic processing or at the lemma level and are not due to the activation of picture name segments at the level of phonological retrieval.postprin

How to improve learning from video, using an eye tracker

The initial trigger of this research about learning from video was the availability of log files from users of video material. Video modality is seen as attractive as it is associated with the relaxed mood of watching TV. The experiments in this research have the goal to gain more insight in viewing patterns of students when viewing video. Students received an awareness instruction about the use of possible alternative viewing behaviors to see whether this would enhance their learning effects. We found that: - the learning effects of students with a narrow viewing repertoire were less than the learning effects of students with a broad viewing repertoire or strategic viewers. - students with some basic knowledge of the topics covered in the videos benefited most from the use of possible alternative viewing behaviors and students with low prior knowledge benefited the least. - the knowledge gain of students with low prior knowledge disappeared after a few weeks; knowledge construction seems worse when doing two things at the same time. - media players could offer more options to help students with their search for the content they want to view again. - there was no correlation between pervasive personality traits and viewing behavior of students. The right use of video in higher education will lead to students and teachers that are more aware of their learning and teaching behavior, to better videos, to enhanced media players, and, finally, to higher learning effects that let users improve their learning from video

Pattern codes for perceived gaze direction revealed by functional MRI

Perceiving the direction of another's attention is a critical component of normal social behaviour. Seminal electrophysiology studies demonstrated that single cells in macaque superior temporal sulcus (STS) are tuned to specific directions of social cues, including gaze direction, head view, and body posture. Furthermore, a subset of such neurons respond to a single direction across multiple cues, suggesting that the code is driven by the direction of another's social attention regardless of how this is conveyed. Attempts to reveal similar gaze representations in humans using fMRI have provided mixed results. This thesis describes research where multivariate pattern analysis (MVPA) methods are applied to fMRI data in order to better explain how the human brain and particularly STS codes perceived gaze direction. After describing the MVPA methods applied in this thesis, I first demonstrate that fMRI response patterns in anterior STS distinguish between the direction of dynamic head turns, but not between the direction of rotation in non-social ellipsoids. In subsequent work, anterior STS is found to code the direction of another's gaze in a head view-invariant manner, thus demonstrating a potential parallel to previous macaque evidence for single cells that code the direction of another's attention. However, comparisons that run both across species (macaque, human) and methods (electrophysiology, fMRI) are problematic. To overcome this limitation I next tested whether macaque STS distinguishes gaze direction and head view when responses are measured with fMRI. In conclusion, this thesis demonstrates the utility of applying MVPA to fMRI data to reveal socially-relevant representations of the direction of another's attention. The thesis particularly highlights anterior STS as a key region in supporting direction-specific representations of social cues. These results advance our understanding of how the brain codes socially-relevant information, and highlight possible similarities and dissimilarities between humans and macaques.This work was supported by the Medical Research Council

An Investigation of Attention to Social and Non-social Stimuli in Autism Spectrum Disorder and Typical Development

Autism Spectrum Disorder (ASD) is a pervasive, neurodevelopmental disorder that can be characterized by deficits in three symptom domains: significant impairments in language, social deficits, and restricted and repetitive behaviors. Narrowed interests, perseverative patterns of attention and reduced visual exploration have been conceptually linked to repetitive behaviors in ASD, specifically what are known as circumscribed interests (CI) within a narrow range of subject areas. Individuals with ASD that have CI partake in activities around their interest (collecting, manipulating, reading, playing, conversing, etc.) and these activities often lead to functional impairments. Eye-tracking research has investigated different responses to categories of images reflecting CI that capture attention during passive viewing tasks. Children and adults with ASD display an attentional bias towards certain categories of nonsocial images (e.g. train, automobiles, electronic devices, computers). This bias has been conceptualized to reflect an increased salience of nonsocial images relative to social images (e.g. faces) or other, more commonplace, nonsocial information (e.g. furniture, clothing, dishes). This dissertation aimed to extend these findings regarding atypical patterns of attention to social and nonsocial information in children with ASD by investigating reflexive attention and cognitive control over attention of images related to CI, skills essential for behavioral and brain development in children with ASD. Analyses included group comparisons across children with ASD and typically developing controls. Children with ASD demonstrated a visual preference for non-social objects relative to social information during a passive viewing attention task. During a visual saccade task, though children with ASD did demonstrate an increased rate of directional errors, task performance did not differ across social and non-social targets. Eye-tracking measures were found to be significantly related to symptom measures of social-communication impairments and restricted and repetitive behaviors in ASD. Exploratory comparisons across children with ASD and a pediatric OCD sample, another development disorder characterized by repetitive behaviors, are also discussed. This study provides support for the use of visual attention and oculomotor behavior to quantify impairments in ASD as well as discrete aspects of the repetitive behavior phenotype.Doctor of Philosoph

Word Learning in 6-16 Month Old Infants

Understanding words requires infants to not only isolate words from the speech around them and delineate concepts from their world experience, but also to establish which words signify which concepts, in all and only the right set of circumstances. Previous research places the onset of this ability around infants\u27 first birthdays, at which point they have begun to solidify their native language phonology, and have learned a good deal about categories, objects, and people. In this dissertation, I present research that alters this accepted timeline. In Study 1, I find that by 6 months of age, infants demonstrate understanding of around a dozen words for foods and body parts. Around 13-14 months of age, performance increases significantly. In Study 2, I find that for a set of early non-nouns, e.g. `uh-oh\u27 and `eat\u27, infants do not show understanding until 10 months, but again show a big comprehension boost around 13-14 months. I discuss possible reasons for the onset of noun-comprehension at 6 months, the relative delay in non-noun comprehension, and the performance boost for both word-types around 13-14 months. In Study 3, I replicate and extend Study 1\u27s findings, showing that around 6 months infants also understand food and body-part words when these words are spoken by a new person, but conversely, by 12 months, show poor word comprehension if a single vowel in the word is changed, even when the speaker is highly familiar. Taken together, these results suggest that word learning begins before infants have fully solidified their native language phonology, that certain generalizations about words are available to infants at the outset of word comprehension, and that infants are able to learn words for complex object and event categories before their first birthday. Implications for language acquisition and cognitive development more broadly are discussed