Towards End-to-end Video-based Eye-Tracking

Estimating eye-gaze from images alone is a challenging task, in large parts due to un-observable person-specific factors. Achieving high accuracy typically requires labeled data from test users which may not be attainable in real applications. We observe that there exists a strong relationship between what users are looking at and the appearance of the user's eyes. In response to this understanding, we propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. Our video dataset consists of time-synchronized screen recordings, user-facing camera views, and eye gaze data, which allows for new benchmarks in temporal gaze tracking as well as label-free refinement of gaze. Importantly, we demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures acquired through supervised personalization. Our final method yields significant performance improvements on our proposed EVE dataset, with up to a 28 percent improvement in Point-of-Gaze estimates (resulting in 2.49 degrees in angular error), paving the path towards high-accuracy screen-based eye tracking purely from webcam sensors. The dataset and reference source code are available at https://ait.ethz.ch/projects/2020/EVEComment: Accepted at ECCV 202

Protein Kinase A Dependent Phosphorylation of Apical Membrane Antigen 1 Plays an Important Role in Erythrocyte Invasion by the Malaria Parasite

Apicomplexan parasites are obligate intracellular parasites that infect a variety of hosts, causing significant diseases in livestock and humans. The invasive forms of the parasites invade their host cells by gliding motility, an active process driven by parasite adhesion proteins and molecular motors. A crucial point during host cell invasion is the formation of a ring-shaped area of intimate contact between the parasite and the host known as a tight junction. As the invasive zoite propels itself into the host-cell, the junction moves down the length of the parasite. This process must be tightly regulated and signalling is likely to play a role in this event. One crucial protein for tight-junction formation is the apical membrane antigen 1 (AMA1). Here we have investigated the phosphorylation status of this key player in the invasion process in the human malaria parasite Plasmodium falciparum. We show that the cytoplasmic tail of P. falciparum AMA1 is phosphorylated at serine 610. We provide evidence that the enzyme responsible for serine 610 phosphorylation is the cAMP regulated protein kinase A (PfPKA). Importantly, mutation of AMA1 serine 610 to alanine abrogates phosphorylation of AMA1 in vivo and dramatically impedes invasion. In addition to shedding unexpected new light on AMA1 function, this work represents the first time PKA has been implicated in merozoite invasion

An Inhibitory Antibody Blocks Interactions between Components of the Malarial Invasion Machinery

Host cell invasion by apicomplexan pathogens such as the malaria parasite Plasmodium spp. and Toxoplasma gondii involves discharge of proteins from secretory organelles called micronemes and rhoptries. In Toxoplasma a protein complex comprising the microneme apical membrane antigen 1 (AMA1), two rhoptry neck proteins, and a protein called Ts4705, localises to the moving junction, a region of close apposition between parasite and host cell during invasion. Antibodies against AMA1 prevent invasion and are protective in vivo, and so AMA1 is of widespread interest as a malaria vaccine candidate. Here we report that the AMA1 complex identified in Toxoplasma is conserved in Plasmodium falciparum. We demonstrate that the invasion-inhibitory monoclonal antibody (mAb) 4G2, which recognises P. falciparum AMA1 (PfAMA1), cannot bind when PfAMA1 is in a complex with its partner proteins. We further show that a single completely conserved PfAMA1 residue, Tyr251, lying within a conserved hydrophobic groove adjacent to the mAb 4G2 epitope, is required for complex formation. We propose that mAb 4G2 inhibits invasion by preventing PfAMA1 from interacting with other components of the invasion complex. Our findings should aid the rational design of subunit malaria vaccines based on PfAMA1

Looking to Score: The Dissociation of Goal Influence on Eye Movement and Meta-Attentional Allocation in a Complex Dynamic Natural Scene

Several studies have reported that task instructions influence eye-movement behavior during static image observation. In contrast, during dynamic scene observation we show that while the specificity of the goal of a task influences observers’ beliefs about where they look, the goal does not in turn influence eye-movement patterns. In our study observers watched short video clips of a single tennis match and were asked to make subjective judgments about the allocation of visual attention to the items presented in the clip (e.g., ball, players, court lines, and umpire). However, before attending to the clips, observers were either told to simply watch clips (non-specific goal), or they were told to watch the clips with a view to judging which of the two tennis players was awarded the point (specific goal). The results of subjective reports suggest that observers believed that they allocated their attention more to goal-related items (e.g. court lines) if they performed the goal-specific task. However, we did not find the effect of goal specificity on major eye-movement parameters (i.e., saccadic amplitudes, inter-saccadic intervals, and gaze coherence). We conclude that the specificity of a task goal can alter observer’s beliefs about their attention allocation strategy, but such task-driven meta-attentional modulation does not necessarily correlate with eye-movement behavior

What is the role of the film viewer? The effects of narrative comprehension and viewing task on gaze control in film

Film is ubiquitous, but the processes that guide viewers' attention while viewing film narratives are poorly understood. In fact, many film theorists and practitioners disagree on whether the film stimulus (bottom-up) or the viewer (top-down) is more important in determining how we watch movies. Reading research has shown a strong connection between eye movements and comprehension, and scene perception studies have shown strong effects of viewing tasks on eye movements, but such idiosyncratic top-down control of gaze in film would be anathema to the universal control mainstream filmmakers typically aim for. Thus, in two experiments we tested whether the eye movements and comprehension relationship similarly held in a classic film example, the famous opening scene of Orson Welles' Touch of Evil (Welles & Zugsmith, Touch of Evil, 1958). Comprehension differences were compared with more volitionally controlled task-based effects on eye movements. To investigate the effects of comprehension on eye movements during film viewing, we manipulated viewers' comprehension by starting participants at different points in a film, and then tracked their eyes. Overall, the manipulation created large differences in comprehension, but only produced modest differences in eye movements. To amplify top-down effects on eye movements, a task manipulation was designed to prioritize peripheral scene features: a map task. This task manipulation created large differences in eye movements when compared to participants freely viewing the clip for comprehension. Thus, to allow for strong, volitional top-down control of eye movements in film, task manipulations need to make features that are important to narrative comprehension irrelevant to the viewing task. The evidence provided by this experimental case study suggests that filmmakers' belief in their ability to create systematic gaze behavior across viewers is confirmed, but that this does not indicate universally similar comprehension of the film narrative

Juxtamembrane Shedding of Plasmodium falciparum AMA1 Is Sequence Independent and Essential, and Helps Evade Invasion-Inhibitory Antibodies

The malarial life cycle involves repeated rounds of intraerythrocytic replication interspersed by host cell rupture which releases merozoites that rapidly invade fresh erythrocytes. Apical membrane antigen-1 (AMA1) is a merozoite protein that plays a critical role in invasion. Antibodies against AMA1 prevent invasion and can protect against malaria in vivo, so AMA1 is of interest as a malaria vaccine candidate. AMA1 is efficiently shed from the invading parasite surface, predominantly through juxtamembrane cleavage by a membrane-bound protease called SUB2, but also by limited intramembrane cleavage. We have investigated the structural requirements for shedding of Plasmodium falciparum AMA1 (PfAMA1), and the consequences of its inhibition. Mutagenesis of the intramembrane cleavage site by targeted homologous recombination abolished intramembrane cleavage with no effect on parasite viability in vitro. Examination of PfSUB2-mediated shedding of episomally-expressed PfAMA1 revealed that the position of cleavage is determined primarily by its distance from the parasite membrane. Certain mutations at the PfSUB2 cleavage site block shedding, and parasites expressing these non-cleavable forms of PfAMA1 on a background of expression of the wild type gene invade and replicate normally in vitro. The non-cleavable PfAMA1 is also functional in invasion. However – in contrast to the intramembrane cleavage site - mutations that block PfSUB2-mediated shedding could not be stably introduced into the genomic pfama1 locus, indicating that some shedding of PfAMA1 by PfSUB2 is essential. Remarkably, parasites expressing shedding-resistant forms of PfAMA1 exhibit enhanced sensitivity to antibody-mediated inhibition of invasion. Drugs that inhibit PfSUB2 activity should block parasite replication and may also enhance the efficacy of vaccines based on AMA1 and other merozoite surface proteins

Does narrative drive dynamic attention to a prolonged stimulus?

Attention in the "real world" fluctuates over time, but these fluctuations are hard to examine using a timed trial-based experimental paradigm. Here we use film to study attention. To achieve short-term engagement, filmmakers make use of low-level cinematic techniques such as color, movement and sound design to influence attention. To engage audiences over prolonged periods of time, narrative structure is used. In this experiment, participants performed a secondary auditory choice reaction time (RT) task to measure attention while watching a film. In order to explore the role of narrative on attention, we manipulated the order that film segments were presented. The influence of narrative was then compared to the contribution of low-level features (extracted using a computer-based saliency model) in a multiple regression analysis predicting choice RT. The regression model successfully predicted 28% of the variance in choice RT: 13% was due to low-level saliency, and 8% due to the narrative. This study shows the importance of narrative in determining attention and the value of studying attention with a prolonged stimulus such as film

Towards End-to-End Video-Based Eye-Tracking

Estimating eye-gaze from images alone is a challenging task, in large parts due to un-observable person-specific factors. Achieving high accuracy typically requires labeled data from test users which may not be attainable in real applications. We observe that there exists a strong relationship between what users are looking at and the appearance of the user’s eyes. In response to this understanding, we propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. Our video dataset consists of time-synchronized screen recordings, user-facing camera views, and eye gaze data, which allows for new benchmarks in temporal gaze tracking as well as label-free refinement of gaze. Importantly, we demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures acquired through supervised personalization. Our final method yields significant performance improvements on our proposed EVE dataset, with up to 28% improvement in Point-of-Gaze estimates (resulting in 2.49∘ in angular error), paving the path towards high-accuracy screen-based eye tracking purely from webcam sensors. The dataset and reference source code are available at https://ait.ethz.ch/projects/2020/EVE.ISSN:0302-9743ISSN:1611-334