Modeling Fixation Behavior in Reading with Character-level Neural Attention

Humans read text in a sequence of fixations connected by saccades spanning 7–9 characters. While most words are fixated, some are skipped, and sometimes there are reverse saccades. Previous work has explained this behavior in terms of a trade-off between the accuracy of text comprehension and the efficiency of reading, and modeled this using attention-based sequence-to-sequence neural networks. We extend this line of work by modeling the locations of individual fixations down to the character level. We evaluate our model on an eye-tracking corpus and demonstrate that it reproduces human reading patterns, both quantitatively and qualitatively. It achieves good performance in predicting fixation positions and also captures lexical effects on fixation rate and landing position effects

Optimal Control of Saccades by Spatial-Temporal Activity Patterns in the Monkey Superior Colliculus

A major challenge in computational neurobiology is to understand how populations of noisy, broadly-tuned neurons produce accurate goal-directed actions such as saccades. Saccades are high-velocity eye movements that have stereotyped, nonlinear kinematics; their duration increases with amplitude, while peak eye-velocity saturates for large saccades. Recent theories suggest that these characteristics reflect a deliberate strategy that optimizes a speed-accuracy tradeoff in the presence of signal-dependent noise in the neural control signals. Here we argue that the midbrain superior colliculus (SC), a key sensorimotor interface that contains a topographically-organized map of saccade vectors, is in an ideal position to implement such an optimization principle. Most models attribute the nonlinear saccade kinematics to saturation in the brainstem pulse generator downstream from the SC. However, there is little data to support this assumption. We now present new neurophysiological evidence for an alternative scheme, which proposes that these properties reside in the spatial-temporal dynamics of SC activity. As predicted by this scheme, we found a remarkably systematic organization in the burst properties of saccade-related neurons along the rostral-to-caudal (i.e., amplitude-coding) dimension of the SC motor map: peak firing-rates systematically decrease for cells encoding larger saccades, while burst durations and skewness increase, suggesting that this spatial gradient underlies the increase in duration and skewness of the eye velocity profiles with amplitude. We also show that all neurons in the recruited population synchronize their burst profiles, indicating that the burst-timing of each cell is determined by the planned saccade vector in which it participates, rather than by its anatomical location. Together with the observation that saccade-related SC cells indeed show signal-dependent noise, this precisely tuned organization of SC burst activity strongly supports the notion of an optimal motor-control principle embedded in the SC motor map as it fully accounts for the straight trajectories and kinematic nonlinearity of saccades

Modeling Task Effects in Human Reading with Neural Attention

Humans read by making a sequence of fixations and saccades. They often skip words, without apparent detriment to understanding. We offer a novel explanation for skipping: readers optimize a tradeoff between performing a language-related task and fixating as few words as possible. We propose a neural architecture that combines an attention module (deciding whether to skip words) and a task module (memorizing the input). We show that our model predicts human skipping behavior, while also modeling reading times well, even though it skips 40% of the input. A key prediction of our model is that different reading tasks should result in different skipping behaviors. We confirm this prediction in an eye-tracking experiment in which participants answers questions about a text. We are able to capture these experimental results using the our model, replacing the memorization module with a task module that performs neural question answering

Learning the Optimal Control of Coordinated Eye and Head Movements

Various optimality principles have been proposed to explain the characteristics of coordinated eye and head movements during visual orienting behavior. At the same time, researchers have suggested several neural models to underly the generation of saccades, but these do not include online learning as a mechanism of optimization. Here, we suggest an open-loop neural controller with a local adaptation mechanism that minimizes a proposed cost function. Simulations show that the characteristics of coordinated eye and head movements generated by this model match the experimental data in many aspects, including the relationship between amplitude, duration and peak velocity in head-restrained and the relative contribution of eye and head to the total gaze shift in head-free conditions. Our model is a first step towards bringing together an optimality principle and an incremental local learning mechanism into a unified control scheme for coordinated eye and head movements

Sampling rate influences saccade detection in mobile eye tracking of a reading task

The purpose of this study was to compare saccade detection characteristics in two mobile eye trackers with different sampling rates in a natural task. Gaze data of 11 participants were recorded in one 60 Hz and one 120 Hz mobile eye tracker and compared directly to the saccades detected by a 1000 HZ stationary tracker while a reading task was performed. Saccades and fixations were detected using a velocity based algorithm and their properties analyzed. Results showed that there was no significant difference in the number of detected fixations but mean fixation durations differed between the 60 Hz mobile and the stationary eye tracker. The 120 Hz mobile eye tracker showed a significant increase in the detection rate of saccades and an improved estimation of the mean saccade duration, compared to the 60 Hz eye tracker. To conclude, for the detection and analysis of fast eye movements, such as saccades, it is better to use a 120 Hz mobile eye tracker

Motivation in motor and cognitive control: Effects of dopamine and monetary reward and penalty

Dopamine has been identified as a key player in reward signalling and motivational processes and has been linked to apathy in Parkinson’s disease (PD), its hallmark being dopamine depletion. Direct characterisation of how dopamine modulates reward sensitivity especially in the presence of aversive stimuli is, however, still a matter of controversy. Saccadic eye movements have long been considered reward insensitive due to their high level of stereotypy, but in recent years have been recognised as a precise tool to study motor and cognitive control processes and measure reward sensitivity. This thesis investigates how oculomotor properties are influenced by different dopamine levels and motivation through both reward anticipation and penalty avoidance. Thereby I seek to shed light on the underlying pathomechanisms responsible for motor and non-motor symptoms in diseases characterized by dopamine depletion (e.g., PD). Data from the first experimental chapter suggest a common “net-value” for both incentive valences and confirms similar effects of both incentives on saccadic properties in healthy participants. The second part investigates the role of dopamine in signalling incentive values, which indicates a similar role of dopamine in both rewarding and aversive incentives. Both drugs (haloperidol and levodopa) decreased motor vigour, while having different effects on preparatory and inhibitory processes, which ultimately led to antagonistic effects on precision. Most intriguingly we also found increased reward sensitivity after a single dose of levodopa independent of incentive valence. As some of these effects might reflect motor effects of dopamine, I next examined the high-level cognitive effects using a visual working memory task. This was assessed in health as well as in a cohort of patients who had undergone VTA DBS surgery. No effect of Madopar or motivation was found on working memory in a tablet-based task, while haloperidol was detrimental to memory precision. DBS stimulation in the VTA improved performance potentially by increasing dopamine levels in the mesocorticolimbic pathway. In conclusion, this thesis aims provide a comprehensive picture of the role of nigrostriatal as well as mesolimibic dopamine on motor and cognitive control potentially aiding early diagnosis and optimising treatment strategies in disease

Optimal strategies for throwing accurately

Accuracy of throwing in games and sports is governed by how errors at projectile release are propagated by flight dynamics. To address the question of what governs the choice of throwing strategy, we use a simple model of throwing with an arm modelled as a hinged bar of fixed length that can release a projectile at any angle and angular velocity. We show that the amplification of deviations in launch parameters from a one parameter family of solution curves is quantified by the largest singular value of an appropriate Jacobian. This allows us to predict a preferred throwing style in terms of this singular value, which itself depends on target location and the target shape. Our analysis also allows us to characterize the trade-off between speed and accuracy despite not including any effects of signal-dependent noise. Using nonlinear calculations for propagating finite input-noise, we find that an underarm throw to a target leads to an undershoot, but an overarm throw does not. Finally, we consider the limit of the arm-length vanishing, i.e. shooting a projectile, and find that the most accurate shooting angle bifurcates as the ratio of the relative noisiness of the initial conditions crosses a threshold.Comment: 18 pages, 8 figure

Saccadic Eye Movements Minimize the Consequences of Motor Noise

The durations and trajectories of our saccadic eye movements are remarkably stereotyped. We have no voluntary control over these properties but they are determined by the movement amplitude and, to a smaller extent, also by the movement direction and initial eye orientation. Here we show that the stereotyped durations and trajectories are optimal for minimizing the variability in saccade endpoints that is caused by motor noise. The optimal duration can be understood from the nature of the motor noise, which is a combination of signal-dependent noise favoring long durations, and constant noise, which prefers short durations. The different durations of horizontal vs. vertical and of centripetal vs. centrifugal saccades, and the somewhat surprising properties of saccades in oblique directions are also accurately predicted by the principle of minimizing movement variability. The simple and sensible principle of minimizing the consequences of motor noise thus explains the full stereotypy of saccadic eye movements. This suggests that saccades are so stereotyped because that is the best strategy to minimize movement errors for an open-loop motor system

Dopamine promotes instrumental motivation, but reduces reward-related vigour.

We can be motivated when reward depends on performance, or merely by the prospect of a guaranteed reward. Performance-dependent (contingent) reward is instrumental, relying on an internal action-outcome model, whereas motivation by guaranteed reward may minimise opportunity cost in reward-rich environments. Competing theories propose that each type of motivation should be dependent on dopaminergic activity. We contrasted these two types of motivation with a rewarded saccade task, in patients with Parkinson's disease (PD). When PD patients were ON dopamine, they had greater response vigour (peak saccadic velocity residuals) for contingent rewards, whereas when PD patients were OFF medication, they had greater vigour for guaranteed rewards. These results support the view that reward expectation and contingency drive distinct motivational processes, and can be dissociated by manipulating dopaminergic activity. We posit that dopamine promotes goal-directed motivation, but dampens reward-driven vigour, contradictory to the prediction that increased tonic dopamine amplifies reward expectation