Vision based motion control for a humanoid head

This paper describes the design of a motion control algorithm for a humanoid robotic head, which consists of a neck with four degrees of freedom and two eyes (a stereo pair system) that tilt on a common axis and rotate sideways freely. The kinematic and dynamic properties of the head are analyzed and modeled using screw theory. The motion control algorithm is designed to receive, as an input, the output of a vision processing algorithm and to exploit the redundancy of the system for the realization of the movements. This algorithm is designed to enable the head to focus on and to follow a target, showing human-like motions. The performance of the control algorithm has been tested in a simulated environment and, then, experimentally applied to the real humanoid head

Computational Study of Multisensory Gaze-Shift Planning

In response to appearance of multimodal events in the environment, we often make a gaze-shift in order to focus the attention and gather more information. Planning such a gaze-shift involves three stages: 1) to determine the spatial location for the gaze-shift, 2) to find out the time to initiate the gaze-shift, 3) to work out a coordinated eye-head motion to execute the gaze-shift. There have been a large number of experimental investigations to inquire the nature of multisensory and oculomotor information processing in any of these three levels separately. Here in this thesis, we approach this problem as a single executive program and propose computational models for them in a unified framework. The first spatial problem is viewed as inferring the cause of cross-modal stimuli, whether or not they originate from a common source (chapter 2). We propose an evidence-accumulation decision-making framework, and introduce a spatiotemporal similarity measure as the criterion to choose to integrate the multimodal information or not. The variability of report of sameness, observed in experiments, is replicated as functions of the spatial and temporal patterns of target presentations. To solve the second temporal problem, a model is built upon the first decision-making structure (chapter 3). We introduce an accumulative measure of confidence on the chosen causal structure, as the criterion for initiation of action. We propose that gaze-shift is implemented when this confidence measure reaches a threshold. The experimentally observed variability of reaction time is simulated as functions of spatiotemporal and reliability features of the cross-modal stimuli. The third motor problem is considered to be solved downstream of the two first networks (chapter 4). We propose a kinematic strategy that coordinates eye-in-head and head-on-shoulder movements, in both spatial and temporal dimensions, in order to shift the line of sight towards the inferred position of the goal. The variabilities in contributions of eyes and head movements to gaze-shift are modeled as functions of the retinal error and the initial orientations of eyes and head. The three models should be viewed as parts of a single executive program that integrates perceptual and motor processing across time and space

Examining the influence of task set on eye movements and fixations

The purpose of the present study was to examine the influence of task set on the spatial and temporal characteristics of eye movements during scene perception. In previous work, when strong control was exerted over the viewing task via specification of a target object (as in visual search), task set biased spatial, rather than temporal, parameters of eye movements. Here, we find that more participant-directed tasks (in which the task establishes general goals of viewing rather than specific objects to fixate) affect not only spatial (e.g., saccade amplitude) but also temporal parameters (e.g., fixation duration). Further, task set influenced the rate of change in fixation duration over the course of viewing but not saccade amplitude, suggesting independent mechanisms for control of these parameters

A Distal Model of Congenital Nystagmus as Nonlinear Adaptive Oscillations

Congenital nystagmus (CN) is an incurable pathological spontaneous oscillation of the eyes with an onset in the first few months of life. The pathophysiology of CN is mysterious. There is no consistent neurological abnormality, but the majority of patients have a wide range of unrelated congenital visual abnormalities affecting either the cornea, lens, retina or optic nerve. In this theoretical study, we show that these eye oscillations could develop as an adaptive response to maximize visual contrast with poor foveal function in the infant visuomotor system, at a time of peak neural plasticity. We argue that in a visual system with abnormally poor high spatial frequency sensitivity, image contrast is not only maintained by keeping the image on the fovea (or its remnant) but also by some degree of image motion. Using the calculus of variations, we show that the optimal trade-off between these conflicting goals is to generate oscillatory eye movements with increasing velocity waveforms, as seen in real CN. When we include a stochastic component to the start of each epoch (quick-phase inaccuracy) various observed waveforms (including pseudo-cycloid) emerge as optimal strategies. Using the delay embedding technique, we find a low fractional dimension as reported in real data. We further show that, if a velocity command-based pre-motor circuitry (neural integrator) is harnessed to generate these waveforms, the emergence of a null region is inevitable. We conclude that CN could emerge paradoxically as an ‘optimal’ adaptive response in the infant visual system during an early critical period. This can explain why CN does not emerge later in life and why CN is so refractory to treatment. It also implies that any therapeutic intervention would need to be very early in life

Roles Of Inhibitory Interneurons In Cerebellar Cortical Processing For Oculomotor Control

The cerebellar cortex is usually offered up as the prime example of a well-worked out circuit; indeed, its basic neuronal composition and organization has been known for over one hundred years. Yet mysteries still abound about the computations that are performed within its layers, and how these computations contribute to sensation and behavior. This project was an effort to look inside the cerebellar cortical circuit during behavior to see if I could shed some light on the computations being performed. The dissertation is divided into three main sections. In the first, I present the results of preliminary work performed by myself and my colleagues to advance the aims of the project. This included writing software to train squirrel monkeys and control a variety of vestibulo-oculomotor tasks, characterizing the oculomotor behavioral repertoire of the squirrel monkey in comparison to that of the rhesus macaque, and developing two techniques for examining the roles of interneurons in cerebellar processing. In the second, I present the results of a study of one such interneuron, the Golgi cell, which is the main type of inhibitory interneuron that regulates information flow at the input stage of the cerebellar cortex. I recorded Golgi cells in the ventral paraflocculus: VPFL), a region of the cerebellum known to be involved in oculomotor behavior, while squirrel monkeys performed visual, vestibular, and eye movement tasks, and found that the VPFL Golgi cells only carry information from the eye movement pathways. Further, I found that this eye movement information is highly specific, with individual Golgi cells having relatively narrow directional tuning during saccades and pursuit, and only responding within a range of eye positions. This suggests that Golgi cells, through their powerful inhibition of the main path from the input stage to subsequent levels of processing, may serve as spatio-temporal filters of the information arriving at the cerebellar cortex. I delve deeper into this problem in the third section of the dissertation, where I present results from my recordings of mossy fibers and Purkinje cells, the main input and sole output elements, respectively, of the cerebellar cortex. I recorded these elements while the monkeys performed the same tasks as with the Golgi cells, sometimes while simultaneously recording Golgi cells, and examined how their responses compared with the responses of Golgi cells. I found that mossy fibers as a population are more narrowly tuned than Golgi cells, though many individual Golgi cells share a similar tuning width as the mossy fibers, and have different temporal response properties. When individual mossy fibers were recorded near, or simultaneously with, a Golgi cell, the mossy fiber and Golgi cell responses were usually antiphasic. This suggests that the net effect of mossy fiber activity on Golgi cells is inhibitory. When I examined Purkinje cell responses with respect to mossy fibers and Golgi cells, I found that the Purkinje cells generally had broader tuning and more complex, multimodal responses than Golgi cells, consistent with a greater convergence of inputs to Purkinje cells. Finally, when I examined the potential role of Purkinje cell inhibitory inputs coming from molecular layer interneurons by blocking GABA-A receptors while recording Purkinje cells, I found that this inhibition may serve to suppress bursts that are present in the eye movement-related mossy fibers that provide a dominant input to the VPFL. At the end of that chapter I attempt to synthesize these results with the results on the Golgi cells, and in the concluding chapter I suggest additional experiments to further explore the roles of cerebellar cortical interneurons in sensorimotor processing

A Gaze-Driven Digital Interface for Musical Expression Based on Real-time Physical Modelling Synthesis

Individuals with severely limited physical function such as ALS sufferers are unable to engage in conventional music-making activities, as their bodily capabilities are often limited to eye movements. The rise of modern eye-tracking cameras has led to the development of augmented digital interfaces that can allow these individuals to compose music using only their gaze. This paper presents a gaze-controlled digital interface for musical expression and performance using a real-time physical model of a xylophone. The interface was designed to work with a basic Tobii eye-tracker and a scalable, open-source framework was built using the JUCE programming environment. A usability evaluation was carried out with nine convenience-sampled participants. Whilst the interface was found to be a feasible means for gaze-driven music performance our qualitative results indicate that the utility of the interface can be enhanced by expanding the possibilities for expressive control over the physical model. Potential usability improvements include a more robust gaze calibration method, as well as a redesigned graphical interface that is friendlier to individuals lacking musical training. Overall, we see this work as a step towards accessible and inclusive musical performance interfaces for those with major physical limitations

Task-switching in oculomotor control: Systematic investigations of the unidirectional prosaccade switch-cost

An antisaccade requires suppressing a stimulus-driven prosaccade (i.e., response suppression) and remapping a target’s spatial location to its mirror-symmetrical position (i.e., vector inversion). Notably, my previous work demonstrated that the successful execution of an antisaccade selectively lengthens the reaction time (RT) of a subsequently completed prosaccade (i.e., the unidirectional prosaccade switch-cost; Weiler & Heath, 2012a; Weiler & Heath, 2012b). Thus, the objective of this dissertation was further investigate, and ultimately provide a mechanistic explanation for the unidirectional prosaccade switch-cost. In Chapter Two, I demonstrate that the magnitude of the unidirectional prosaccade switch-cost is not dependent of the number of previously executed antisaccades. Such a finding is noteworthy as it demonstrates that antisaccades do not engender additive inhibitory effects within the oculomotor system. In Chapter Three, I demonstrate that no-go catch-trials and antisaccades impart a comparable increase in RT for subsequently completed prosaccades. In accounting for this result, I propose that the top-down process of response suppression engenders a residual inhibition of the oculomotor networks that support prosaccade planning (i.e., the oculomotor inhibition hypothesis). Notably, however, the unidirectional prosaccade switch-cost could also be attributed to a persistent activation of non-standard antisaccade task-rules (i.e., a task-set) and therefore produce a prosaccade switch-cost (i.e., task-set inertia hypothesis). The goal of the Chapter Four was to test the theoretical predictions of the aforementioned hypotheses. Notably, Chapter Four demonstrates that only antisaccade trial-types – but not prosaccades trials requiring response suppression – lengthen the RT of subsequent prosaccades. As a result I conclude that the oculomotor inhibition hypothesis cannot account for the unidirectional prosaccade switch-cost. Instead I propose that the prosaccade switch-costa is due to a persistently active task-set adopted to complete the previous antisaccade response. In Chapter Five I demonstrate that alternating from an anti- to a prosaccade does not modulate the amplitude of the P3 event related brain potential. This is a notable finding as amplitude modulation of the P3 reflects task-set updating. These electrophysiological results are directly in line with my assertion that a persistently active antisaccade task-set provides the most parsimonious account for the unidirectional prosaccade switch-cost

Match-action: the role of motion and audio in creating global change blindness in film

An everyday example of change blindness is our difficulty to detect cuts in an edited moving-image. Edit Blindness (Smith & Henderson, 2008) is created by adhering to the continuity editing conventions of Hollywood, e.g. coinciding a cut with a sudden onset of motion (Match-Action). In this study we isolated the roles motion and audio play in limiting awareness of match-action cuts by removing motion before and/or after cuts in existing Hollywood film clips and presenting the clips with or without the original soundtrack whilst participants tried to detect cuts. Removing post-cut motion significantly decreased cut detection time and the probability of missing the cut. By comparison, removing pre-cut motion had no effect suggesting, contrary to the editing literature, that the onset of motion before a cut may not be as critical for creating edit blindness as the motion after a cut. Analysis of eye movements indicated that viewers reoriented less to new content across intact match-action cuts than shots with motion removed. Audio played a surprisingly large part in creating edit blindness with edit blindness mostly disappearing without audio. These results extend film editor intuitions and are discussed in the context of the Attentional Theory of Cinematic Continuity (Smith, 2012a)