Neural Representations for Sensory-Motor Control I: Head-Centered 3-D Target Positions from Opponent Eye Commands

This article describes how corollary discharges from outflow eye movement commands can be transformed by two stages of opponent neural processing into a head-centered representation of 3-D target position. This representation implicitly defines a cyclopean coordinate system whose variables approximate the binocular vergence and spherical horizontal and vertical angles with respect to the observer's head. Various psychophysical data concerning binocular distance perception and reaching behavior are clarified by this representation. The representation provides a foundation for learning head-centered and body-centered invariant representations of both foveated and non-foveated 3-D target positions. It also enables a solution to be developed of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space.Air Force Office of Scientific Research (URI 90-0175); Defense Advanced Research Projects Agency (AFOSR-90-0083); National Science Foundation (IRI-87-16960, IRI-90-24877

The use of cues to convergence and accommodation in naive, uninstructed participants

A remote haploscopic video refractor was used to assess vergence and accommodation responses in a group of 32 emmetropic, orthophoric, symptom free, young adults naïve to vision experiments in a minimally instructed setting. Picture targets were presented at four positions between 2 m and 33 cm. Blur, disparity and looming cues were presented in combination or separately to asses their contributions to the total near response in a within-subjects design. Response gain for both vergence and accommodation reduced markedly whenever disparity was excluded, with much smaller effects when blur and proximity were excluded. Despite the clinical homogeneity of the participant group there were also some individual differences

Neural Representations for Sensory-Motor Control, II: Learning a Head-Centered Visuomotor Representation of 3-D Target Position

A neural network model is described for how an invariant head-centered representation of 3-D target position can be autonomously learned by the brain in real time. Once learned, such a target representation may be used to control both eye and limb movements. The target representation is derived from the positions of both eyes in the head, and the locations which the target activates on the retinas of both eyes. A Vector Associative Map, or YAM, learns the many-to-one transformation from multiple combinations of eye-and-retinal position to invariant 3-D target position. Eye position is derived from outflow movement signals to the eye muscles. Two successive stages of opponent processing convert these corollary discharges into a. head-centered representation that closely approximates the azimuth, elevation, and vergence of the eyes' gaze position with respect to a cyclopean origin located between the eyes. YAM learning combines this cyclopean representation of present gaze position with binocular retinal information about target position into an invariant representation of 3-D target position with respect to the head. YAM learning can use a teaching vector that is externally derived from the positions of the eyes when they foveate the target. A YAM can also autonomously discover and learn the invariant representation, without an explicit teacher, by generating internal error signals from environmental fluctuations in which these invariant properties are implicit. YAM error signals are computed by Difference Vectors, or DVs, that are zeroed by the YAM learning process. YAMs may be organized into YAM Cascades for learning and performing both sensory-to-spatial maps and spatial-to-motor maps. These multiple uses clarify why DV-type properties are computed by cells in the parietal, frontal, and motor cortices of many mammals. YAMs are modulated by gating signals that express different aspects of the will-to-act. These signals transform a single invariant representation into movements of different speed (GO signal) and size (GRO signal), and thereby enable YAM controllers to match a planned action sequence to variable environmental conditions.National Science Foundation (IRI-87-16960, IRI-90-24877); Office of Naval Research (N00014-92-J-1309

Neural Representations for Sensory-Motor Control, III: Learning a Body-Centered Representation of 3-D Target Position

A neural model is described of how the brain may autonomously learn a body-centered representation of 3-D target position by combining information about retinal target position, eye position, and head position in real time. Such a body-centered spatial representation enables accurate movement commands to the limbs to be generated despite changes in the spatial relationships between the eyes, head, body, and limbs through time. The model learns a vector representation--otherwise known as a parcellated distributed representation--of target vergence with respect to the two eyes, and of the horizontal and vertical spherical angles of the target with respect to a cyclopean egocenter. Such a vergence-spherical representation has been reported in the caudal midbrain and medulla of the frog, as well as in psychophysical movement studies in humans. A head-centered vergence-spherical representation of foveated target position can be generated by two stages of opponent processing that combine corollary discharges of outflow movement signals to the two eyes. Sums and differences of opponent signals define angular and vergence coordinates, respectively. The head-centered representation interacts with a binocular visual representation of non-foveated target position to learn a visuomotor representation of both foveated and non-foveated target position that is capable of commanding yoked eye movementes. This head-centered vector representation also interacts with representations of neck movement commands to learn a body-centered estimate of target position that is capable of commanding coordinated arm movements. Learning occurs during head movements made while gaze remains fixed on a foveated target. An initial estimate is stored and a VOR-mediated gating signal prevents the stored estimate from being reset during a gaze-maintaining head movement. As the head moves, new estimates arc compared with the stored estimate to compute difference vectors which act as error signals that drive the learning process, as well as control the on-line merging of multimodal information.Air Force Office of Scientific Research (F49620-92-J-0499); National Science Foundation (IRI -87-16960, IRI-90-24877); Office of Naval Research (N00014-92-J-l309

Relative contributions of the two eyes to perceived egocentric visual direction in normal binocular vision

AbstractPerceived egocentric direction (EVD) is based on the sensed position of the eyes in the orbit and the oculocentric visual direction (eye-centered, OVD). Previous reports indicate that in some subjects eye-position information from the two eyes contributes unequally to the perceived EVD. Findings from other studies indicate that the retinal information from the two eyes may not always contribute equally to perceived OVD. The goal of this study was to assess whether these two sources of information covary similarly within the same individuals. Open-loop pointing responses to an isolated target presented randomly at several horizontal locations were collected from 13 subjects during different magnitudes of asymmetric vergence to estimate the contribution of the position information from each eye to perceived EVD. For the same subjects, the direction at which a horizontally or vertically disparate target with different interocular contrast or luminance ratios appeared aligned with a non-disparate target estimated the relative contribution of each eye’s retinal information. The results show that the eye-position and retinal information vary similarly in most subjects, which is consistent with a modified version of Hering’s law of visual direction

An automated and objective cover test to measure heterophoria

Postprint (published version

Using natural versus artificial stimuli to perform calibration for 3D gaze tracking

International audienceThe presented study tests which type of stereoscopic image, natural or artificial, is more adapted to perform efficient and reliable calibration in order to track the gaze of observers in 3D space using classical 2D eye tracker. We measured the horizontal disparities, i.e. the difference between the x coordinates of the two eyes obtained using a 2D eye tracker. This disparity was recorded for each observer and for several target positions he had to fixate. Target positions were equally distributed in the 3D space, some on the screen (with a null disparity), some behind the screen (uncrossed disparity) and others in front of the screen (crossed disparity). We tested different regression models (linear and non linear) to explain either the true disparity or the depth with the measured disparity. Models were tested and compared on their prediction error for new targets at new positions. First of all, we found that we obtained more reliable disparities measures when using natural stereoscopic images rather than artificial. Second, we found that overall a non-linear model was more efficient. Finally, we discuss the fact that our results were observer dependent, with variability's between the observer's behavior when looking at 3D stimuli. Because of this variability, we proposed to compute observer specific model to accurately predict their gaze position when exploring 3D stimuli

Development of new methodologies for the clinical, objective and automated evaluation of visual function based on the analysis of ocular movements : application in visual health

Healthy visual function not only relies on good visual acuity. Other systems such as accommodation or binocular vision need to be effective. Binocular vision is the ability of the visual system to coordinate and integrate the information received separately from the two eyes into a single binocular percept. Anomalies in the binocular vision system lead to dysfunctions which are often associated with symptoms like asthenopia, diplopia, or ocular strain. The most prevalent non-strabismic binocular dysfunction is convergence insufficiency. The tests typically used to evaluate binocular vision and diagnose binocular dysfunctions essentially consist in eliciting eye movements and asking the patients to report when they perceive diplopia or single vision. In a conventional optometric clinical setting, they are generally run subjectively, as their results depend on the answers of the patients or on the examiner’s criteria. However, there exist instruments to determine the gaze position and measure ocular movements objectively, i.e. eye-trackers. These instruments are used as research tools in a wide range of applications (neuroscience, psychology, marketing, computer science, ophthalmology, etc.). Eye tracking systems are seldom used in optometric clinical practice, although it seems reasonable to think that the evaluation of binocular vision could better rely on eye tracking systems rather than subjective observations. In this context, the main objective of this thesis is to develop new methodologies for the clinical, objective and automated evaluation of visual function based on the analysis of ocular movements. This thesis is divided in 4 studies. In the first study, new methods for an eye tracking system based on multiple corneal reflections are proposed. The other 3 studies aim to analyze ocular movements in clinically interesting situations for the objective and automated evaluation of binocular vision. The results of the first study showed that the light sources configurations that produced the reflections in the lower region of the cornea showed higher accuracy. Vertical accuracy was slightly better with a higher number of corneal reflections. However, the proposed normalization methods improved vertical accuracy and counteracted the tendency for increasing accuracy with the number of glints. As a result, if the light sources are optimally positioned to avoid the interference of the eyelids and the normalization methods are applied, there is no need for more than two light sources. In the second study, an automated and objective method to measure phoria was proposed. It was significantly more repeatable than two other conventional clinical methods. However, the phoria results of the three tests were not interchangeable. This study brings to light several advantages of using eye-trackers in optometric clinical settings. The third study analyzes the characteristics of saccadic movements that occur during the near point of convergence test. The results showed that saccadic amplitude increased and rate decrease at closer viewing distances. These changes might be explained by the more rapid change of vergence demand and the greater angular size of the fixation target at near than at far. In general, saccades contributed to correct vergence errors and fixation position errors of the dominant eye. Finally, the fourth study focuses on the effects of the stimulus’ predictability on the latency and response time of vergence step movements. The results confirmed that vergence movements to predictable stimulus had shorter latency and response time than when the stimulus was random. Latency of convergence and divergence movements was influenced by the direction of the phoria. Other factors such as attention or voluntary effort might also affect vergence responses. All these effects might influence the final result of the vergence facility test, although further research is needed to specify the impact on the clinical test.Una bona funció visual no és sinònim exclusivament de bona agudesa visual. Cal que altres sistemes com l’acomodatiu o la visió binocular siguin eficaços. La visió binocular és la capacitat del sistema visual per coordinar i integrar la informació que reben els dos ulls en una única percepció. Anomalies en el sistema de visió binocular poden donar lloc a disfuncions associades a símptomes com astenopia, o visió doble. La disfunció no estràbica de la visió binocular més prevalent és la insuficiència de convergència. Els tests que normalment es fan per avaluar la visió binocular es basen en estimular moviments oculars i demanar als pacients que indiquin quan veuen doble i quan fusionen. A la pràctica clínica convencional, aquests tests solen ser subjectius. No obstant, hi ha instruments que serveixen per determinar objectivament la posició de mirada i mesurar els moviments oculars: els eye-trackers o instruments de seguiment de mirada. Aquests instruments s’utilitzen en recerca en moltes disciplines (neurociència, psicologia, marketing, oftalmologia, etc.). Tot i que una aplicació directa dels eye-trackers podria ser en l’avaluació de la visió binocular, els sistemes de seguiment de mirada gairebé no s’utilitzen en la pràctica clínica optomètrica. En aquest context, l’objectiu principal d’aquesta tesi és desenvolupar noves metodologies per a l’avaluació clínica, objectiva i automatitzada de la funció visual basades en l’anàlisi dels moviments oculars. La tesi està estructurada en 4 estudis. En el primer, es proposen nous mètodes per a un eye-tracker basat en múltiples reflexos corneals. Els altres 3 estudis tenen com a objectiu analitzar els moviments oculars en situacions d’interès clínic per avaluar objectiva i automàticament la visió binocular. Els resultats del primer estudi demostren que les configuracions d’il·luminació dels eye-trackers que formen les reflexions corneals a la zona inferior de la còrnia són més precises. La precisió vertical és lleugerament millor amb més fonts de llum. Tot i això, els mètodes de normalització proposats milloren considerablement la precisió vertical i contraresten la tendència de més precisió amb més fonts de llum. D’aquesta manera, si les fonts de llum no interfereixen amb les parpelles i s’apliquen els mètodes de normalització, no cal que els eye-trackers tinguin més de dues fonts de llum. En el segon estudi s’ha proposat un mètode automàtic i objectiu per mesurar la fòria. Aquest mètode és significativament més repetitiu que dos altres mètodes clínics. Tot i això, els resultats amb els tres mètodes no són intercanviables. Aquest estudi posa de manifest avantatges que podrien tenir els eye-trackers a la pràctica clínica optomètrica. En el tercer estudi s’analitzen les característiques dels moviments sacàdics que es produeixen durant la prova del punt proper de convergència. Els resultats demostren que l’amplitud dels sacàdics augmenta i la freqüència disminueix a mesura que s’escurça la distància. Aquests canvis poden ser deguts a que la demanda de vergència canvia més ràpid a distàncies properes que llunyanes, i a l’increment de la mida angular de l’objecte de fixació. En general, els sacàdics contribueixen a corregir els errors de vergència i els errors de fixació de l’ull dominant. Finalment, el quart estudi se centra en els efectes de la predictibilitat de l’estímul en la latència i temps de resposta dels salts de vergència. Els resultats confirmen que els moviments de vergència tenen una latència i temps de resposta més curts quan l’estímul es predictible que quan és aleatori. La latència dels moviments de convergència i divergència està influenciada per la direcció de la fòria. Altres factors com el grau d’atenció o d’esforç voluntari pot ser que afectin els moviments de vergència. Tots aquests efectes probablement influencien en el resultat final de la prova de la instal·lació de vergència, encara que es necessiten més investigacions per especificar l'impacte en la prova clínica

Binocular alignment and vergence errors in free space

The human, along with other primates, has forward placed eyes, and an area of acute vision (the fovea) on each retina. The overlap of the visual fields and the hemi-decussation of the visual pathways at the optic chiasm provide the basis for binocular vision, in particular stereopsis, the accurate perception of the position of objects in three dimensional space and an improved ability to perceive the form of solid objects. An intricate system of eye movements is needed to achieve and maintain stable foveal fixation on each eye in an environment where visual targets vary in direction and depth, where the visual environment may be moving, the eyes or the rest of the body is moving. The purpose of this study is to evaluate the accuracy of binocular alignment for far and near fixations, under relatively natural conditions. To achieve binocular fixation, accurate vergence eye movements are required to align the eyes, and to maintain this alignment when a person changes fixation to objects situated at different distances from the eyes. ‘Pure’ vergence eye movements occur when these objects are situated along the mid sagittal plane, however, in natural conditions other eye movement systems are also involved. To understand the contribution of different eye movement systems to binocular fixation at different distances, the accuracy of binocular alignment in subjects with normal binocular single vision was evaluated in subjects with normal binocular vision under the following conditions • Fixation on targets along the mid sagittal plane (vergence eye movements only) • Fixation on targets displaced to either side of the mid sagittal plane (combined vergence eye movements and saccades • Fixation on earth fixed targets situated straight ahead in space, but with the head tilted to either side (combined vergence eye movements, saccades and torsional eye movements). The protocol for all experiments was approved by the Human Ethics Committee of the University of Sydney and followed the tenets of the Declaration of Helsinki. Throughout this thesis the term ‘binocular alignment’ will be used to describe the position of each eye during or following a change in vergence. The term ‘vergence error’ will refer to situations where the angle of vergence alignment is different from that required, so that the image of the fixation target does not fall on the fovea of one or both eyes

Development of new methodologies for the clinical, objective and automated evaluation of visual function based on the analysis of ocular movements : application in visual health

Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit de CiènciesHealthy visual function not only relies on good visual acuity. Other systems such as accommodation or binocular vision need to be effective. Binocular vision is the ability of the visual system to coordinate and integrate the information received separately from the two eyes into a single binocular percept. Anomalies in the binocular vision system lead to dysfunctions which are often associated with symptoms like asthenopia, diplopia, or ocular strain. The most prevalent non-strabismic binocular dysfunction is convergence insufficiency. The tests typically used to evaluate binocular vision and diagnose binocular dysfunctions essentially consist in eliciting eye movements and asking the patients to report when they perceive diplopia or single vision. In a conventional optometric clinical setting, they are generally run subjectively, as their results depend on the answers of the patients or on the examiner’s criteria. However, there exist instruments to determine the gaze position and measure ocular movements objectively, i.e. eye-trackers. These instruments are used as research tools in a wide range of applications (neuroscience, psychology, marketing, computer science, ophthalmology, etc.). Eye tracking systems are seldom used in optometric clinical practice, although it seems reasonable to think that the evaluation of binocular vision could better rely on eye tracking systems rather than subjective observations. In this context, the main objective of this thesis is to develop new methodologies for the clinical, objective and automated evaluation of visual function based on the analysis of ocular movements. This thesis is divided in 4 studies. In the first study, new methods for an eye tracking system based on multiple corneal reflections are proposed. The other 3 studies aim to analyze ocular movements in clinically interesting situations for the objective and automated evaluation of binocular vision. The results of the first study showed that the light sources configurations that produced the reflections in the lower region of the cornea showed higher accuracy. Vertical accuracy was slightly better with a higher number of corneal reflections. However, the proposed normalization methods improved vertical accuracy and counteracted the tendency for increasing accuracy with the number of glints. As a result, if the light sources are optimally positioned to avoid the interference of the eyelids and the normalization methods are applied, there is no need for more than two light sources. In the second study, an automated and objective method to measure phoria was proposed. It was significantly more repeatable than two other conventional clinical methods. However, the phoria results of the three tests were not interchangeable. This study brings to light several advantages of using eye-trackers in optometric clinical settings. The third study analyzes the characteristics of saccadic movements that occur during the near point of convergence test. The results showed that saccadic amplitude increased and rate decrease at closer viewing distances. These changes might be explained by the more rapid change of vergence demand and the greater angular size of the fixation target at near than at far. In general, saccades contributed to correct vergence errors and fixation position errors of the dominant eye. Finally, the fourth study focuses on the effects of the stimulus’ predictability on the latency and response time of vergence step movements. The results confirmed that vergence movements to predictable stimulus had shorter latency and response time than when the stimulus was random. Latency of convergence and divergence movements was influenced by the direction of the phoria. Other factors such as attention or voluntary effort might also affect vergence responses. All these effects might influence the final result of the vergence facility test, although further research is needed to specify the impact on the clinical test.Una bona funció visual no és sinònim exclusivament de bona agudesa visual. Cal que altres sistemes com l’acomodatiu o la visió binocular siguin eficaços. La visió binocular és la capacitat del sistema visual per coordinar i integrar la informació que reben els dos ulls en una única percepció. Anomalies en el sistema de visió binocular poden donar lloc a disfuncions associades a símptomes com astenopia, o visió doble. La disfunció no estràbica de la visió binocular més prevalent és la insuficiència de convergència. Els tests que normalment es fan per avaluar la visió binocular es basen en estimular moviments oculars i demanar als pacients que indiquin quan veuen doble i quan fusionen. A la pràctica clínica convencional, aquests tests solen ser subjectius. No obstant, hi ha instruments que serveixen per determinar objectivament la posició de mirada i mesurar els moviments oculars: els eye-trackers o instruments de seguiment de mirada. Aquests instruments s’utilitzen en recerca en moltes disciplines (neurociència, psicologia, marketing, oftalmologia, etc.). Tot i que una aplicació directa dels eye-trackers podria ser en l’avaluació de la visió binocular, els sistemes de seguiment de mirada gairebé no s’utilitzen en la pràctica clínica optomètrica. En aquest context, l’objectiu principal d’aquesta tesi és desenvolupar noves metodologies per a l’avaluació clínica, objectiva i automatitzada de la funció visual basades en l’anàlisi dels moviments oculars. La tesi està estructurada en 4 estudis. En el primer, es proposen nous mètodes per a un eye-tracker basat en múltiples reflexos corneals. Els altres 3 estudis tenen com a objectiu analitzar els moviments oculars en situacions d’interès clínic per avaluar objectiva i automàticament la visió binocular. Els resultats del primer estudi demostren que les configuracions d’il·luminació dels eye-trackers que formen les reflexions corneals a la zona inferior de la còrnia són més precises. La precisió vertical és lleugerament millor amb més fonts de llum. Tot i això, els mètodes de normalització proposats milloren considerablement la precisió vertical i contraresten la tendència de més precisió amb més fonts de llum. D’aquesta manera, si les fonts de llum no interfereixen amb les parpelles i s’apliquen els mètodes de normalització, no cal que els eye-trackers tinguin més de dues fonts de llum. En el segon estudi s’ha proposat un mètode automàtic i objectiu per mesurar la fòria. Aquest mètode és significativament més repetitiu que dos altres mètodes clínics. Tot i això, els resultats amb els tres mètodes no són intercanviables. Aquest estudi posa de manifest avantatges que podrien tenir els eye-trackers a la pràctica clínica optomètrica. En el tercer estudi s’analitzen les característiques dels moviments sacàdics que es produeixen durant la prova del punt proper de convergència. Els resultats demostren que l’amplitud dels sacàdics augmenta i la freqüència disminueix a mesura que s’escurça la distància. Aquests canvis poden ser deguts a que la demanda de vergència canvia més ràpid a distàncies properes que llunyanes, i a l’increment de la mida angular de l’objecte de fixació. En general, els sacàdics contribueixen a corregir els errors de vergència i els errors de fixació de l’ull dominant. Finalment, el quart estudi se centra en els efectes de la predictibilitat de l’estímul en la latència i temps de resposta dels salts de vergència. Els resultats confirmen que els moviments de vergència tenen una latència i temps de resposta més curts quan l’estímul es predictible que quan és aleatori. La latència dels moviments de convergència i divergència està influenciada per la direcció de la fòria. Altres factors com el grau d’atenció o d’esforç voluntari pot ser que afectin els moviments de vergència. Tots aquests efectes probablement influencien en el resultat final de la prova de la instal·lació de vergència, encara que es necessiten més investigacions per especificar l'impacte en la prova clínica.Award-winningPostprint (published version