Seeing other perspectives: Evaluating the use of virtual and augmented reality to simulate visual impairments (OpenVisSim)

Simulations of visual impairment are often used to educate and inform the public. However, evidence regarding their accuracy remains lacking. Here we evaluated the effectiveness of modern digital technologies to simulate the everyday difficulties caused by glaucoma. Twenty5 three normally-sighted adults performed two everyday tasks that glaucoma patients often report difficulties with: a visual search task in which they attempted to locate a mobile phone in virtual domestic environments (Virtual Reality; VR), and a visual mobility task in which impairments were overlaid onto a real-world environment using Augmented Reality (AR). On some trials, a gaze-contingent simulated scotoma --- generated using perimetric data from a patient with advanced glaucoma --- was presented in either the superior or inferior hemifield. The main outcome measure was task completion time. Eye and head movements were also tracked and used to assess individual differences in looking behaviors. The results showed that the simulated impairments substantial impaired performance in both the VR (visual search) and AR (visual mobility) tasks (both P < 0.001). Furthermore, and in line with previous patient data: impairments were greatest when the simulated VFL was inferior versus superior (P < 0.001), participants made more eye and head movements in the inferior VFL condition (P < 0.001), and participants rated the inferior VFL condition as more difficult (P < 0.001). Notably, the difference in performance between the inferior and superior conditions was almost as great as the difference between a superior VFL and no impairment at all (VR: 71%; AR: 70%). We conclude that modern digital simulators are able to replicate and objectively quantify some of the key everyday difficulties associated with visual impairments. Advantages, limitations and possible applications of current technologies are discussed. Instructions are also given for how to freely obtain the software described (OpenVisSim)

Rapid Adaptation of Visual Search in Simulated Hemianopia

Patients with homonymous hemianopia have altered visual search patterns, but it is unclear how rapidly this develops and whether it reflects a strategic adaptation to altered perception or plastic changes to tissue damage. To study the temporal dynamics of adaptation alone, we used a gaze-contingent display to simulate left or right hemianopia in 10 healthy individuals as they performed 25 visual search trials. Visual search was slower and less accurate in hemianopic than in full-field viewing. With full-field viewing, there were improvements in search speed, fixation density, and number of fixations over the first 9 trials, then stable performance. With hemianopic viewing, there was a rapid shift of fixation into the blind field over the first 5-7 trials, followed by continuing gradual improvements in completion time, number of fixations, and fixation density over all 25 trials. We conclude that in the first minutes after onset of hemianopia, there is a biphasic pattern of adaptation to altered perception: an early rapid qualitative change that shifts visual search into the blind side, followed by more gradual gains in the efficiency of using this new strategy, a pattern that has parallels in other studies of motor learnin

Saccade Landing Point Prediction Based on Fine-Grained Learning Method

The landing point of a saccade defines the new fixation region, the new region of interest. We asked whether it was possible to predict the saccade landing point early in this very fast eye movement. This work proposes a new algorithm based on LSTM networks and a fine-grained loss function for saccade landing point prediction in real-world scenarios. Predicting the landing point is a critical milestone toward reducing the problems caused by display-update latency in gaze-contingent systems that make real-time changes in the display based on eye tracking. Saccadic eye movements are some of the fastest human neuro-motor activities with angular velocities of up to 1,000°/s. We present a comprehensive analysis of the performance of our method using a database with almost 220,000 saccades from 75 participants captured during natural viewing of videos. We include a comparison with state-of-the-art saccade landing point prediction algorithms. The results obtained using our proposed method outperformed existing approaches with improvements of up to 50% error reduction. Finally, we analyzed some factors that affected prediction errors including duration, length, age, and user intrinsic characteristics.This work was supported in part by the Project BIBECA through MINECO/FEDER under Grant RTI2018-101248-B-100, in part by the Jose Castillejo Program through MINECO under Grant CAS17/00117, and in part by the National Institutes of Health (NIH) under Grant P30EY003790 and Grant R21EY023724

Eye movements in hemianopia and the rehabilitation of hemianopic dyslexia

This thesis is a study of the nature and rehabilitation of the functional impairments in unilateral homonymous hemianopia (HH), with a major focus on hemianopic dyslexia. The reading, visual exploration and line bisection impairments associated with homonymous visual field loss are frequent and well-established clinical phenomena. Yet, it is still unknown whether the reading and visual exploration impairments are caused by the visual field defect or by additional extrastriate injury preventing efficient spontaneous oculomotor adaptation. It is also unclear whether the line bisection impairment directly arises from the visual field defect or its adaptive oculomotor consequences, or whether it indicates an associated visual-spatial deficit that is caused by injury to regions involved in visual-spatial perception (Introduction). Based on a critical review of research into hemianopic dyslexia since its original description in 1881, it is suggested that the visual field defect is a major component of hemianopic dyslexia but possibly not its sole cause (Chapter 1). This assumption was confirmed in six experiments whose purpose was to establish the extent to which the reading, visual exploration and line bisection impairments associated with HH are purely visually elicited. To study the behavioural changes associated with the visual field defect that are not caused by brain injury, a gaze-contingent display paradigm was used to simulate HH in healthy participants. Simulated HH induced the reading and visual exploration impairments of hemianopic patients. However, all participants showed efficient spontaneous oculomotor adaptation to simulated HH which was associated with highly specific and task-dependent improvements in reading and visual exploration (Chapters 2 and 3). Moreover, simulated HH did not induce the main feature of the hemianopic line bisection impairment, i.e., the contralateral line bisection error, albeit it nevertheless impaired line bisection performance (Chapter 4). The final study investigated the basis and specificity of the therapeutic effect of an efficient compensatory oculomotor treatment method for hemianopic dyslexia in patients with unilateral homonymous visual field loss. The results demonstrate that using text-material and, thus, lexical-semantic processes, is not critical to the treatment effect, which was also found to be specific to reading (Chapter 5). The concluding chapter reviews the main findings and suggests that the functional impairments associated with visual field loss may not simply be failures of vision. Although the hemianopic visual field defect is a major component of hemianopic dyslexia and possibly contributes to the visual exploration and line bisection impairments, additional injury to specific extrastriate regions seems to be the critical causative factor. The implications for understanding, assessing and rehabilitating functional impairments in homonymous visual field disorders are discussed. The important future research directions arising from this thesis are also identified and presented (Conclusion)

Modelling visual search for surface defects

Much work has been done on developing algorithms for automated surface defect detection. However, comparisons between these models and human perception are rarely carried out. This thesis aims to investigate how well human observers can nd defects in textured surfaces, over a wide range of task di culties. Stimuli for experiments will be generated using texture synthesis methods and human search strategies will be captured by use of an eye tracker. Two di erent modelling approaches will be explored. A computational LNL-based model will be developed and compared to human performance in terms of the number of xations required to find the target. Secondly, a stochastic simulation, based on empirical distributions of saccades, will be compared to human search strategies

Inefficient search strategies in simulated hemianopia.

We investigated whether healthy participants can spontaneously adopt effective eye movement strategies to compensate for information loss similar to that experienced by patients with damage to visual cortex (hemianopia). Visual information in 1 hemifield was removed or degraded while participants searched for an emotional face among neutral faces or a line tilted 45° to the right among lines of varying degree of tilt. A bias to direct saccades toward the sighted field was observed across all 4 experiments. The proportion of saccades directed toward the “blind” field increased with the amount of information available in that field, suggesting fixations are driven toward salient visual stimuli rather than toward locations that maximize information gain. In Experiments 1 and 2, the sighted-field bias had a minimal impact on search efficiency, because the target was difficult to find. However, the sighted-field bias persisted even when the target was visually distinct from the distractors and could easily be detected in the periphery (Experiments 3 and 4). This surprisingly inefficient search behavior suggests that eye movements are biased to salient visual stimuli even when it comes at a clear cost to search efficiency, and efficient strategies to compensate for visual deficits are not spontaneously adopted by healthy participants

The worse eye revisited: Evaluating the impact of asymmetric peripheral vision loss on everyday function

In instances of asymmetric peripheral vision loss (e.g., glaucoma), binocular performance on simple psychophysical tasks (e.g., static threshold perimetry) is well-predicted by the better seeing eye alone. This suggests that peripheral vision is largely ‘better-eye limited’. In the present study, we examine whether this also holds true for real-world tasks, or whether even a degraded fellow eye contributes important information for tasks of daily living. Twelve normally-sighted adults performed an everyday visually-guided action (finding a mobile phone) in a virtual-reality domestic environment, while levels of peripheral vision loss were independently manipulated in each eye (gaze-contingent blur). The results showed that even when vision in the better eye was held constant, participants were significantly slower to locate the target, and made significantly more head- and eye-movements, as peripheral vision loss in the worse eye increased. A purely unilateral peripheral impairment increased response times by up to 25%, although the effect of bilateral vision loss was much greater (>200%). These findings indicate that even a degraded visual field still contributes important information for performing everyday visually-guided actions. This may have clinical implications for how patients with visual field loss are managed or prioritized, and for our understanding of how binocular information in the periphery is integrated

Learning to see and hear in 3D: Virtual reality as a platform for multisensory perceptual learning

Virtual reality (VR) is an emerging technology which allows for the presentation of immersive and realistic yet tightly controlled audiovisual scenes. In comparison to conventional displays, the VR system can include depth, 3D audio, fully integrated eye, head, and hand tracking, all over a much larger field of view than a desktop monitor provides. These properties demonstrate great potential for use in vision science experiments, especially those that can benefit from more naturalistic stimuli, particularly in the case of visual rehabilitation. Prior work using conventional displays has demonstrated that that visual loss due to stroke can be partially rehabilitated through laboratory-based tasks designed to promote long-lasting changes to visual sensitivity. In this work, I will explore how VR can provide a platform for new, more complex training paradigms which leverage multisensory stimuli. In this dissertation, I will (I) provide context to motivate the use of multisensory perceptual training in the context of visual rehabilitation, (II) demonstrate best practices for the appropriate use of VR in a controlled psychophysics setting, (III) describe a prototype integrated hardware system for improved eye tracking in VR, and (IV, V) discuss results from two audiovisual perceptual training studies, one using multisensory stimuli and the other with cross-modal audiovisual stimuli. This dissertation provides the foundation for future work in rehabilitating visual deficits, by both improving the hardware and software systems used to present the training paradigm as well as validating new techniques which use multisensory training not previously accessible with conventional desktop displays

Eye movements, search and perception of visual field defects in glaucoma

Glaucoma is a progressive disease of the optic nerve that can result in irreversible loss of visual function and impairment in everyday visual tasks. The experimental studies described in this thesis primarily aim to investigate the performance of people with glaucoma on search and other visual tasks whilst simultaneously monitoring eye movements, making comparison with age-related visually healthy people. In an experiment focussing on visual search, a patient group (n=30) took significantly longer on average to find a target in images of everyday scenes than controls (n=30). Furthermore, comparison of eye movements made by the participants during this task revealed there was a statistically significant reduction (6%) in saccade rate in the patients compared to the controls, and that saccade rate correlated with performance. Similar differences in eye movements were observed when the same groups passively viewed a selection of images in a slideshow. A bivariate contour ellipse (BCE) analysis revealed that, on average, patients viewed smaller regions of the images compared to the controls. Eye movement differences between patients and controls were also examined in a different cohort of people with glaucoma (n=14) and visually healthy controls (n=22) whilst they watched a selection of Hazard Perception Test driving films. Saccade rate of the patients was found to increase by 9%, though results from the BCE analysis suggested the average size of viewing area was similar in both groups. Finally, a novel interview-based study of 50 people with glaucoma provides evidence that patients do not perceive their visual field defect as a black ‘tunnel’ effect, or as ‘black patches’, but more like blurred regions: this finding may, for example, impact on how glaucomatous visual field loss is depicted in patient information about the condition. In conclusion, the results from this thesis show how visual loss from glaucoma influences how patients perceive and react to their visual environment. The principal findings from the studies described in this thesis also show, for the first time, that eye movement analysis could provide a window into the functional deficits associated with glaucoma