The Relative Importance of Depth Cues and Semantic Edges for Indoor Mobility Using Simulated Prosthetic Vision in Immersive Virtual Reality

Visual neuroprostheses (bionic eyes) have the potential to treat degenerative eye diseases that often result in low vision or complete blindness. These devices rely on an external camera to capture the visual scene, which is then translated frame-by-frame into an electrical stimulation pattern that is sent to the implant in the eye. To highlight more meaningful information in the scene, recent studies have tested the effectiveness of deep-learning based computer vision techniques, such as depth estimation to highlight nearby obstacles (DepthOnly mode) and semantic edge detection to outline important objects in the scene (EdgesOnly mode). However, nobody has attempted to combine the two, either by presenting them together (EdgesAndDepth) or by giving the user the ability to flexibly switch between them (EdgesOrDepth). Here, we used a neurobiologically inspired model of simulated prosthetic vision (SPV) in an immersive virtual reality (VR) environment to test the relative importance of semantic edges and relative depth cues to support the ability to avoid obstacles and identify objects. We found that participants were significantly better at avoiding obstacles using depth-based cues as opposed to relying on edge information alone, and that roughly half the participants preferred the flexibility to switch between modes (EdgesOrDepth). This study highlights the relative importance of depth cues for SPV mobility and is an important first step towards a visual neuroprosthesis that uses computer vision to improve a user's scene understanding

Towards Immersive Virtual Reality Simulations of Bionic Vision

Bionic vision is a rapidly advancing field aimed at developing visual neuroprostheses ('bionic eyes') to restore useful vision to people who are blind. However, a major outstanding challenge is predicting what people 'see' when they use their devices. The limited field of view of current devices necessitates head movements to scan the scene, which is difficult to simulate on a computer screen. In addition, many computational models of bionic vision lack biological realism. To address these challenges, we propose to embed biologically realistic models of simulated prosthetic vision (SPV) in immersive virtual reality (VR) so that sighted subjects can act as 'virtual patients' in real-world tasks.Comment: 3 pages, 2 figures, to be presented at Augmented Human

A model of ganglion axon pathways accounts for percepts elicited by retinal implants.

Degenerative retinal diseases such as retinitis pigmentosa and macular degeneration cause irreversible vision loss in more than 10 million people worldwide. Retinal prostheses, now implanted in over 250 patients worldwide, electrically stimulate surviving cells in order to evoke neuronal responses that are interpreted by the brain as visual percepts ('phosphenes'). However, instead of seeing focal spots of light, current implant users perceive highly distorted phosphenes that vary in shape both across subjects and electrodes. We characterized these distortions by asking users of the Argus retinal prosthesis system (Second Sight Medical Products Inc.) to draw electrically elicited percepts on a touchscreen. Using ophthalmic fundus imaging and computational modeling, we show that elicited percepts can be accurately predicted by the topographic organization of optic nerve fiber bundles in each subject's retina, successfully replicating visual percepts ranging from 'blobs' to oriented 'streaks' and 'wedges' depending on the retinal location of the stimulating electrode. This provides the first evidence that activation of passing axon fibers accounts for the rich repertoire of phosphene shape commonly reported in psychophysical experiments, which can severely distort the quality of the generated visual experience. Overall our findings argue for more detailed modeling of biological detail across neural engineering applications

Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses

Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli. Alternatively, deep learning models can optimize stimulus encoding strategies, but typically assume perfect knowledge of patient-specific variations. Here we propose a novel, practically feasible approach that overcomes both of these fundamental limitations. First, a deep encoder network is trained to produce optimal stimuli for any individual patient by inverting a forward model mapping electrical stimuli to visual percepts. Second, a preferential Bayesian optimization strategy utilizes this encoder to optimize patient-specific parameters for a new patient, using a minimal number of pairwise comparisons between candidate stimuli. We demonstrate the viability of this approach on a novel, state-of-the-art visual prosthesis model. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback and misspecifications in the underlying forward model. Overall, our results suggest that combining the strengths of deep learning and Bayesian optimization could significantly improve the perceptual experience of patients fitted with visual prostheses and may prove a viable solution for a range of neuroprosthetic technologies

A Systematic Review of Extended Reality (XR) for Understanding and Augmenting Vision Loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on augmentation of a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the last decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the suitability and usability of different XR-based accessibility aids. By broadening end-user participation to early stages of the design process and shifting the focus from behavioral performance to qualitative assessments of usability, future research has the potential to develop XR technologies that may not only allow for studying vision loss, but also enable novel visual accessibility aids with the potential to impact the lives of millions of people living with vision loss

Retrospective adjustment of self-assessed medical competencies – noteworthy in the evaluation of postgraduate practical training courses

Aim: The efficacy of postgraduate practical training courses is frequently evaluated by self-assessment instruments. The present study analyses the effect of a basic course in laparoscopic surgery on self-assessed medical competencies

Modeling Peste des Petits Ruminants (PPR) Disease Propagation and Control Strategies Using Memoryless State Transitions

Peste des Petits Ruminants (PPR) is an infectious disease affecting goats and sheep. PPR has a mortality rate of 80% and a morbidity rate of 100% in naïve herds. This disease is currently of concern to Afghani goat and sheep herders as conditions in Afghanistan are conducive to the disease becoming an epidemic. PPR is similar to Rinderpest, but is not as well studied. There is a lack of empirical data on how the disease spreads or effective large-scale mitigation strategies. We developed a herd-level, event-driven model of PPR, using memoryless state transitions, to study how the virus propagates through a herd, and to identify effective control strategies for disparate herd configurations and environments. This model allows us to perform Sensitivity Analyses (SA) on environmental and disease parameters for which we do not have empirical data and to simulate the effectiveness of various control strategies. We find that reducing the amount of time from the identification of PPR in a herd to the vaccination of the herd will radically reduce the number of deaths that result from PPR. The goal of this model is to give policy makers a tool to develop effective containment strategies for managing outbreaks of PPR

Managing Bay and Estuarine Ecosystems for Multiple Services

Abstract Managers are moving from a model of managing individual sectors, human activities, or ecosystem services to an ecosystem-based management (EBM) approach which attempts to balance the range of services provided by ecosystems. Applying EBM is often difficult due to inherent tradeoffs in managing for different services. This challenge particularly holds for estuarine systems, which have been heavily altered in most regions and are often subject to intense management interventions. Estuarine managers can often choose among a range of management tactics to enhance a particular service; although some management actions will result in strong tradeoffs, others may enhance multiple services simultaneously. Management of estuarine ecosystems could be improved by distinguishing between optimal management actions for enhancing multiple services and those that have severe tradeoffs. This requires a framework that evaluates tradeoff scenarios and identifies management actions likely to benefit multiple services. We created a management action-services matrix as a first step towards assessing tradeoffs and providing managers with a DOI 10.1007/s12237-013-9602-7 decision support tool. We found that management actions that restored or enhanced natural vegetation (e.g., salt marsh and mangroves) and some shellfish (particularly oysters and oyster reef habitat) benefited multiple services. In contrast, management actions such as desalination, salt pond creation, sand mining, and large container shipping had large net negative effects on several of the other services considered in the matrix. Our framework provides resource managers a simple way to inform EBM decisions and can also be used as a first step in more sophisticated approaches that model service delivery