In Vivo Photovoltaic Performance of a Silicon Nanowire Photodiode-Based Retinal Prosthesis.

Purpose:For more than 20 years, there has been an international, multidisciplinary effort to develop retinal prostheses to restore functional vision to patients blinded by retinal degeneration. We developed a novel subretinal prosthesis with 1512 optically addressed silicon nanowire photodiodes, which transduce incident light into an electrical stimulation of the remaining retinal circuitry. This study was conducted to evaluate the efficacy of optically driving the subretinal prosthesis to produce visual cortex activation via electrical stimulation of the retina. Methods:We measured electrically evoked potential responses (EEPs) in rabbit visual cortex in response to illumination of the subretinal nanowire prosthesis with pulsed 852-nm infrared (IR) light. We compared the EEP responses to visually evoked potential responses (VEPs) to pulsed 532-nm visible light (positive control) and pulsed 852-nm IR light (negative control). Results:Activating the devices with IR light produced EEP responses with a significantly higher trough-to-peak amplitude (54.17 ± 33.4 μV) than IR light alone (24.07 ± 22.1 μV) or background cortical activity (23.22 ± 17.2 μV). EEP latencies were significantly faster than focal VEP latencies. Focal VEPs produced significantly higher amplitudes (94.88 ± 43.3 μV) than EEPs. We also demonstrated how an electrode placed on the cornea can be used as a noninvasive method to monitor the function of the implant. Conclusions:These results show that subretinal electrical stimulation with nanowire electrodes can elicit EEPs in the visual cortex, providing evidence for the viability of a subretinal nanowire prosthetic approach for vision restoration

The Argus II Retinal Prosthesis System

The field of retinal prosthetics has seen significant advances in the past 3 decades. Encouraging results from different groups have shown coarse objective functional improvement, using a range of technological and surgical approaches. The Argus II retinal prosthesis system was the first of its kind to receive regulatory approval for commercial use in Europe and the USA. The device is designed to replicate the function of photoreceptors in converting visual information into electrical neural signals in patients with profound visual loss secondary to degenerative retinal disease. Results from a phase II study of 30 patients have demonstrated improved performance in basic tests of visual function, object recognition, letter reading, prehension, orientation and mobility tasks. It is now the most widely implanted retinal prosthetic device worldwide. This chapter provides an overview of the requirements of a retinal prosthetic system, the results from the Argus II device to date, and an insight into some of the challenges and future directions of visually restorative therapies

A Systematic Review of Extended Reality (XR) for Understanding and Augmenting Vision Loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on augmentation of a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the last decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the suitability and usability of different XR-based accessibility aids. By broadening end-user participation to early stages of the design process and shifting the focus from behavioral performance to qualitative assessments of usability, future research has the potential to develop XR technologies that may not only allow for studying vision loss, but also enable novel visual accessibility aids with the potential to impact the lives of millions of people living with vision loss

Viia-hand: a Reach-and-grasp Restoration System Integrating Voice interaction, Computer vision and Auditory feedback for Blind Amputees

Visual feedback plays a crucial role in the process of amputation patients completing grasping in the field of prosthesis control. However, for blind and visually impaired (BVI) amputees, the loss of both visual and grasping abilities makes the "easy" reach-and-grasp task a feasible challenge. In this paper, we propose a novel multi-sensory prosthesis system helping BVI amputees with sensing, navigation and grasp operations. It combines modules of voice interaction, environmental perception, grasp guidance, collaborative control, and auditory/tactile feedback. In particular, the voice interaction module receives user instructions and invokes other functional modules according to the instructions. The environmental perception and grasp guidance module obtains environmental information through computer vision, and feedbacks the information to the user through auditory feedback modules (voice prompts and spatial sound sources) and tactile feedback modules (vibration stimulation). The prosthesis collaborative control module obtains the context information of the grasp guidance process and completes the collaborative control of grasp gestures and wrist angles of prosthesis in conjunction with the user's control intention in order to achieve stable grasp of various objects. This paper details a prototyping design (named viia-hand) and presents its preliminary experimental verification on healthy subjects completing specific reach-and-grasp tasks. Our results showed that, with the help of our new design, the subjects were able to achieve a precise reach and reliable grasp of the target objects in a relatively cluttered environment. Additionally, the system is extremely user-friendly, as users can quickly adapt to it with minimal training

Egocentric Computer Vision and Machine Learning for Simulated Prosthetic Vision

Las prótesis visuales actuales son capaces de proporcionar percepción visual a personas con cierta ceguera. Sin pasar por la parte dañada del camino visual, la estimulación eléctrica en la retina o en el sistema nervioso provoca percepciones puntuales conocidas como “fosfenos”. Debido a limitaciones fisiológicas y tecnológicas, la información que reciben los pacientes tiene una resolución muy baja y un campo de visión y rango dinámico reducido afectando seriamente la capacidad de la persona para reconocer y navegar en entornos desconocidos. En este contexto, la inclusión de nuevas técnicas de visión por computador es un tema clave activo y abierto. En esta tesis nos centramos especialmente en el problema de desarrollar técnicas para potenciar la información visual que recibe el paciente implantado y proponemos diferentes sistemas de visión protésica simulada para la experimentación.Primero, hemos combinado la salida de dos redes neuronales convolucionales para detectar bordes informativos estructurales y siluetas de objetos. Demostramos cómo se pueden reconocer rápidamente diferentes escenas y objetos incluso en las condiciones restringidas de la visión protésica. Nuestro método es muy adecuado para la comprensión de escenas de interiores comparado con los métodos tradicionales de procesamiento de imágenes utilizados en prótesis visuales.Segundo, presentamos un nuevo sistema de realidad virtual para entornos de visión protésica simulada más realistas usando escenas panorámicas, lo que nos permite estudiar sistemáticamente el rendimiento de la búsqueda y reconocimiento de objetos. Las escenas panorámicas permiten que los sujetos se sientan inmersos en la escena al percibir la escena completa (360 grados).En la tercera contribución demostramos cómo un sistema de navegación de realidad aumentada para visión protésica ayuda al rendimiento de la navegación al reducir el tiempo y la distancia para alcanzar los objetivos, incluso reduciendo significativamente el número de colisiones de obstáculos. Mediante el uso de un algoritmo de planificación de ruta, el sistema encamina al sujeto a través de una ruta más corta y sin obstáculos. Este trabajo está actualmente bajo revisión.En la cuarta contribución, evaluamos la agudeza visual midiendo la influencia del campo de visión con respecto a la resolución espacial en prótesis visuales a través de una pantalla montada en la cabeza. Para ello, usamos la visión protésica simulada en un entorno de realidad virtual para simular la experiencia de la vida real al usar una prótesis de retina. Este trabajo está actualmente bajo revisión.Finalmente, proponemos un modelo de Spiking Neural Network (SNN) que se basa en mecanismos biológicamente plausibles y utiliza un esquema de aprendizaje no supervisado para obtener mejores algoritmos computacionales y mejorar el rendimiento de las prótesis visuales actuales. El modelo SNN propuesto puede hacer uso de la señal de muestreo descendente de la unidad de procesamiento de información de las prótesis retinianas sin pasar por el análisis de imágenes retinianas, proporcionando información útil a los ciegos. Esté trabajo está actualmente en preparación.<br /

An update on retinal prostheses

Retinal prostheses are designed to restore a basic sense of sight to people with profound vision loss. They require a relatively intact posterior visual pathway (optic nerve, lateral geniculate nucleus and visual cortex). Retinal implants are options for people with severe stages of retinal degenerative disease such as retinitis pigmentosa and age-related macular degeneration. There have now been three regulatory-approved retinal prostheses. Over five hundred patients have been implanted globally over the past 15 years. Devices generally provide an improved ability to localize high-contrast objects, navigate, and perform basic orientation tasks. Adverse events have included conjunctival erosion, retinal detachment, loss of light perception, and the need for revision surgery, but are rare. There are also specific device risks, including overstimulation (which could cause damage to the retina) or delamination of implanted components, but these are very unlikely. Current challenges include how to improve visual acuity, enlarge the field-of-view, and reduce a complex visual scene to its most salient components through image processing. This review encompasses the work of over 40 individual research groups who have built devices, developed stimulation strategies, or investigated the basic physiology underpinning retinal prostheses. Current technologies are summarized, along with future challenges that face the field

An efficient telemetry system for restoring sight

PhD ThesisThe human nervous system can be damaged as a result of disease or trauma, causing conditions such as Parkinson’s disease. Most people try pharmaceuticals as a primary method of treatment. However, drugs cannot restore some cases, such as visual disorder. Alternatively, this impairment can be treated with electronic neural prostheses. A retinal prosthesis is an example of that for restoring sight, but it is not efficient and only people with retinal pigmentosa benefit from it. In such treatments, stimulation of the nervous system can be achieved by electrical or optical means. In the latter case, the nerves need to be rendered light sensitive via genetic means (optogenetics). High radiance photonic devices are then required to deliver light to the target tissue. Such optical approaches hold the potential to be more effective while causing less harm to the brain tissue. As these devices are implanted in tissue, wireless means need to be used to communicate with them. For this, IEEE 802.15.6 or Bluetooth protocols at 2.4GHz are potentially compatible with most advanced electronic devices, and are also safe and secure. Also, wireless power delivery can operate the implanted device. In this thesis, a fully wireless and efficient visual cortical stimulator was designed to restore the sight of the blind. This system is likely to address 40% of the causes of blindness. In general, the system can be divided into two parts, hardware and software. Hardware parts include a wireless power transfer design, the communication device, power management, a processor and the control unit, and the 3D design for assembly. The software part contains the image simplification, image compression, data encoding, pulse modulation, and the control system. Real-time video streaming is processed and sent over Bluetooth, and data are received by the LPC4330 six layer implanted board. After retrieving the compressed data, the processed data are again sent to the implanted electrode/optrode to stimulate the brain’s nerve cells

Interactions of prosthetic and natural vision in animals with local retinal degeneration

Prosthetic restoration of partial sensory loss leads to interactions between artificial and natural inputs. Ideally, the rehabilitation should allow perceptual fusion of the two modalities. Here we studied the interactions between normal and prosthetic vision in a rodent model of local retinal degeneration.  Implantation of a photovoltaic array in the subretinal space of normally sighted rats induced local degeneration of the photoreceptors above the chip, and the inner retinal neurons in this area were electrically stimulated by the photovoltaic implant powered by near-infrared (NIR) light. We studied prosthetic and natural visually evoked potentials (VEP) in response to simultaneous stimulation by NIR and visible light patterns.  We demonstrate that electrical and natural VEPs summed linearly in the visual cortex, and both responses decreased under brighter ambient light. Responses to visible light flashes increased over 3 orders of magnitude of contrast (flash/background), while for electrical stimulation the contrast range was limited to 1 order of magnitude. The maximum amplitude of the prosthetic VEP was three times lower than the maximum response to a visible flash over the same area on the retina.  Ambient light affects prosthetic responses, albeit much less than responses to visible stimuli. Prosthetic representation of contrast in the visual scene can be encoded, to a limited extent, by the appropriately calibrated stimulus intensity, which also depends on the ambient light conditions. Such calibration will be important for patients combining central prosthetic vision with natural peripheral sight, such as in age-related macular degeneration