Egocentric Computer Vision and Machine Learning for Simulated Prosthetic Vision

Las prótesis visuales actuales son capaces de proporcionar percepción visual a personas con cierta ceguera. Sin pasar por la parte dañada del camino visual, la estimulación eléctrica en la retina o en el sistema nervioso provoca percepciones puntuales conocidas como “fosfenos”. Debido a limitaciones fisiológicas y tecnológicas, la información que reciben los pacientes tiene una resolución muy baja y un campo de visión y rango dinámico reducido afectando seriamente la capacidad de la persona para reconocer y navegar en entornos desconocidos. En este contexto, la inclusión de nuevas técnicas de visión por computador es un tema clave activo y abierto. En esta tesis nos centramos especialmente en el problema de desarrollar técnicas para potenciar la información visual que recibe el paciente implantado y proponemos diferentes sistemas de visión protésica simulada para la experimentación.Primero, hemos combinado la salida de dos redes neuronales convolucionales para detectar bordes informativos estructurales y siluetas de objetos. Demostramos cómo se pueden reconocer rápidamente diferentes escenas y objetos incluso en las condiciones restringidas de la visión protésica. Nuestro método es muy adecuado para la comprensión de escenas de interiores comparado con los métodos tradicionales de procesamiento de imágenes utilizados en prótesis visuales.Segundo, presentamos un nuevo sistema de realidad virtual para entornos de visión protésica simulada más realistas usando escenas panorámicas, lo que nos permite estudiar sistemáticamente el rendimiento de la búsqueda y reconocimiento de objetos. Las escenas panorámicas permiten que los sujetos se sientan inmersos en la escena al percibir la escena completa (360 grados).En la tercera contribución demostramos cómo un sistema de navegación de realidad aumentada para visión protésica ayuda al rendimiento de la navegación al reducir el tiempo y la distancia para alcanzar los objetivos, incluso reduciendo significativamente el número de colisiones de obstáculos. Mediante el uso de un algoritmo de planificación de ruta, el sistema encamina al sujeto a través de una ruta más corta y sin obstáculos. Este trabajo está actualmente bajo revisión.En la cuarta contribución, evaluamos la agudeza visual midiendo la influencia del campo de visión con respecto a la resolución espacial en prótesis visuales a través de una pantalla montada en la cabeza. Para ello, usamos la visión protésica simulada en un entorno de realidad virtual para simular la experiencia de la vida real al usar una prótesis de retina. Este trabajo está actualmente bajo revisión.Finalmente, proponemos un modelo de Spiking Neural Network (SNN) que se basa en mecanismos biológicamente plausibles y utiliza un esquema de aprendizaje no supervisado para obtener mejores algoritmos computacionales y mejorar el rendimiento de las prótesis visuales actuales. El modelo SNN propuesto puede hacer uso de la señal de muestreo descendente de la unidad de procesamiento de información de las prótesis retinianas sin pasar por el análisis de imágenes retinianas, proporcionando información útil a los ciegos. Esté trabajo está actualmente en preparación.<br /

Large scale retinal modeling for the design of new generation retinal prostheses

With the help of modern technology, blindness caused by retinal diseases such as age-related macular degeneration or retinitis pigmentosa is now considered reversible. Scientists from various fields such as Neuroscience, Electrical Engineering, Computer Science, and Bioscience have been collaborating to design and develop retinal prostheses, with the aim of replacing malfunctioning parts of the retina and restoring vision in the blind. Human trials conducted to test retinal prostheses have yielded encouraging results, showing the potential of this approach in vision recovery. However, a retinal prosthesis has several limitations with regard to its hardware and biological functions, and several attempts have been made to overcome these limitations. This thesis focuses on the biological aspects of retinal prostheses: the biological processes occurring inside the retina and the limitations of retinal prostheses corresponding to those processes have been analysed. Based on these analyses, three major findings regarding information processing inside the retina have been presented and these findings have been used to conceptualise retinal prostheses that have the characteristics of asymmetrical and separate pathway stimulations. In the future, when nanotechnology gains more popularity and is completely integrated inside the prosthesis, this concept can be utilized to restore useful visual information such as colour, depth, and contrast to achieve high-quality vision in the blind

Information transmission in normal vision and optogenetically resensitised dystrophic retinas

Phd ThesisThe retina is a sophisticated image processing machine, transforming the visual scene as detected by the photoreceptors into a pattern of action potentials that is sent to the brain by the retinal ganglion cells (RGCs), where it is further processed to help us understand and navigate the world. Understanding this encoding process is important on a number of levels. First, it informs the study of upstream visual processing by elucidating the signals higher visual areas receive as input and how they relate to the outside world. Second, it is important for the development of treatments for retinal blindness, such as retinal prosthetics. In this thesis, I present work using multielectrode array (MEA) recordings of RGC populations from ex-vivo retinal wholemounts to study various aspects of retinal information processing. My results fall into two main themes. In the rst part, in collaboration with Dr Geo rey Portelli and Dr Pierre Kornprobst of INRIA, I use ashed gratings of varying spatial frequency and phase to compare di erent coding strategies that the retina might use. These results show that information is encoded synergistically by pairs of neurons and that, of the codes tested, a Rank Order Code based on the relative order of ring of the rst spikes of a population of neurons following a stimulus provides information about the stimulus faster and more e ciently than other codes. In the later parts, I use optogenetic stimulation of RGCs in congenitally blind retinas to study how visual information is corrupted by the spontaneous hyperactivity that arises as a result of photoreceptor degeneration. I show that by dampening this activity with the gap junction blocker meclofenamic acid, I can improve the signal-to-noise ratio, spatial acuity and contrast sensitivity of prosthetically evoked responses. Taken together, this work provides important insights for the future development of retinal prostheses

Elicitation of retinal neural circuitry with vision prosthetic devices

Vision prostheses currently under development by several research groups aim to restore functional sight to the profoundly blind suffering from retinal neural degenerative diseases. Human clinical trials in the last decade have demonstrated the ability of these devices to elicit simple percepts, such as bright spots of light. However, further improvements in implant perceptual efficacy will critically depend on improved understanding of the retinal neural mechanisms underlying the electrically evoked responses, and on how these mechanisms could be controlled artificially. In the first part of this thesis I quantitatively study, using a new statistical analysis technique, the temporal response properties of the retinal ganglion cells (RGCs) following electrical stimulation of the retina. I also demonstrate conclusively, for the first time, that small electrodes placed in the subretinal space could reliably elicit direct RGC spiking responses. In the second part of the thesis I investigate the mechanisms underlying the previously observed RGC response depression during repeated electrical stimulation of these cells. The experimental findings lead me to the development of a new stimulation method for preventing the response depression. The image processor is a crucial component of a vision prosthesis. It replaces some of the neural computations that occur in a healthy retina by converting visual stimuli into electrical stimuli. In the final part of the thesis I implement an image processor for a vision prosthesis. I show that such devices could be built with appropriate embedded hardware. Benchmark testing suggests that, depending on the complexity of the image processing strategies, care should be exercised in generalising the performance of algorithms developed on standard computers to these embedded devices

Electrochemical Safety Studies of Cochlear Implant Electrodes Using the Finite Element Method

Cochlear implants, amongst other neural prostheses, utilise platinum electrodes as an interface between the synthetic implant and the biological tissue environment. If excessive electrical charge is injected via these electrodes, injury to the tissue may result. Empirically derived stimulation limits have been defined to prevent tissue damage, however the injurious mechanisms are still unclear. Evidence suggests that the non-uniform distribution of charge on electrodes influences the electrochemical generation of toxic by-products. However, in vivo and in vitro techniques are limited in their ability to systematically explore the factors and mechanisms that contribute to stimulation-induced tissue injury. To this end, an in silico approach was used to develop a time-domain model of cochlear implant stimulation electrodes. A constant phase angle impedance was used to model the reversible processes on the electrode surface, and Butler-Volmer reaction kinetics were used to define the behaviour of the water window irreversible electrochemical reactions. The resulting model provided time-domain responses of the current density distributions, and net charge consumed by the hydrolysis reactions. This model was then used to perform systematic evaluations of various electrode geometries and stimulation parameters. The modelling results showed the current associated with irreversible reactions was non-uniform and tended towards the periphery of the electrode. A comparison of electrode geometries revealed interactions between electrode size, shape and recess depth. Stimulation mode, electrode position, and electrolyte conductivity were found to impact the shape of the electric field and the extent of irreversible reactions. This emphasised the influence of the physiological environment on the stimulation safety. In vitro experiments were conducted to validate the model. The implications of the results described in this thesis can be used to inform the design of safer electrodes