Visual Acuity of Simulated Thalamic Visual Prostheses in Normally Sighted Humans

Simulation in normally sighted individuals is a crucial tool to evaluate the performance of potential visual prosthesis designs prior to human implantation of a device. Here, we investigated the effects of electrode count on visual acuity, learning rate and response time in 16 normally sighted subjects using a simulated thalamic visual prosthesis, providing the first performance reports for thalamic designs. A new letter recognition paradigm using a multiple-optotype two-alternative forced choice task was adapted from the Snellen eye chart, and specifically devised to be readily communicated to both human and non-human primate subjects. Validation of the method against a standard Snellen acuity test in 21 human subjects showed no significant differences between the two tests. The novel task was then used to address three questions about simulations of the center-weighted phosphene patterns typical of thalamic designs: What are the expected Snellen acuities for devices with varying numbers of contacts, do subjects display rapid adaptation to the new visual modality, and can response time in the task provide clues to the mechanisms of perception in low-resolution artificial vision? Population performance (hit rate) was significantly above chance when viewing Snellen 20/200 optotypes (Log MAR 1.0) with 370 phosphenes in the central 10 degrees of vision, ranging to Snellen 20/800 (Log MAR 1.6) with 25 central phosphenes. Furthermore, subjects demonstrated learning within the 1–2 hours of task experience indicating the potential for an effective rehabilitation and possibly better visual performance after a longer period of training. Response time differences suggest that direct letter perception occurred when hit rate was above 75%, whereas a slower strategy like feature-based pattern matching was used in conditions of lower relative resolution. As pattern matching can substantially boost effective acuity, these results suggest post-implant therapy should specifically address feature detection skills

Simulation of thalamic prosthetic vision: reading accuracy, speed, and acuity in sighted humans

The psychophysics of reading with artificial sight has received increasing attention as visual prostheses are becoming a real possibility to restore useful function to the blind through the coarse, pseudo-pixelized vision they generate. Studies to date have focused on simulating retinal and cortical prostheses; here we extend that work to report on thalamic designs. This study examined the reading performance of normally sighted human subjects using a simulation of three thalamic visual prostheses that varied in phosphene count, to help understand the level of functional ability afforded by thalamic designs in a task of daily living. Reading accuracy, reading speed, and reading acuity of 20 subjects were measured as a function of letter size, using a task based on the MNREAD chart. Results showed that fluid reading was feasible with appropriate combinations of letter size and phosphene count, and performance degraded smoothly as font size was decreased, with an approximate doubling of phosphene count resulting in an increase of 0.2 logMAR in acuity. Results here were consistent with previous results from our laboratory. Results were also consistent with those from the literature, despite using naive subjects who were not trained on the simulator, in contrast to other reports

Central nervous system microstimulation: Towards selective micro-neuromodulation

Electrical stimulation technologies capable of modulating neural activity are well established for neuroscientific research and neurotherapeutics. Recent micro-neuromodulation experimental results continue to explain neural processing complexity and suggest the potential for assistive technologies capable of restoring or repairing of basic function. Nonetheless, performance is dependent upon the specificity of the stimulation. Increasingly specific stimulation is hypothesized to be achieved by progressively smaller interfaces. Miniaturization is a current focus of neural implants due to improvements in mitigation of the body's foreign body response. It is likely that these exciting technologies will offer the promise to provide large-scale micro-neuromodulation in the future. Here, we highlight recent successes of assistive technologies through bidirectional neuroprostheses currently being used to repair or restore basic brain functionality. Furthermore, we introduce recent neuromodulation technologies that might improve the effectiveness of these neuroprosthetic interfaces by increasing their chronic stability and microstimulation specificity. We suggest a vision where the natural progression of innovative technologies and scientific knowledge enables the ability to selectively micro-neuromodulate every neuron in the brain

Eye Movement Compensation and Spatial Updating in Visual Prosthetics: Mechanisms, Limitations and Future Directions

Despite appearing automatic and effortless, perceiving the visual world is a highly complex process that depends on intact visual and oculomotor function. Understanding the mechanisms underlying spatial updating (i.e., gaze contingency) represents an important, yet unresolved issue in the fields of visual perception and cognitive neuroscience. Many questions regarding the processes involved in updating visual information as a function of the movements of the eyes are still open for research. Beyond its importance for basic research, gaze contingency represents a challenge for visual prosthetics as well. While most artificial vision studies acknowledge its importance in providing accurate visual percepts to the blind implanted patients, the majority of the current devices do not compensate for gaze position. To-date, artificial percepts to the blind population have been provided either by intraocular light-sensing circuitry or by using external cameras. While the former commonly accounts for gaze shifts, the latter requires the use of eye-tracking or similar technology in order to deliver percepts based on gaze position. Inspired by the need to overcome the hurdle of gaze contingency in artificial vision, we aim to provide a thorough overview of the research addressing the neural underpinnings of eye compensation, as well as its relevance in visual prosthetics. The present review outlines what is currently known about the mechanisms underlying spatial updating and reviews the attempts of current visual prosthetic devices to overcome the hurdle of gaze contingency. We discuss the limitations of the current devices and highlight the need to use eye-tracking methodology in order to introduce gaze-contingent information to visual prosthetics

The brain as image processor and generator:towards function-restoring brain-computer-interfaces

As neuroscientists are slowly unraveling the mysteries of the brain, neurotechnology like brain-computer-interfaces (BCIs) might become a new standard for medical applications in those with brain injuries. BCIs allow for direct communication between the brain and a device, and could potentially restore links that are broken due to brain damage. In addition, a better understanding of the human mind and its mechanisms could greatly boost the success of these devices. This dissertation features (high-field) functional magnetic resonance imaging (fMRI) to study human cognitive functioning, as fMRI allows for studying the brain of living humans in great spatial detail. Firstly, the dissertation describes how well brain regions that are important for visual perception can be located between individuals. Some of these regions are in part responsible for recognizing objects like faces, bodies, places and motion. Secondly, differences in functional organization of the brain were explored between individuals by simulating the placement of a visual cortical prosthesis. Such a prosthesis can bypass the (broken) connections between the eye and brain in blind people, and potentially restore a rudimentary form of vision. Finally, new techniques were presented that show that visual perception and mental imagery are closely related, and allow for reading letter shapes directly from the mind. Together, this dissertation adds new foundations for the development of neurotechnological applications

Látásjavító implantátumok látóhártya-degenerációkban = Vision restoration with implants in retinal degenerations

Az ideghártya fényérzékelő sejtjeinek maradandó károsodásával járó és vaksághoz vezető betegségek eddig gyógyíthatatlannak bizonyultak. Jelenleg a szembe ültethető retinaimplantátumok fejlesztése biztat leghamarabb a klinikai gyakorlatba is bevezethető eredménnyel e betegek számára. A közlemény célja az eltérő működési elv alapján csoportosított, különböző fejlesztési szakaszban levő implantátumokkal kapcsolatos kutatások ismertetése és jellemzőinek kiemelése, valamint a fejlesztések hazai vonatkozásainak bemutatása. Az összefoglaló a nemzetközi szakirodalomban megjelent publikációk áttekintésével és feldolgozásával, valamint személyes tapasztalatok alapján kíván áttekintést nyújtani a retina degeneratív betegségei esetén beültethető retinaimplantátumokról. Az elmúlt évek mikroelektronikai fejlesztései tették lehetővé, hogy a retina elpusztult fotoreceptorainak helyettesítése elektromos ingerléssel sikeresen megoldható legyen. Több egymástól mind felépítésében, mind egyéb tulajdonságaiban jelentősen eltérő implantátum fejlesztése folyik jelenleg is egymással párhuzamosan. Ezek közül két, az ideghártyával közvetlen kapcsolatban álló, a szemgolyóba ültethető rendszer emelkedik ki. Az ideghártya alá ültethető, subretinalis típusú implantátumokkal sikerült eddig a legfinomabb felbontást elérni. Az ideghártya felszínére rögzített implantátumnak ugyan csekélyebb a felbontása, de rövidebb műtétet igényel a beültetése. A retinaimplantátumok segítségével egyes ideghártya-betegségekben immár bizonyított, hogy látásszerű élmény váltható ki. A multicentrikus klinikai vizsgálatok lezárását követően néhány éven belül várható, hogy többfajta implantátumtípus is megjelenik a klinikai gyakorlatban. Orv. Hetil., 2011, 152, 537–545. | Up until now there has been no available treatment for diseases causing the permanent impairment of retinal photoreceptors. Currently the development of the retinal prostheses is the earliest to promise a result that can be implemented in the clinical treatment of these patients. Implants with different operating principles and in various stages of progress are presented in details, highlighting the characteristics, as well as the Hungarian aspects of the development. This survey intends to provide an overview on retinal prostheses, implantable in case of degenerative diseases of the retina, by reviewing and assessing the papers published in relevant journals and based on personal experience. Developments in microelectronics in recent years made it possible and proved to be feasible to replace the degenerated elements in the retina with electrical stimulation. Multiple comparable approaches are running simultaneously. Two types of these implants are directly stimulating the remaining living cells in the retina. Hitherto the finest resolution has been achieved with the subretinal implants. Although the epiretinal implant offer lower resolution, but requires shorter surgery for implantation. Retinal implants in certain retinal diseases are proved to be capable of generating vision-like experiences. A number of types of retinal implants can be expected to appear in clinical practice a few years after the successful conclusion of clinical trials. Orv. Hetil., 2011, 152, 537–545

Perceptual learning in a non-human primate model of artificial vision

Visual perceptual grouping, the process of forming global percepts from discrete elements, is experience-dependent. Here we show that the learning time course in an animal model of artificial vision is predicted primarily from the density of visual elements. Three naïve adult non-human primates were tasked with recognizing the letters of the Roman alphabet presented at variable size and visualized through patterns of discrete visual elements, specifically, simulated phosphenes mimicking a thalamic visual prosthesis. The animals viewed a spatially static letter using a gaze-contingent pattern and then chose, by gaze fixation, between a matching letter and a non-matching distractor. Months of learning were required for the animals to recognize letters using simulated phosphene vision. Learning rates increased in proportion to the mean density of the phosphenes in each pattern. Furthermore, skill acquisition transferred from trained to untrained patterns, not depending on the precise retinal layout of the simulated phosphenes. Taken together, the findings suggest that learning of perceptual grouping in a gaze-contingent visual prosthesis can be described simply by the density of visual activation

Argus® II Retinal Prosthesis System: Clinical & Functional Outcomes

Developing artificial visual systems to restore sight in blind patients has long been the dream of scientists, clinicians and the public at large. After decades of research, the greatest success in the field has been achieved with electronic retinal prostheses. To date, 3 retinal prosthetic systems have made the transition from laboratory / clinical research to entering the commercial market for clinical use, namely the Argus® II Retinal Prosthesis System (Second Sight), the alpha-IMS system (Retinal Implant AG), and the IRIS® II (Pixium Vision). The following body of work describes the Argus® II Retinal Prosthesis system, which obtained regulatory approval in the European Economic Area in 2011 (CE marking) and later on in the USA (FDA approval in February 2013), based on the results of an international multi-centre clinical feasibility trial (Clinical Trial identifier: NCT 00407602). This thesis aims to examine the long-term clinical and functional outcomes in an early cohort of subjects chronically implanted with the Argus® II system, from the original feasibility study. A further aim is to elucidate the characteristics of the artificial vision that is perceived and its long-term repeatability and reproducibility in individual subjects. These two broad aims will assist in understanding the nature of the visual performance provided by this device, as well as to add to the current data that is defining the feasibility of constructing predictable pixelated patterns to achieve useful artificial vision in the future. Finally, we explored the feasibility of real-time imaging of visual cortex activation in response to electrical retinal stimulation with the Argus® II system, using functional near infra-red spectroscopy (fNIRS). Development of this real-time imaging tool will enable future investigations into the differences in the cortical activities in response to different stimulations and in different subjects. This may in turn help us understand the variability in their visual performance, as well as to further explore the extent and effect of cross-modal plasticity at the cortical level, in this cohort of patients who have been deprived of visual inputs for decades. Visual function was assessed in terms of: a) form recognition and b) spatial localisation under both 2-dimensional (2D) screen-based laboratory settings and 3-dimensional (3D) paradigms simulating real-life settings. A prospective study of 11 Argus® II subjects showed that the subjects could identify distinct geometric shapes presented in high contrast better with the prosthetic system switched on (median % of correct identification = 20.0%, IQR = 18.8), versus off (median = 12.5%, IQR = 5.0). The accuracy of shapes identification could be further improved by enhancing the outlines of the geometric shape (median = 33.1%, IQR = 21.6). A further prospective study from a subset of 7 subjects showed that this 2D shape identification could be translated into improved identification of 3D objects. These subjects could identify 8 common daily-life objects presented in high contrast with the prosthetic system switched on (median = 31.3%, IQR = 20.3) versus off (median = 12.5%, IQR = 12.5). Scrambling of the transmission signals within the prosthetic system in order to separate light information from form information (i.e. “scrambled mode”) hindered the identification in some but not all subjects (median = 25.0%, IQR = 12.5). The accuracy of object identification could also be improved by enhancing the edges of objects (median = 43.8%, IQR = 15.6). Previously published data showed that Argus® II subjects were able to locate and point to white squares presented on touch screens against a black background more accurately with the prosthetic system switched on versus off. We demonstrated with a prospective study of 5 subjects that they could localise an object on the table, reach out and grasp the object (prehension) with great accuracy (66.7 – 100%) when the prosthetic system was switched on, versus no object prehension (0%) with the system switched off. A prospective study of 6 Argus® II subjects illustrated that while there was a wide variation in the shape and size of the phosphenes perceived by individual subjects, the elicited phosphenes were consistently reproducible in each subject using fixed stimulating parameters, with inter-stimuli intervals ranging from 20 minutes apart, down to 1 second. The perceived location of the phosphenes grossly matched retinotopic agreement, with 4 subjects drawing phosphenes in the same visual field quadrant as predicted by the relative stimulus-fovea position, and 2 subjects depicting phosphenes in the same hemi-field as the expected locations. A retrospective study of 3 Argus® II subjects who underwent MRI brain scan (for unrelated medical reasons) showed that MRI brain scans of up to 1.5 Tesla field strength appeared to have no detrimental effect on the subjects and their implant function. The Argus® II implant produced an artefact of around 50mm x 50mm in size which would prevent visualisation of structures within the orbit, but visualisation of surrounding tissues outside this areas are unaffected. The use of functional MRI as a tool of exploring visual cortex activation in Argus® II subjects was discounted, due to concerns of signal interference from the radiofrequency telemetry of Argus® II system with that of MRI. Subsequently, we have demonstrated in a prospective study that an alternative neuro-imaging technique, functional near infra-red spectroscopy (fNIRS), was capable of capturing real-time cortical activation in 5 out of 6 Argus® II subjects, and maybe a feasible tool for future investigation into cortical function and interactions. The work in this thesis has shown that the Argus® II retinal prosthesis system could improve visual function both in terms of form recognition, as well as object localisation in 3D in situations simulating real-life settings, in a cohort of patients with end-stage retinitis pigmentosa or other outer retinal diseases such as choroideremia. The wide variation in the visual performance level observed could in part be attributable to the diversity in the phosphene features perceived by these subjects. Nevertheless, the consistency and reproducibility with which these phosphenes could be elicited, with fixed stimulating parameters within each subject, provides an encouraging basis for the construction of more complicated pixelated images. Future work to determine the underlying factors influencing the perceived phosphene characteristics, may allow for better prediction of functional outcome, which could in turn be useful for patient selection and tailored preoperative counselling. For those subjects already implanted with the Argus® II system, future work into determining the suitable stimulating parameters for each electrode / quad stimulation may be required for individual subjects, to achieve the construction of optimised and useful, pixelated prosthetic vision

Egocentric Computer Vision and Machine Learning for Simulated Prosthetic Vision

Las prótesis visuales actuales son capaces de proporcionar percepción visual a personas con cierta ceguera. Sin pasar por la parte dañada del camino visual, la estimulación eléctrica en la retina o en el sistema nervioso provoca percepciones puntuales conocidas como “fosfenos”. Debido a limitaciones fisiológicas y tecnológicas, la información que reciben los pacientes tiene una resolución muy baja y un campo de visión y rango dinámico reducido afectando seriamente la capacidad de la persona para reconocer y navegar en entornos desconocidos. En este contexto, la inclusión de nuevas técnicas de visión por computador es un tema clave activo y abierto. En esta tesis nos centramos especialmente en el problema de desarrollar técnicas para potenciar la información visual que recibe el paciente implantado y proponemos diferentes sistemas de visión protésica simulada para la experimentación.Primero, hemos combinado la salida de dos redes neuronales convolucionales para detectar bordes informativos estructurales y siluetas de objetos. Demostramos cómo se pueden reconocer rápidamente diferentes escenas y objetos incluso en las condiciones restringidas de la visión protésica. Nuestro método es muy adecuado para la comprensión de escenas de interiores comparado con los métodos tradicionales de procesamiento de imágenes utilizados en prótesis visuales.Segundo, presentamos un nuevo sistema de realidad virtual para entornos de visión protésica simulada más realistas usando escenas panorámicas, lo que nos permite estudiar sistemáticamente el rendimiento de la búsqueda y reconocimiento de objetos. Las escenas panorámicas permiten que los sujetos se sientan inmersos en la escena al percibir la escena completa (360 grados).En la tercera contribución demostramos cómo un sistema de navegación de realidad aumentada para visión protésica ayuda al rendimiento de la navegación al reducir el tiempo y la distancia para alcanzar los objetivos, incluso reduciendo significativamente el número de colisiones de obstáculos. Mediante el uso de un algoritmo de planificación de ruta, el sistema encamina al sujeto a través de una ruta más corta y sin obstáculos. Este trabajo está actualmente bajo revisión.En la cuarta contribución, evaluamos la agudeza visual midiendo la influencia del campo de visión con respecto a la resolución espacial en prótesis visuales a través de una pantalla montada en la cabeza. Para ello, usamos la visión protésica simulada en un entorno de realidad virtual para simular la experiencia de la vida real al usar una prótesis de retina. Este trabajo está actualmente bajo revisión.Finalmente, proponemos un modelo de Spiking Neural Network (SNN) que se basa en mecanismos biológicamente plausibles y utiliza un esquema de aprendizaje no supervisado para obtener mejores algoritmos computacionales y mejorar el rendimiento de las prótesis visuales actuales. El modelo SNN propuesto puede hacer uso de la señal de muestreo descendente de la unidad de procesamiento de información de las prótesis retinianas sin pasar por el análisis de imágenes retinianas, proporcionando información útil a los ciegos. Esté trabajo está actualmente en preparación.<br /