Egocentric Computer Vision and Machine Learning for Simulated Prosthetic Vision

Las prótesis visuales actuales son capaces de proporcionar percepción visual a personas con cierta ceguera. Sin pasar por la parte dañada del camino visual, la estimulación eléctrica en la retina o en el sistema nervioso provoca percepciones puntuales conocidas como “fosfenos”. Debido a limitaciones fisiológicas y tecnológicas, la información que reciben los pacientes tiene una resolución muy baja y un campo de visión y rango dinámico reducido afectando seriamente la capacidad de la persona para reconocer y navegar en entornos desconocidos. En este contexto, la inclusión de nuevas técnicas de visión por computador es un tema clave activo y abierto. En esta tesis nos centramos especialmente en el problema de desarrollar técnicas para potenciar la información visual que recibe el paciente implantado y proponemos diferentes sistemas de visión protésica simulada para la experimentación.Primero, hemos combinado la salida de dos redes neuronales convolucionales para detectar bordes informativos estructurales y siluetas de objetos. Demostramos cómo se pueden reconocer rápidamente diferentes escenas y objetos incluso en las condiciones restringidas de la visión protésica. Nuestro método es muy adecuado para la comprensión de escenas de interiores comparado con los métodos tradicionales de procesamiento de imágenes utilizados en prótesis visuales.Segundo, presentamos un nuevo sistema de realidad virtual para entornos de visión protésica simulada más realistas usando escenas panorámicas, lo que nos permite estudiar sistemáticamente el rendimiento de la búsqueda y reconocimiento de objetos. Las escenas panorámicas permiten que los sujetos se sientan inmersos en la escena al percibir la escena completa (360 grados).En la tercera contribución demostramos cómo un sistema de navegación de realidad aumentada para visión protésica ayuda al rendimiento de la navegación al reducir el tiempo y la distancia para alcanzar los objetivos, incluso reduciendo significativamente el número de colisiones de obstáculos. Mediante el uso de un algoritmo de planificación de ruta, el sistema encamina al sujeto a través de una ruta más corta y sin obstáculos. Este trabajo está actualmente bajo revisión.En la cuarta contribución, evaluamos la agudeza visual midiendo la influencia del campo de visión con respecto a la resolución espacial en prótesis visuales a través de una pantalla montada en la cabeza. Para ello, usamos la visión protésica simulada en un entorno de realidad virtual para simular la experiencia de la vida real al usar una prótesis de retina. Este trabajo está actualmente bajo revisión.Finalmente, proponemos un modelo de Spiking Neural Network (SNN) que se basa en mecanismos biológicamente plausibles y utiliza un esquema de aprendizaje no supervisado para obtener mejores algoritmos computacionales y mejorar el rendimiento de las prótesis visuales actuales. El modelo SNN propuesto puede hacer uso de la señal de muestreo descendente de la unidad de procesamiento de información de las prótesis retinianas sin pasar por el análisis de imágenes retinianas, proporcionando información útil a los ciegos. Esté trabajo está actualmente en preparación.<br /

Exploiting Temporal Image Information in Minimally Invasive Surgery

Minimally invasive procedures rely on medical imaging instead of the surgeons direct vision. While preoperative images can be used for surgical planning and navigation, once the surgeon arrives at the target site real-time intraoperative imaging is needed. However, acquiring and interpreting these images can be challenging and much of the rich temporal information present in these images is not visible. The goal of this thesis is to improve image guidance for minimally invasive surgery in two main areas. First, by showing how high-quality ultrasound video can be obtained by integrating an ultrasound transducer directly into delivery devices for beating heart valve surgery. Secondly, by extracting hidden temporal information through video processing methods to help the surgeon localize important anatomical structures. Prototypes of delivery tools, with integrated ultrasound imaging, were developed for both transcatheter aortic valve implantation and mitral valve repair. These tools provided an on-site view that shows the tool-tissue interactions during valve repair. Additionally, augmented reality environments were used to add more anatomical context that aids in navigation and in interpreting the on-site video. Other procedures can be improved by extracting hidden temporal information from the intraoperative video. In ultrasound guided epidural injections, dural pulsation provides a cue in finding a clear trajectory to the epidural space. By processing the video using extended Kalman filtering, subtle pulsations were automatically detected and visualized in real-time. A statistical framework for analyzing periodicity was developed based on dynamic linear modelling. In addition to detecting dural pulsation in lumbar spine ultrasound, this approach was used to image tissue perfusion in natural video and generate ventilation maps from free-breathing magnetic resonance imaging. A second statistical method, based on spectral analysis of pixel intensity values, allowed blood flow to be detected directly from high-frequency B-mode ultrasound video. Finally, pulsatile cues in endoscopic video were enhanced through Eulerian video magnification to help localize critical vasculature. This approach shows particular promise in identifying the basilar artery in endoscopic third ventriculostomy and the prostatic artery in nerve-sparing prostatectomy. A real-time implementation was developed which processed full-resolution stereoscopic video on the da Vinci Surgical System

Apport de la vision par ordinateur dans l'utilisabilité des neuroprothèses visuelles

L'OMS estime que 45 millions de personnes dans le monde sont aveugles. Avec le vieillissement de la population, ce chiffre ne cesse de progresser car la cécité touche majoritairement les personnes âgées. Les neuroprothèses visuelles ont pour objectif de restaurer une forme de vision. Ces systèmes convertissent les informations de la scène visuelle en percepts lumineux via des microstimulations électriques du système visuel. La perception visuelle ainsi générée consiste en un ensemble restreint de phosphènes. Ces systèmes sont, à ce jour, inutilisables dans un environnement naturel : l'information visuelle restituée est insuffisante pour que les personnes implantées puissent se déplacer, localiser des objets et les reconnaître. Au cours des dernières décennies, la vision par ordinateur a connu d'énormes avancées, grâce aux améliorations apportées aux algorithmes de traitement d'images et à l'augmentation de la puissance de calcul disponible. Il est désormais possible de localiser de manière fiable des objets, des visages ou du texte dans un environnement naturel. Or, la plupart des neuroprothèses visuelles intègrent une caméra facilement associable à un module de traitement d'images. Partant de ces constatations, nous avons montré qu'il est possible d'améliorer l'utilisabilité de ces systèmes, en utilisant des algorithmes de traitement d'images performants. En détectant des zones d'intérêt dans une scène naturelle et en les restituant à l'utilisateur par le biais d'un nombre limité de phosphènes, nos résultats indiquent qu'il est possible de restaurer des comportements visuo-moteurs adaptés : localisation d'objets, de visages ou encore de textes.The WHO estimates that 45 million people worldwide are blind. This figure is rapidly increasing because of the ageing of the world population, as blindness primarily affects elderly people. Visual neuroprostheses aim at restoring a sort of vision. These systems convert visual information captured by a camera into dots-like percepts via electrical microstimulation of the visual system. The evoked visual perception corresponds to a black and white image with a few dozen of pixels with gaps separating them. Although these systems give great hope to blind people, they are still inefficient in a natural environment: the restored visual information is too coarse to allow complex functions such as navigation, object localization and recognition, or reading at a convenient speed. Over the last decades, computer vision has been steadily improving, thanks to the development of new image processing algorithms and the increase of processing power. For instance, this is now possible to localize objects, faces or texts in real outdoor conditions. Interestingly, most of the current visual neuroprostheses include an external camera making it possible to process the input images in order to adapt the phosphenes display. In the current work, we showed that real-time image processing can improve the usability of low resolution visual neuroprostheses relying on the extraction of high-level information from the input images. Indeed, our results showed that the augmentation of the phosphene display with a limited number of phosphenes allows restoring visuomotor behaviors, such as localizing pertinent objects, faces or texts within a natural scene

Biomedical Sensing and Imaging

This book mainly deals with recent advances in biomedical sensing and imaging. More recently, wearable/smart biosensors and devices, which facilitate diagnostics in a non-clinical setting, have become a hot topic. Combined with machine learning and artificial intelligence, they could revolutionize the biomedical diagnostic field. The aim of this book is to provide a research forum in biomedical sensing and imaging and extend the scientific frontier of this very important and significant biomedical endeavor

Color space analysis for iris recognition

This thesis investigates issues related to the processing of multispectral and color infrared images of the iris. When utilizing the color bands of the electromagnetic spectrum, the eye color and the components of texture (luminosity and chromaticity) must be considered. This work examines the effect of eye color on texture-based iris recognition in both the near-IR and visible bands. A novel score level fusion algorithm for multispectral iris recognition is presented in this regard. The fusion algorithm - based on evidence that matching performance of a texture-based encoding scheme is impacted by the quality of texture within the original image - ranks the spectral bands of the image based on texture quality and designs a fusion rule based on these rankings. Color space analysis, to determine an optimal representation scheme, is also examined in this thesis. Color images are transformed from the sRGB color space to the CIE Lab, YCbCr, CMYK and HSV color spaces prior to encoding and matching. Also, enhancement methods to increase the contrast of the texture within the iris, without altering the chromaticity of the image, are discussed. Finally, cross-band matching is performed to illustrate the correlation between eye color and specific bands of the color image

Rehabilitation Engineering

Population ageing has major consequences and implications in all areas of our daily life as well as other important aspects, such as economic growth, savings, investment and consumption, labour markets, pensions, property and care from one generation to another. Additionally, health and related care, family composition and life-style, housing and migration are also affected. Given the rapid increase in the aging of the population and the further increase that is expected in the coming years, an important problem that has to be faced is the corresponding increase in chronic illness, disabilities, and loss of functional independence endemic to the elderly (WHO 2008). For this reason, novel methods of rehabilitation and care management are urgently needed. This book covers many rehabilitation support systems and robots developed for upper limbs, lower limbs as well as visually impaired condition. Other than upper limbs, the lower limb research works are also discussed like motorized foot rest for electric powered wheelchair and standing assistance device

Decision fusion in healthcare and medicine : a narrative review

Objective: To provide an overview of the decision fusion (DF) technique and describe the applications of the technique in healthcare and medicine at prevention, diagnosis, treatment and administrative levels. Background: The rapid development of technology over the past 20 years has led to an explosion in data growth in various industries, like healthcare. Big data analysis within the healthcare systems is essential for arriving to a value-based decision over a period of time. Diversity and uncertainty in big data analytics have made it impossible to analyze data by using conventional data mining techniques and thus alternative solutions are required. DF is a form of data fusion techniques that could increase the accuracy of diagnosis and facilitate interpretation, summarization and sharing of information. Methods: We conducted a review of articles published between January 1980 and December 2020 from various databases such as Google Scholar, IEEE, PubMed, Science Direct, Scopus and web of science using the keywords decision fusion (DF), information fusion, healthcare, medicine and big data. A total of 141 articles were included in this narrative review. Conclusions: Given the importance of big data analysis in reducing costs and improving the quality of healthcare; along with the potential role of DF in big data analysis, it is recommended to know the full potential of this technique including the advantages, challenges and applications of the technique before its use. Future studies should focus on describing the methodology and types of data used for its applications within the healthcare sector

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden