Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

Multi-modal and multi-dimensional biomedical image data analysis using deep learning

There is a growing need for the development of computational methods and tools for automated, objective, and quantitative analysis of biomedical signal and image data to facilitate disease and treatment monitoring, early diagnosis, and scientific discovery. Recent advances in artificial intelligence and machine learning, particularly in deep learning, have revolutionized computer vision and image analysis for many application areas. While processing of non-biomedical signal, image, and video data using deep learning methods has been very successful, high-stakes biomedical applications present unique challenges such as different image modalities, limited training data, need for explainability and interpretability etc. that need to be addressed. In this dissertation, we developed novel, explainable, and attention-based deep learning frameworks for objective, automated, and quantitative analysis of biomedical signal, image, and video data. The proposed solutions involve multi-scale signal analysis for oraldiadochokinesis studies; ensemble of deep learning cascades using global soft attention mechanisms for segmentation of meningeal vascular networks in confocal microscopy; spatial attention and spatio-temporal data fusion for detection of rare and short-term video events in laryngeal endoscopy videos; and a novel discrete Fourier transform driven class activation map for explainable-AI and weakly-supervised object localization and segmentation for detailed vocal fold motion analysis using laryngeal endoscopy videos. Experiments conducted on the proposed methods showed robust and promising results towards automated, objective, and quantitative analysis of biomedical data, that is of great value for potential early diagnosis and effective disease progress or treatment monitoring.Includes bibliographical references

Structural and functional brain imaging using extended-focus optical coherence tomography and microscopy

Neuroimaging techniques aim at revealing the anatomy and functional organisation of cerebral structures. Over the past decades, functional magnetic resonance imaging (fMRI) has revolutionized our understanding of human cerebral physiology through its ability to probe neural activity throughout the entire brain in a non-invasive fashion. Nevertheless, despite recent technological improvements, the spatial resolution of fMRI remains limited to a few hundreds of microns, restricting its use to macroscopic studies. Microscopic imaging solutions have been proposed to circumvent this limitation and have enabled revealing the existence of various cerebral structures, such as neuronal and vascular networks and their contribution to information processing and blood flow regulation within the brain. Optical imaging has proven, through its increased resolution and available contrast mechanisms, to be a valuable complement to fMRI for cellular-scale imaging. In this context, we demonstrate here the capabilities of an extension of optical coherence tomography, termed extended-focus optical coherence tomography (xf-OCT), in imaging cerebral structure and function at high resolution and very high acquisitions rates. Optical coherence tomography is an interferometric imaging technique using a low-coherence illumination source to provide fast, three-dimensional imaging of the back-scattering of tissue and cells. By multiplexing the interferometric ranging over several spectral channels, Fourier-domain OCT performs depth-resolved imaging at very high acquisition rates and high sensitivity. Increasing the lateral resolution of optical systems typically reduces the available depth-of-field and thus hampers this depth multiplexing advantage of OCT. Extended-focus systems aim at alleviating this trade-off between imaging depth and lateral resolution through the use of diffraction-less beams such as Bessel beams, providing high resolution imaging over large depths. The xf-OCT system therefore combines fast acquisition rates and high resolution, both characteristics required to image and study the structure and function of microscopic constituents of cerebral tissue. In this work, we performed functional brain imaging using the ability of xf-OCT to obtain quantita- tive measurements of blood flow in the brain. Changes in blood velocity evoked by neuronal activation were monitored and maps of hemodynamic activity were generated by adapting tools routinely used in fMRI to xf-OCT imaging. Additionally, three novel xf-OCT instruments are presented, wherein the advantages of different spectral ranges are exploited to reach specific imaging parameters. The increased contrast and resolution afforded by an illumination in the visible spectral range was used in two extended-focus optical coherence microscopy (xf-OCM) implementations for subcellular imaging of ex-vivo brain slices and cellular imaging of neurons, capillaries and myelinated axons in the superficial cortex in-vivo. Subsequently, an xf-OCT system is presented, operating in the infrared spectral range, wherein the reduced scattering enabled imaging the smallest capillaries deep in the murine cortex in-vivo

Anatomical Modeling of Cerebral Microvascular Structures: Application to Identify Biomarkers of Microstrokes

Les réseaux microvasculaires corticaux sont responsables du transport de l’oxygène et des substrats énergétiques vers les neurones. Ces réseaux réagissent dynamiquement aux demandes énergétiques lors d’une activation neuronale par le biais du couplage neurovasculaire. Afin d’élucider le rôle de la composante microvasculaire dans ce processus de couplage, l’utilisation de la modélisation in-formatique pourrait se révéler un élément clé. Cependant, la manque de méthodologies de calcul appropriées et entièrement automatisées pour modéliser et caractériser les réseaux microvasculaires reste l’un des principaux obstacles. Le développement d’une solution entièrement automatisée est donc important pour des explorations plus avancées, notamment pour quantifier l’impact des mal-formations vasculaires associées à de nombreuses maladies cérébrovasculaires. Une observation courante dans l’ensemble des troubles neurovasculaires est la formation de micro-blocages vascu-laires cérébraux (mAVC) dans les artérioles pénétrantes de la surface piale. De récents travaux ont démontré l’impact de ces événements microscopiques sur la fonction cérébrale. Par conséquent, il est d’une importance vitale de développer une approche non invasive et comparative pour identifier leur présence dans un cadre clinique. Dans cette thèse,un pipeline de traitement entièrement automatisé est proposé pour aborder le prob-lème de la modélisation anatomique microvasculaire. La méthode de modélisation consiste en un réseau de neurones entièrement convolutif pour segmenter les capillaires sanguins, un générateur de modèle de surface 3D et un algorithme de contraction de la géométrie pour produire des mod-èles graphiques vasculaires ne comportant pas de connections multiples. Une amélioration de ce pipeline est développée plus tard pour alléger l’exigence de maillage lors de la phase de représen-tation graphique. Un nouveau schéma permettant de générer un modèle de graphe est développé avec des exigences d’entrée assouplies et permettant de retenir les informations sur les rayons des vaisseaux. Il est inspiré de graphes géométriques déformants construits en respectant les morpholo-gies vasculaires au lieu de maillages de surface. Un mécanisme pour supprimer la structure initiale du graphe à chaque exécution est implémenté avec un critère de convergence pour arrêter le pro-cessus. Une phase de raffinement est introduite pour obtenir des modèles vasculaires finaux. La modélisation informatique développée est ensuite appliquée pour simuler les signatures IRM po-tentielles de mAVC, combinant le marquage de spin artériel (ASL) et l’imagerie multidirectionnelle pondérée en diffusion (DWI). L’hypothèse est basée sur des observations récentes démontrant une réorientation radiale de la microvascularisation dans la périphérie du mAVC lors de la récupéra-tion chez la souris. Des lits capillaires synthétiques, orientés aléatoirement et radialement, et des angiogrammes de tomographie par cohérence optique (OCT), acquis dans le cortex de souris (n = 5) avant et après l’induction d’une photothrombose ciblée, sont analysés. Les graphes vasculaires informatiques sont exploités dans un simulateur 3D Monte-Carlo pour caractériser la réponse par résonance magnétique (MR), tout en considérant les effets des perturbations du champ magnétique causées par la désoxyhémoglobine, et l’advection et la diffusion des spins nucléaires. Le pipeline graphique proposé est validé sur des angiographies synthétiques et réelles acquises avec différentes modalités d’imagerie. Comparé à d’autres méthodes effectuées dans le milieu de la recherche, les expériences indiquent que le schéma proposé produit des taux d’erreur géométriques et topologiques amoindris sur divers angiogrammes. L’évaluation confirme également l’efficacité de la méthode proposée en fournissant des modèles représentatifs qui capturent tous les aspects anatomiques des structures vasculaires. Ensuite, afin de trouver des signatures de mAVC basées sur le signal IRM, la modélisation vasculaire proposée est exploitée pour quantifier le rapport de perte de signal intravoxel minimal lors de l’application de plusieurs directions de gradient, à des paramètres de séquence variables avec et sans ASL. Avec l’ASL, les résultats démontrent une dif-férence significative (p <0,05) entre le signal calculé avant et 3 semaines après la photothrombose. La puissance statistique a encore augmenté (p <0,005) en utilisant des angiogrammes capturés à la semaine suivante. Sans ASL, aucun changement de signal significatif n’est trouvé. Des rapports plus élevés sont obtenus à des intensités de champ magnétique plus faibles (par exemple, B0 = 3) et une lecture TE plus courte (<16 ms). Cette étude suggère que les mAVC pourraient être carac-térisés par des séquences ASL-DWI, et fournirait les informations nécessaires pour les validations expérimentales postérieures et les futurs essais comparatifs.----------ABSTRACT Cortical microvascular networks are responsible for carrying the necessary oxygen and energy substrates to our neurons. These networks react to the dynamic energy demands during neuronal activation through the process of neurovascular coupling. A key element in elucidating the role of the microvascular component in the brain is through computational modeling. However, the lack of fully-automated computational frameworks to model and characterize these microvascular net-works remains one of the main obstacles. Developing a fully-automated solution is thus substantial for further explorations, especially to quantify the impact of cerebrovascular malformations associ-ated with many cerebrovascular diseases. A common pathogenic outcome in a set of neurovascular disorders is the formation of microstrokes, i.e., micro occlusions in penetrating arterioles descend-ing from the pial surface. Recent experiments have demonstrated the impact of these microscopic events on brain function. Hence, it is of vital importance to develop a non-invasive and translatable approach to identify their presence in a clinical setting. In this thesis, a fully automatic processing pipeline to address the problem of microvascular anatom-ical modeling is proposed. The modeling scheme consists of a fully-convolutional neural network to segment microvessels, a 3D surface model generator and a geometry contraction algorithm to produce vascular graphical models with a single connected component. An improvement on this pipeline is developed later to alleviate the requirement of water-tight surface meshes as inputs to the graphing phase. The novel graphing scheme works with relaxed input requirements and intrin-sically captures vessel radii information, based on deforming geometric graphs constructed within vascular boundaries instead of surface meshes. A mechanism to decimate the initial graph struc-ture at each run is formulated with a convergence criterion to stop the process. A refinement phase is introduced to obtain final vascular models. The developed computational modeling is then ap-plied to simulate potential MRI signatures of microstrokes, combining arterial spin labeling (ASL) and multi-directional diffusion-weighted imaging (DWI). The hypothesis is driven based on recent observations demonstrating a radial reorientation of microvasculature around the micro-infarction locus during recovery in mice. Synthetic capillary beds, randomly- and radially oriented, and op-tical coherence tomography (OCT) angiograms, acquired in the barrel cortex of mice (n=5) before and after inducing targeted photothrombosis, are analyzed. The computational vascular graphs are exploited within a 3D Monte-Carlo simulator to characterize the magnetic resonance (MR) re-sponse, encompassing the effects of magnetic field perturbations caused by deoxyhemoglobin, and the advection and diffusion of the nuclear spins. The proposed graphing pipeline is validated on both synthetic and real angiograms acquired with different imaging modalities. Compared to other efficient and state-of-the-art graphing schemes, the experiments indicate that the proposed scheme produces the lowest geometric and topological error rates on various angiograms. The evaluation also confirms the efficiency of the proposed scheme in providing representative models that capture all anatomical aspects of vascular struc-tures. Next, searching for MRI-based signatures of microstokes, the proposed vascular modeling is exploited to quantify the minimal intravoxel signal loss ratio when applying multiple gradient di-rections, at varying sequence parameters with and without ASL. With ASL, the results demonstrate a significant difference (p<0.05) between the signal-ratios computed at baseline and 3 weeks after photothrombosis. The statistical power further increased (p<0.005) using angiograms captured at week 4. Without ASL, no reliable signal change is found. Higher ratios with improved significance are achieved at low magnetic field strengths (e.g., at 3 Tesla) and shorter readout TE (<16 ms). This study suggests that microstrokes might be characterized through ASL-DWI sequences, and provides necessary insights for posterior experimental validations, and ultimately, future transla-tional trials

Automatic system for personal authentication using the retinal vessel tree as biometric pattern

[Resumen] La autenticación fiable de personas es un servicio cuya demanda aumenta en muchos campos, no sólo en entornos policiales o militares sino también en aplicaciones civiles tales como el control de acceso a zonas restringidas o la gestión de transacciones nancieras. Los sistemas de autenticación tradicionales están basados en el conocimiento (una palabra clave o un PIN ) o en la posesión (una tarjeta, o una llave). Dichos sistemas no son su cientemente ables en numerosos entornos, debido a su incapacidad común para diferenciar entre un usuario verdaderamente autorizado y otro que fraudulentamente haya adquirido el privilegio. Una solución para estos problemas se encuentra en las tecnologías de autenticación basadas en biometría. Un sistema biométrico es un sistema de reconocimiento de patrones que establece la autenticidad de los individuos caracterizándolos por medio de alguna característica física o de comportamiento. Existen muchas tecnologías de autenticación, algunas de ellas ya implementadas en paquetes comerciales. Las técnicas biométricas más comunes son la huella digital, probablemente la característica más antigua usada en biometría, iris, cara, geometría de la mano y, en cuanto a las características de comportamiento, reconocimiento de voz y rma. Hoy en día, la mayoría de los esfuerzos en los sistemas biométricos van encaminados al diseño de entornos más xi xii seguros donde sea más difícil, o virtualmente imposible, crear una copia de las propiedades utilizadas en el sistema para discriminar entre usuarios autorizados y no autorizados. En este contexto, el patrón de vasos sanguíneos en la retina se presenta como una característica biométrica relativamente joven pero muy interesante debido a sus propiedades inherentes. La más importante es que se trata de un patrón único para cada individuo. Además, al ser una característica interna es casi imposible crear una copia falsa. Por último, otra propiedad interesante es que el patrón no cambia signi cativamente a lo largo del tiempo excepto en casos de algunas patologías serias y no muy comunes. Por todo ello, el patrón de retina puede ser considerado un rasgo biométrico válido para la autenticación personal ya que es único, invariante en el tiempo y casi imposible de imitar. Por otra parte, el mayor incoveniente en el uso del patrón de vasos de la retina como característica biométrica radica en la etapa de adquisición todav ía percibida por el usuario como invasiva e incómoda. Hoy en día, existen mecanismos para obtener imágenes digitales de manera instantánea a través de cámaras no invasivas pero estos avances requieren a su vez una mayor tolerancia a variaciones en la calidad de la imagen adquirida y, por tanto, métodos computacionales más elaborados que sean capaces de procesar la información en entornos más heterogéneos. En esta tesis se presenta un nuevo sistema de autenticación automático usando el árbol retiniano como característica biométrica. El objetivo es diseñar y desarrollar un patrón biométrico robusto y compacto que sea fácilmente manejable y almacenable en dispositivos móviles de hoy en día como tarjetas con chip. La plantilla biométrica desarrollada a partir del árbol retiniano consiste en sus puntos característicos (bifurcaciones y cruces entre vasos) de forma que no sea necesario el almacenamiento y procesado de todo el árbol para realizar la autenticación

Visual Impairment and Blindness

Blindness and vision impairment affect at least 2.2 billion people worldwide with most individuals having a preventable vision impairment. The majority of people with vision impairment are older than 50 years, however, vision loss can affect people of all ages. Reduced eyesight can have major and long-lasting effects on all aspects of life, including daily personal activities, interacting with the community, school and work opportunities, and the ability to access public services. This book provides an overview of the effects of blindness and visual impairment in the context of the most common causes of blindness in older adults as well as children, including retinal disorders, cataracts, glaucoma, and macular or corneal degeneration