Using machine learning on the sources of retinal images for diagnosis by proxy of diabetes mellitus and diabetic retinopathy

In current research in ophthalmology, images of the vascular system in the human retina are used as exploratory proxies for pathologies affecting different organs. This thesis addresses the analysis, using machine learning and computer vision techniques, of retinal images acquired with different techniques (Fundus retinographies, optical coherence tomography and optical coherence tomography angiography), with the objective of using them to assist diagnostic decision making in diabetes mellitus and diabetic retinopathy. This thesis explores the use of matrix factorization-based source extraction techniques, as the basis to transform the retinal images for classification. The proposed approach consists on preprocessing the images to enable the learning of an unsupervised parts-based representation prior to the classification. As a result of the use of interpretable models, with this approach we unveiled an important bias in the data. After correcting for the bias, promising results were still obtained which merit for further exploration

Analysis of Retinal Image Data to Support Glaucoma Diagnosis

Fundus kamera je široce dostupné zobrazovací zařízení, které umožňuje relativně rychlé a nenákladné vyšetření zadního segmentu oka – sítnice. Z těchto důvodů se mnoho výzkumných pracovišť zaměřuje právě na vývoj automatických metod diagnostiky nemocí sítnice s využitím fundus fotografií. Tato dizertační práce analyzuje současný stav vědeckého poznání v oblasti diagnostiky glaukomu s využitím fundus kamery a navrhuje novou metodiku hodnocení vrstvy nervových vláken (VNV) na sítnici pomocí texturní analýzy. Spolu s touto metodikou je navržena metoda segmentace cévního řečiště sítnice, jakožto další hodnotný příspěvek k současnému stavu řešené problematiky. Segmentace cévního řečiště rovněž slouží jako nezbytný krok předcházející analýzu VNV. Vedle toho práce publikuje novou volně dostupnou databázi snímků sítnice se zlatými standardy pro účely hodnocení automatických metod segmentace cévního řečiště.Fundus camera is widely available imaging device enabling fast and cheap examination of the human retina. Hence, many researchers focus on development of automatic methods towards assessment of various retinal diseases via fundus images. This dissertation summarizes recent state-of-the-art in the field of glaucoma diagnosis using fundus camera and proposes a novel methodology for assessment of the retinal nerve fiber layer (RNFL) via texture analysis. Along with it, a method for the retinal blood vessel segmentation is introduced as an additional valuable contribution to the recent state-of-the-art in the field of retinal image processing. Segmentation of the blood vessels also serves as a necessary step preceding evaluation of the RNFL via the proposed methodology. In addition, a new publicly available high-resolution retinal image database with gold standard data is introduced as a novel opportunity for other researches to evaluate their segmentation algorithms.

Detección automática de la presencia de patología ocular en retinografías empleando técnicas de procesado de imágenes

La vista es uno de los sentidos de mayor importancia para la vida humana. En los últimos años el número de enfermedades oculares ha aumentado y las predicciones de los científicos es que van a seguir aumentando en los próximos años. Existen enfermedades oculares que se han convertido en importantes causas de pérdida de visión a nivel mundial como la retinopatía diabética (RD), el glaucoma, la degeneración macular asociada a la edad (DMAE) y las cataratas. Estas enfermedades oculares suelen provocar alteraciones en el ojo humano, que pueden detectarse observando el ojo. Una de las técnicas más extendidas para observar el fondo del ojo es la retinografía, que es una imagen digital a color de la retina. Esta imagen es muy útil para el diagnóstico de enfermedades que afectan al ojo como RD y DMAE, entre otras. No obstante, la creciente incidencia de algunas enfermedades oculares y la escasez de oftalmólogos especialistas provoca que el análisis de las retinografías sea una tarea compleja y laboriosa. El objetivo de este Trabajo Fin de Grado (TFG) ha sido el diseño y desarrollo de un método automático para diferenciar entre retinografías patológicas y no patológicas. Este método permitiría ayudar en el diagnóstico y cribado de los pacientes con enfermedades oculares y reducir la carga de trabajo a los oftalmólogos. Para ello, se partió de una base de datos (BD) formada por 1044 imágenes de calidad adecuada para su procesado automático. De ellas, 326 pertenecían a sujetos sanos y a 819 pacientes con algún tipo de patología. Estas imágenes se dividieron en un conjunto de entrenamiento (559 imágenes) y un conjunto de test (585 imágenes). En todos los casos, un oftalmólogo especialista indicó si las imágenes eran normales o patológicas.Grado en Ingeniería de Tecnologías de Telecomunicació

Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

Investigation of the Retinal Biomarkers of Alzheimer’s Disease and Atherosclerosis Using Hyperspectral Images

Le fait que l'oeil puisse être visualisé de manière non invasive ouvre des possibilités de mesure de biomarqueurs pour le diagnostic de conditions à long terme. Selon de nombreuses études, plusieurs maladies cardiovasculaires et neurodégénératives telles que la maladie d’Alzheimer (AD) et l’athérosclérose (ATH) se manifestent dans la rétine sous forme de modifications morphologiques pathologiques et / ou vasculaires. Des méthodes d'imagerie oculaire en deux dimensions et des techniques de tomographie par cohérence optique (OCT) en trois dimensions ont été développées pour fournir des descriptions des structures rétiniennes. Cependant, les images acquises par ces techniques permettent principalement de mesurer les caractéristiques spatiales et pas la variance relative de l’intensité des pixels sur différentes longueurs d’onde, de sorte que d’importantes caractéristiques liées aux tissus peuvent encore rester à découvrir. Dans cette étude, une caméra rétinienne métabolique hyperspectrale (MHRC) a été utilisée pour permettre l'acquisition d'une série d'images rétiniennes obtenues à des longueurs d'onde spécifiques couvrant le spectre du visible au proche infrarouge (NIR). Dans cette technique, le facteur de transmission, l'absorption et la diffusion de la lumière sont reflétés dans le spectre de la lumière émise par le tissu. Par conséquent, non seulement les caractéristiques spatiales communes mais également les « signatures spectrales » de biomolécules pourraient être révélées. Cela aide à trouver une plus grande variété de caractéristiques spatiales / spectrales pour une investigation plus précise des biomarqueurs rétiniens des maladies. En ce qui concerne les coûts et les limites associés aux diagnostics actuels de l’AD et de l’ATH, le but de cette thèse était d’analyser le contenu en informations d’images rétiniennes hyperspectrales riches en données dans le but de caractériser des informations discriminantes cachées liées aux tissus afin d’identifier des biomarqueurs possibles de ces deux maladies. À cette fin, une combinaison de caractéristiques vasculaires et de mesures de textures spatiales-spectrales ont été extraites de différentes régions anatomiques de la rétine. Dans le contexte de la maladie d'Alzheimer, des images rétiniennes de 20 cas présentant une altération cognitive et de 26 cas normaux cognitivement ont été acquises à l'aide de la caméra MHRC. Le statut amyloïde cérébral a été déterminé à partir de lectures binaires effectuées par un panel de 3 experts noteurs ayant participé à des études de TEP au 18F-Florbetaben. Des caractéristiques de l’image rétinienne ont été calculées, notamment la tortuosité et le diamètre des vaisseaux, ainsi que les mesures de textures spatiales-spectrales sur les artérioles, les veinules et le tissu environnant. Les veinules rétiniennes des sujets amyloïdes positifs (Aβ +) ont présenté une tortuosité moyenne plus élevée par rapport aux sujets amyloïdes négatifs (Aβ-). Le diamètre artériolaire des sujets Aβ + s'est avéré supérieur à celui des sujets Aβ- dans une zone adjacente à la tête du nerf optique. De plus, une différence significative entre les mesures de texture construites sur les artérioles rétiniennes et leurs régions adjacentes a été observée chez les sujets Aβ + par rapport aux Aβ-. Dans le contexte de l'ATH, 60 images rétiniennes de 30 ATH probables sur le plan clinique et 30 cas de contrôle ont été acquises. Les critères d'inclusion pour les sujets souffrant d'ATH comprenaient: l'infarctus du myocarde; angiographie coronaire montrant au moins une sténose coronaire (plus de 50%); et / ou une angioplastie coronaire; et /ou pontage coronaire. Les artérioles rétiniennes des sujets ATH ont montré un rétrécissement significatif par rapport aux sujets témoins. En outre, une différence significative entre les mesures de textures d'images prises sur les artérioles et les veinules rétiniennes et leurs régions adjacentes a été trouvée entre les sujets ATH et les sujets témoins. Nos études transversales ont montré que l’analyse hyperspectrale des images rétiniennes pouvait discerner avec une précision acceptable l’AD et l’ATH des sujets témoins correspondants.----------ABSTRACT The fact that eye can be visualized non-invasively, opens up possibilities to measure biomarkers for diagnosis of long-term conditions. A significant body of literature has demonstrated that many of the neurodegenerative and cardiovascular diseases such as Alzheimer’s disease (AD) and atherosclerosis (ATH) manifest themselves in retina as pathological and/or vasculature morphological changes. Methods for two-dimensional fundus imaging and techniques for three-dimensional optical coherence tomography (OCT) have been developed to provide descriptions of retinal structures. However, images acquired by these techniques mostly allow for measuring the spatial characteristics of the tissue and lack of the relative variances across differing wavelengths, thus important spectral features may remain uncovered. In this study, a Metabolic Hyperspectral Retinal Camera (MHRC) was used that permits the acquisition of a series of retinal images obtained at specific wavelengths covering the visible and near infrared (NIR) spectrum. In this technique, light transmittance, absorption, and scatter are reflected in the spectrum of light emitted from the tissue. Use of MHRC in this study was aimed to extract not only the common spatial features but also “spectral signatures” of biomolecules in retinal tissue. Regarding the costs and limitations of the current diagnostic methods for AD and ATH, the purpose of this thesis was to analyze the information content of data-rich hyperspectral retinal images to characterize tissue-related discriminatory information to identify possible biomarkers of Alzheimer’s disease and atherosclerosis. To this end, a combination of vascular features and spatial/spectral texture measures were extracted from different anatomical regions of the retina. In the context of AD, retinal images from 20 cognitively impaired and 26 cognitively unimpaired cases were acquired using MHRC. The cerebral amyloid status was determined from binary reads by a panel of three expert raters on 18F-Florbetaben PET studies. Our approach did not aim to visualize directly Aβ deposits in the retina but rather to determine a likely amyloid status based on sets of retinal image features highly correlated with the cerebral amyloid status. Retinal image features were calculated including vessels’ tortuosity and diameter. Spatial/spectral texture measures over arterioles, venules, and tissue around were also extracted. Retinal venules of amyloid positive subjects (Aβ+) showed a higher mean tortuosity compared to the amyloid negative (Aβ-) subjects. Arteriolar diameter of Aβ+ subjects was found to be higher than the Aβ- subjects in a zone adjacent to the optical nerve head. Furthermore, a significant difference between spatial/spectral texture measures built over retinal arterioles and surrounding tissues were observed in Aβ+ subjects when compared to the Aβ-. In the context of ATH, 60 retinal images from 30 clinically probable ATH and 30 control cases were acquired. Inclusion criteria for subjects suffering from ATH included: myocardial infarction; coronary angiography showing at least one coronary stenosis (more than 50%); and/or coronary angioplasty; and/or coronary bypass. Retinal arterioles of ATH subjects showed a significant narrowing when compared to control subjects. Moreover, a significant difference between image texture measures taken over retinal arterioles and retinal venules and their adjacent regions was observed between ATH subjects and control subjects. Our cross-sectional studies have shown that hyperspectral retinal image analysis could be used to discriminate AD and ATH from corresponding control subjects based on a non-invasive eye scan

Face Recognition: Study and Comparison of PCA and EBGM Algorithms

Face recognition is a complex and difficult process due to various factors such as variability of illumination, occlusion, face specific characteristics like hair, glasses, beard, etc., and other similar problems affecting computer vision problems. Using a system that offers robust and consistent results for face recognition, various applications such as identification for law enforcement, secure system access, computer human interaction, etc., can be automated successfully. Different methods exist to solve the face recognition problem. Principal component analysis, Independent component analysis, and linear discriminant analysis are few other statistical techniques that are commonly used in solving the face recognition problem. Genetic algorithm, elastic bunch graph matching, artificial neural network, etc. are few of the techniques that have been proposed and implemented. The objective of this thesis paper is to provide insight into different methods available for face recognition, and explore methods that provided an efficient and feasible solution. Factors affecting the result of face recognition and the preprocessing steps that eliminate such abnormalities are also discussed briefly. Principal Component Analysis (PCA) is the most efficient and reliable method known for at least past eight years. Elastic bunch graph matching (EBGM) technique is one of the promising techniques that we studied in this thesis work. We also found better results with EBGM method than PCA in the current thesis paper. We recommend use of a hybrid technique involving the EBGM algorithm to obtain better results. Though, the EBGM method took a long time to train and generate distance measures for the given gallery images compared to PCA. But, we obtained better cumulative match score (CMS) results for the EBGM in comparison to the PCA method. Other promising techniques that can be explored separately in other paper include Genetic algorithm based methods, Mixture of principal components, and Gabor wavelet techniques

Multispectral image analysis in laparoscopy – A machine learning approach to live perfusion monitoring

Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physician’s perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by “imitating” the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the technique’s full potential. The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible. To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible. In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part of the next generation OR

BEst (Biomarker Estimation): Health Biomarker Estimation Non-invasively and Ubiquitously

This dissertation focuses on the non-invasive assessment of blood-hemoglobin levels. The primary goal of this research is to investigate a reliable, affordable, and user-friendly point-of-care solution for hemoglobin-level determination using fingertip videos captured by a smartphone. I evaluated videos obtained from five patient groups, three from the United States and two from Bangladesh, under two sets of lighting conditions. In the last group, based on human tissue optical transmission modeling data, I used near-infrared light-emitting diode sources of three wavelengths. I developed novel image processing techniques for fingertip video analysis to estimate hemoglobin levels. I studied video images creating image histogram and subdividing each image into multiple blocks. I determined the region of interest in a video and created photoplethysmogram signals. I created features from image histograms and PPG signals. I used the Partial Least Squares Regression and Support Vector Machine Regression tools to analyze input features and to build hemoglobin prediction models. Using data from the last and largest group of patients studied, I was able to develop a model with a strong linear correlation between estimated and clinically-measured hemoglobin levels. With further data and methodological refinements, the approach I have developed may be able to define a clinically accurate public health applicable tool for hemoglobin level and other blood constituent assessment

Machine Learning Techniques, Detection and Prediction of Glaucoma– A Systematic Review

Globally, glaucoma is the most common factor in both permanent blindness and impairment. However, the majority of patients are unaware they have the condition, and clinical practise continues to face difficulties in detecting glaucoma progression using current technology. An expert ophthalmologist examines the retinal portion of the eye to see how the glaucoma is progressing. This method is quite time-consuming, and doing it manually takes more time. Therefore, using deep learning and machine learning techniques, this problem can be resolved by automatically diagnosing glaucoma. This systematic review involved a comprehensive analysis of various automated glaucoma prediction and detection techniques. More than 100 articles on Machine learning (ML) techniques with understandable graph and tabular column are reviewed considering summery, method, objective, performance, advantages and disadvantages. In the ML techniques such as support vector machine (SVM), and K-means. Fuzzy c-means clustering algorithm are widely used in glaucoma detection and prediction. Through the systematic review, the most accurate technique to detect and predict glaucoma can be determined which can be utilized for future betterment