Extraction of arterial and venous trees from disconnected vessel segments in fundus images

The accurate automated extraction of arterial and venous (AV) trees in fundus images subserves investigation into the correlation of global features of the retinal vasculature with retinal abnormalities. The accurate extraction of AV trees also provides the opportunity to analyse the physiology and hemodynamic of blood flow in retinal vessel trees. A number of common diseases, including Diabetic Retinopathy, Cardiovascular and Cerebrovascular diseases, directly affect the morphology of the retinal vasculature. Early detection of these pathologies may prevent vision loss and reduce the risk of other life-threatening diseases. Automated extraction of AV trees requires complete segmentation and accurate classification of retinal vessels. Unfortunately, the available segmentation techniques are susceptible to a number of complications including vessel contrast, fuzzy edges, variable image quality, media opacities, and vessel overlaps. Due to these sources of errors, the available segmentation techniques produce partially segmented vascular networks. Thus, extracting AV trees by accurately connecting and classifying the disconnected segments is extremely complex. This thesis provides a novel graph-based technique for accurate extraction of AV trees from a network of disconnected and unclassified vessel segments in fundus viii images. The proposed technique performs three major tasks: junction identification, local configuration, and global configuration. A probabilistic approach is adopted that rigorously identifies junctions by examining the mutual associations of segment ends. These associations are determined by dynamically specifying regions at both ends of all segments. A supervised Naïve Bayes inference model is developed that estimates the probability of each possible configuration at a junction. The system enumerates all possible configurations and estimates posterior probability of each configuration. The likelihood function estimates the conditional probability of the configuration using the statistical parameters of distribution of colour and geometrical features of joints. The parameters of feature distributions and priors of configuration are obtained through supervised learning phases. A second Naïve Bayes classifier estimates class probabilities of each vessel segment utilizing colour and spatial properties of segments. The global configuration works by translating the segment network into an STgraph (a specialized form of dependency graph) representing the segments and their possible connective associations. The unary and pairwise potentials for ST-graph are estimated using the class and configuration probabilities obtained earlier. This translates the classification and configuration problems into a general binary labelling graph problem. The ST-graph is interpreted as a flow network for energy minimization a minimum ST-graph cut is obtained using the Ford-Fulkerson algorithm, from which the estimated AV trees are extracted. The performance is evaluated by implementing the system on test images of DRIVE dataset and comparing the obtained results with the ground truth data. The ground truth data is obtained by establishing a new dataset for DRIVE images with manually classified vessels. The system outperformed benchmark methods and produced excellent results

Vascular Tree Tracking and Bifurcation Points Detection in Retinal Images Using a Hierarchical Probabilistic Model

Background and Objective Retinal vascular tree extraction plays an important role in computer-aided diagnosis and surgical operations. Junction point detection and classification provide useful information about the structure of the vascular network, facilitating objective analysis of retinal diseases. Methods In this study, we present a new machine learning algorithm for joint classification and tracking of retinal blood vessels. Our method is based on a hierarchical probabilistic framework, where the local intensity cross sections are classified as either junction or vessel points. Gaussian basis functions are used for intensity interpolation, and the corresponding linear coefficients are assumed to be samples from class-specific Gamma distributions. Hence, a directed Probabilistic Graphical Model (PGM) is proposed and the hyperparameters are estimated using a Maximum Likelihood (ML) solution based on Laplace approximation. Results The performance of proposed method is evaluated using precision and recall rates on the REVIEW database. Our experiments show the proposed approach reaches promising results in bifurcation point detection and classification, achieving 88.67% precision and 88.67% recall rates. Conclusions This technique results in a classifier with high precision and recall when comparing it with Xu’s method

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Probabilistic labelling for enhancement of vessel networks applied to retinal images

Occlusive vascular disease affecting arterial circulations is the major and fastest growing health problem worldwide, and underlies common conditions such as heart attack, stroke and peripheral vascular disease. Although vascular diseases may be assessed according to clinical history, screening may be required to evaluate health conditions or courses of treatment. Vasculature in the retina and other organs such as the brain have similar anatomical properties and regulatory mechanisms. Changes in the morphology of retinal vasculature may be associated with vascular-related conditions such as hypertension and stroke. Owing to its high cost-effectiveness, eye fundus photography is often used to study changes in the retinal vasculature. This research proposes a probabilistic pixel labelling method based on analysis of local and global features of the image to enhance the detail of vessel structures. Our approach produces a probability map that could be further used by contextual approaches (e.g. Markov Random Fields) for segmenting vessel networks as future application. We first correct contrast variation due to non-uniform illumination and reflections produced by eye tissue using statistical methods to locally estimate the contrast behind vasculature structures. Our labelling method is based on the Hessian matrix to locally estimate the maximum probability of the principal local curvature—given by eigenvalues—matching an ideal vessel curvature. We defined a realistic model based on imaging physics to produce the ideal vessel curvature governed by the Beer-Lambert Law for estimating the absorption of energy as it is propagated through uniformly filled objects. The local maximum posterior probability—based on Bayes’ rule—was eventually estimated by combining the prior (using the proposed background estimation) and the likelihood produced by Monte Carlo simulations. The proposed method in this research was compared with one of the most popular vessel detectors due to Frangi showing similar results

A retinal vasculature tracking system guided by a deep architecture

Many diseases such as diabetic retinopathy (DR) and cardiovascular diseases show their early signs on retinal vasculature. Analysing the vasculature in fundus images may provide a tool for ophthalmologists to diagnose eye-related diseases and to monitor their progression. These analyses may also facilitate the discovery of new relations between changes on retinal vasculature and the existence or progression of related diseases or to validate present relations. In this thesis, a data driven method, namely a Translational Deep Belief Net (a TDBN), is adapted to vasculature segmentation. The segmentation performance of the TDBN on low resolution images was found to be comparable to that of the best-performing methods. Later, this network is used for the implementation of super-resolution for the segmentation of high resolution images. This approach provided an acceleration during segmentation, which relates to down-sampling ratio of an input fundus image. Finally, the TDBN is extended for the generation of probability maps for the existence of vessel parts, namely vessel interior, centreline, boundary and crossing/bifurcation patterns in centrelines. These probability maps are used to guide a probabilistic vasculature tracking system. Although segmentation can provide vasculature existence in a fundus image, it does not give quantifiable measures for vasculature. The latter has more practical value in medical clinics. In the second half of the thesis, a retinal vasculature tracking system is presented. This system uses Particle Filters to describe vessel morphology and topology. Apart from previous studies, the guidance for tracking is provided with the combination of probability maps generated by the TDBN. The experiments on a publicly available dataset, REVIEW, showed that the consistency of vessel widths predicted by the proposed method was better than that obtained from observers. Moreover, very noisy and low contrast vessel boundaries, which were hardly identifiable to the naked eye, were accurately estimated by the proposed tracking system. Also, bifurcation/crossing locations during the course of tracking were detected almost completely. Considering these promising initial results, future work involves analysing the performance of the tracking system on automatic detection of complete vessel networks in fundus images.Open Acces

Segmentation and Characterization of Small Retinal Vessels in Fundus Images Using the Tensor Voting Approach

RÉSUMÉ La rétine permet de visualiser facilement une partie du réseau vasculaire humain. Elle offre ainsi un aperçu direct sur le développement et le résultat de certaines maladies liées au réseau vasculaire dans son entier. Chaque complication visible sur la rétine peut avoir un impact sur la capacité visuelle du patient. Les plus petits vaisseaux sanguins sont parmi les premières structures anatomiques affectées par la progression d’une maladie, être capable de les analyser est donc crucial. Les changements dans l’état, l’aspect, la morphologie, la fonctionnalité, ou même la croissance des petits vaisseaux indiquent la gravité des maladies. Le diabète est une maladie métabolique qui affecte des millions de personnes autour du monde. Cette maladie affecte le taux de glucose dans le sang et cause des changements pathologiques dans différents organes du corps humain. La rétinopathie diabétique décrit l’en- semble des conditions et conséquences du diabète au niveau de la rétine. Les petits vaisseaux jouent un rôle dans le déclenchement, le développement et les conséquences de la rétinopa- thie. Dans les dernières étapes de cette maladie, la croissance des nouveaux petits vaisseaux, appelée néovascularisation, présente un risque important de provoquer la cécité. Il est donc crucial de détecter tous les changements qui ont lieu dans les petits vaisseaux de la rétine dans le but de caractériser les vaisseaux sains et les vaisseaux anormaux. La caractérisation en elle-même peut faciliter la détection locale d’une rétinopathie spécifique. La segmentation automatique des structures anatomiques comme le réseau vasculaire est une étape cruciale. Ces informations peuvent être fournies à un médecin pour qu’elles soient considérées lors de son diagnostic. Dans les systèmes automatiques d’aide au diagnostic, le rôle des petits vaisseaux est significatif. Ne pas réussir à les détecter automatiquement peut conduire à une sur-segmentation du taux de faux positifs des lésions rouges dans les étapes ultérieures. Les efforts de recherche se sont concentrés jusqu’à présent sur la localisation précise des vaisseaux de taille moyenne. Les modèles existants ont beaucoup plus de difficultés à extraire les petits vaisseaux sanguins. Les modèles existants ne sont pas robustes à la grande variance d’apparence des vaisseaux ainsi qu’à l’interférence avec l’arrière-plan. Les modèles de la littérature existante supposent une forme générale qui n’est pas suffisante pour s’adapter à la largeur étroite et la courbure qui caractérisent les petits vaisseaux sanguins. De plus, le contraste avec l’arrière-plan dans les régions des petits vaisseaux est très faible. Les méthodes de segmentation ou de suivi produisent des résultats fragmentés ou discontinus. Par ailleurs, la segmentation des petits vaisseaux est généralement faite aux dépends de l’amplification du bruit. Les modèles déformables sont inadéquats pour segmenter les petits vaisseaux. Les forces utilisées ne sont pas assez flexibles pour compenser le faible contraste, la largeur, et vii la variance des vaisseaux. Enfin, les approches de type apprentissage machine nécessitent un entraînement avec une base de données étiquetée. Il est très difficile d’obtenir ces bases de données dans le cas des petits vaisseaux. Cette thèse étend les travaux de recherche antérieurs en fournissant une nouvelle mé- thode de segmentation des petits vaisseaux rétiniens. La détection de ligne à échelles multiples (MSLD) est une méthode récente qui démontre une bonne performance de segmentation dans les images de la rétine, tandis que le vote tensoriel est une méthode proposée pour reconnecter les pixels. Une approche combinant un algorithme de détection de ligne et de vote tensoriel est proposée. L’application des détecteurs de lignes a prouvé son efficacité à segmenter les vais- seaux de tailles moyennes. De plus, les approches d’organisation perceptuelle comme le vote tensoriel ont démontré une meilleure robustesse en combinant les informations voisines d’une manière hiérarchique. La méthode de vote tensoriel est plus proche de la perception humain que d’autres modèles standards. Comme démontré dans ce manuscrit, c’est un outil pour segmenter les petits vaisseaux plus puissant que les méthodes existantes. Cette combinaison spécifique nous permet de surmonter les défis de fragmentation éprouvés par les méthodes de type modèle déformable au niveau des petits vaisseaux. Nous proposons également d’utiliser un seuil adaptatif sur la réponse de l’algorithme de détection de ligne pour être plus robuste aux images non-uniformes. Nous illustrons également comment une combinaison des deux méthodes individuelles, à plusieurs échelles, est capable de reconnecter les vaisseaux sur des distances variables. Un algorithme de reconstruction des vaisseaux est également proposé. Cette dernière étape est nécessaire car l’information géométrique complète est requise pour pouvoir utiliser la segmentation dans un système d’aide au diagnostic. La segmentation a été validée sur une base de données d’images de fond d’oeil à haute résolution. Cette base contient des images manifestant une rétinopathie diabétique. La seg- mentation emploie des mesures de désaccord standards et aussi des mesures basées sur la perception. En considérant juste les petits vaisseaux dans les images de la base de données, l’amélioration dans le taux de sensibilité que notre méthode apporte par rapport à la méthode standard de détection multi-niveaux de lignes est de 6.47%. En utilisant les mesures basées sur la perception, l’amélioration est de 7.8%. Dans une seconde partie du manuscrit, nous proposons également une méthode pour caractériser les rétines saines ou anormales. Certaines images contiennent de la néovascula- risation. La caractérisation des vaisseaux en bonne santé ou anormale constitue une étape essentielle pour le développement d’un système d’aide au diagnostic. En plus des défis que posent les petits vaisseaux sains, les néovaisseaux démontrent eux un degré de complexité encore plus élevé. Ceux-ci forment en effet des réseaux de vaisseaux à la morphologie com- plexe et inhabituelle, souvent minces et à fortes courbures. Les travaux existants se limitent viii à l’utilisation de caractéristiques de premier ordre extraites des petits vaisseaux segmentés. Notre contribution est d’utiliser le vote tensoriel pour isoler les jonctions vasculaires et d’uti- liser ces jonctions comme points d’intérêts. Nous utilisons ensuite une statistique spatiale de second ordre calculée sur les jonctions pour caractériser les vaisseaux comme étant sains ou pathologiques. Notre méthode améliore la sensibilité de la caractérisation de 9.09% par rapport à une méthode de l’état de l’art. La méthode développée s’est révélée efficace pour la segmentation des vaisseaux réti- niens. Des tenseurs d’ordre supérieur ainsi que la mise en œuvre d’un vote par tenseur via un filtrage orientable pourraient être étudiés pour réduire davantage le temps d’exécution et résoudre les défis encore présents au niveau des jonctions vasculaires. De plus, la caractéri- sation pourrait être améliorée pour la détection de la rétinopathie proliférative en utilisant un apprentissage supervisé incluant des cas de rétinopathie diabétique non proliférative ou d’autres pathologies. Finalement, l’incorporation des méthodes proposées dans des systèmes d’aide au diagnostic pourrait favoriser le dépistage régulier pour une détection précoce des rétinopathies et d’autres pathologies oculaires dans le but de réduire la cessité au sein de la population.----------ABSTRACT As an easily accessible site for the direct observation of the circulation system, human retina can offer a unique insight into diseases development or outcome. Retinal vessels are repre- sentative of the general condition of the whole systematic circulation, and thus can act as a "window" to the status of the vascular network in the whole body. Each complication on the retina can have an adverse impact on the patient’s sight. In this direction, small vessels’ relevance is very high as they are among the first anatomical structures that get affected as diseases progress. Moreover, changes in the small vessels’ state, appearance, morphology, functionality, or even growth indicate the severity of the diseases. This thesis will focus on the retinal lesions due to diabetes, a serious metabolic disease affecting millions of people around the world. This disorder disturbs the natural blood glucose levels causing various pathophysiological changes in different systems across the human body. Diabetic retinopathy is the medical term that describes the condition when the fundus and the retinal vessels are affected by diabetes. As in other diseases, small vessels play a crucial role in the onset, the development, and the outcome of the retinopathy. More importantly, at the latest stage, new small vessels, or neovascularizations, growth constitutes a factor of significant risk for blindness. Therefore, there is a need to detect all the changes that occur in the small retinal vessels with the aim of characterizing the vessels to healthy or abnormal. The characterization, in turn, can facilitate the detection of a specific retinopathy locally, like the sight-threatening proliferative diabetic retinopathy. Segmentation techniques can automatically isolate important anatomical structures like the vessels, and provide this information to the physician to assist him in the final decision. In comprehensive systems for the automatization of DR detection, small vessels role is significant as missing them early in a CAD pipeline might lead to an increase in the false positive rate of red lesions in subsequent steps. So far, the efforts have been concentrated mostly on the accurate localization of the medium range vessels. In contrast, the existing models are weak in case of the small vessels. The required generalization to adapt an existing model does not allow the approaches to be flexible, yet robust to compensate for the increased variability in the appearance as well as the interference with the background. So far, the current template models (matched filtering, line detection, and morphological processing) assume a general shape for the vessels that is not enough to approximate the narrow, curved, characteristics of the small vessels. Additionally, due to the weak contrast in the small vessel regions, the current segmentation and the tracking methods produce fragmented or discontinued results. Alternatively, the small vessel segmentation can be accomplished at the expense of x background noise magnification, in the case of using thresholding or the image derivatives methods. Furthermore, the proposed deformable models are not able to propagate a contour to the full extent of the vasculature in order to enclose all the small vessels. The deformable model external forces are ineffective to compensate for the low contrast, the low width, the high variability in the small vessel appearance, as well as the discontinuities. Internal forces, also, are not able to impose a global shape constraint to the contour that could be able to approximate the variability in the appearance of the vasculature in different categories of vessels. Finally, machine learning approaches require the training of a classifier on a labelled set. Those sets are difficult to be obtained, especially in the case of the smallest vessels. In the case of the unsupervised methods, the user has to predefine the number of clusters and perform an effective initialization of the cluster centers in order to converge to the global minimum. This dissertation expanded the previous research work and provides a new segmentation method for the smallest retinal vessels. Multi-scale line detection (MSLD) is a recent method that demonstrates good segmentation performance in the retinal images, while tensor voting is a method first proposed for reconnecting pixels. For the first time, we combined the line detection with the tensor voting framework. The application of the line detectors has been proved an effective way to segment medium-sized vessels. Additionally, perceptual organization approaches like tensor voting, demonstrate increased robustness by combining information coming from the neighborhood in a hierarchical way. Tensor voting is closer than standard models to the way human perception functions. As we show, it is a more powerful tool to segment small vessels than the existing methods. This specific combination allows us to overcome the apparent fragmentation challenge of the template methods at the smallest vessels. Moreover, we thresholded the line detection response adaptively to compensate for non-uniform images. We also combined the two individual methods in a multi-scale scheme in order to reconnect vessels at variable distances. Finally, we reconstructed the vessels from their extracted centerlines based on pixel painting as complete geometric information is required to be able to utilize the segmentation in a CAD system. The segmentation was validated on a high-resolution fundus image database that in- cludes diabetic retinopathy images of varying stages, using standard discrepancy as well as perceptual-based measures. When only the smallest vessels are considered, the improve- ments in the sensitivity rate for the database against the standard multi-scale line detection method is 6.47%. For the perceptual-based measure, the improvement is 7.8% against the basic method. The second objective of the thesis was to implement a method for the characterization of isolated retinal areas into healthy or abnormal cases. Some of the original images, from which xi these patches are extracted, contain neovascularizations. Investigation of image features for the vessels characterization to healthy or abnormal constitutes an essential step in the direction of developing CAD system for the automatization of DR screening. Given that the amount of data will significantly increase under CAD systems, the focus on this category of vessels can facilitate the referral of sight-threatening cases to early treatment. In addition to the challenges that small healthy vessels pose, neovessels demonstrate an even higher degree of complexity as they form networks of convolved, twisted, looped thin vessels. The existing work is limited to the use of first-order characteristics extracted from the small segmented vessels that limits the study of patterns. Our contribution is in using the tensor voting framework to isolate the retinal vascular junctions and in turn using those junctions as points of interests. Second, we exploited second-order statistics computed on the junction spatial distribution to characterize the vessels as healthy or neovascularizations. In fact, the second-order spatial statistics extracted from the junction distribution are combined with widely used features to improve the characterization sensitivity by 9.09% over the state of art. The developed method proved effective for the segmentation of the retinal vessels. Higher order tensors along with the implementation of tensor voting via steerable filtering could be employed to further reduce the execution time, and resolve the challenges at vascular junctions. Moreover, the characterization could be advanced to the detection of prolifera- tive retinopathy by extending the supervised learning to include non-proliferative diabetic retinopathy cases or other pathologies. Ultimately, the incorporation of the methods into CAD systems could facilitate screening for the effective reduction of the vision-threatening diabetic retinopathy rates, or the early detection of other than ocular pathologies

Exploring variability in medical imaging

Although recent successes of deep learning and novel machine learning techniques improved the perfor- mance of classification and (anomaly) detection in computer vision problems, the application of these methods in medical imaging pipeline remains a very challenging task. One of the main reasons for this is the amount of variability that is encountered and encapsulated in human anatomy and subsequently reflected in medical images. This fundamental factor impacts most stages in modern medical imaging processing pipelines. Variability of human anatomy makes it virtually impossible to build large datasets for each disease with labels and annotation for fully supervised machine learning. An efficient way to cope with this is to try and learn only from normal samples. Such data is much easier to collect. A case study of such an automatic anomaly detection system based on normative learning is presented in this work. We present a framework for detecting fetal cardiac anomalies during ultrasound screening using generative models, which are trained only utilising normal/healthy subjects. However, despite the significant improvement in automatic abnormality detection systems, clinical routine continues to rely exclusively on the contribution of overburdened medical experts to diagnosis and localise abnormalities. Integrating human expert knowledge into the medical imaging processing pipeline entails uncertainty which is mainly correlated with inter-observer variability. From the per- spective of building an automated medical imaging system, it is still an open issue, to what extent this kind of variability and the resulting uncertainty are introduced during the training of a model and how it affects the final performance of the task. Consequently, it is very important to explore the effect of inter-observer variability both, on the reliable estimation of model’s uncertainty, as well as on the model’s performance in a specific machine learning task. A thorough investigation of this issue is presented in this work by leveraging automated estimates for machine learning model uncertainty, inter-observer variability and segmentation task performance in lung CT scan images. Finally, a presentation of an overview of the existing anomaly detection methods in medical imaging was attempted. This state-of-the-art survey includes both conventional pattern recognition methods and deep learning based methods. It is one of the first literature surveys attempted in the specific research area.Open Acces

Retinal vessel segmentation using textons

Segmenting vessels from retinal images, like segmentation in many other medical image domains, is a challenging task, as there is no unified way that can be adopted to extract the vessels accurately. However, it is the most critical stage in automatic assessment of various forms of diseases (e.g. Glaucoma, Age-related macular degeneration, diabetic retinopathy and cardiovascular diseases etc.). Our research aims to investigate retinal image segmentation approaches based on textons as they provide a compact description of texture that can be learnt from a training set. This thesis presents a brief review of those diseases and also includes their current situations, future trends and techniques used for their automatic diagnosis in routine clinical applications. The importance of retinal vessel segmentation is particularly emphasized in such applications. An extensive review of previous work on retinal vessel segmentation and salient texture analysis methods is presented. Five automatic retinal vessel segmentation methods are proposed in this thesis. The first method focuses on addressing the problem of removing pathological anomalies (Drusen, exudates) for retinal vessel segmentation, which have been identified by other researchers as a problem and a common source of error. The results show that the modified method shows some improvement compared to a previously published method. The second novel supervised segmentation method employs textons. We propose a new filter bank (MR11) that includes bar detectors for vascular feature extraction and other kernels to detect edges and photometric variations in the image. The k-means clustering algorithm is adopted for texton generation based on the vessel and non-vessel elements which are identified by ground truth. The third improved supervised method is developed based on the second one, in which textons are generated by k-means clustering and texton maps representing vessels are derived by back projecting pixel clusters onto hand labelled ground truth. A further step is implemented to ensure that the best combinations of textons are represented in the map and subsequently used to identify vessels in the test set. The experimental results on two benchmark datasets show that our proposed method performs well compared to other published work and the results of human experts. A further test of our system on an independent set of optical fundus images verified its consistent performance. The statistical analysis on experimental results also reveals that it is possible to train unified textons for retinal vessel segmentation. In the fourth method a novel scheme using Gabor filter bank for vessel feature extraction is proposed. The ii method is inspired by the human visual system. Machine learning is used to optimize the Gabor filter parameters. The experimental results demonstrate that our method significantly enhances the true positive rate while maintaining a level of specificity that is comparable with other approaches. Finally, we proposed a new unsupervised texton based retinal vessel segmentation method using derivative of SIFT and multi-scale Gabor filers. The lack of sufficient quantities of hand labelled ground truth and the high level of variability in ground truth labels amongst experts provides the motivation for this approach. The evaluation results reveal that our unsupervised segmentation method is comparable with the best other supervised methods and other best state of the art methods