424 research outputs found
Computational Analysis of Fundus Images: Rule-Based and Scale-Space Models
Fundus images are one of the most important imaging examinations in modern ophthalmology
because they are simple, inexpensive and, above all, noninvasive.
Nowadays, the acquisition and
storage of highresolution
fundus images is relatively easy and fast. Therefore, fundus imaging
has become a fundamental investigation in retinal lesion detection, ocular health monitoring and
screening programmes. Given the large volume and clinical complexity associated with these images,
their analysis and interpretation by trained clinicians becomes a timeconsuming
task and is
prone to human error. Therefore, there is a growing interest in developing automated approaches
that are affordable and have high sensitivity and specificity. These automated approaches need to
be robust if they are to be used in the general population to diagnose and track retinal diseases. To
be effective, the automated systems must be able to recognize normal structures and distinguish
them from pathological clinical manifestations.
The main objective of the research leading to this thesis was to develop automated systems capable
of recognizing and segmenting retinal anatomical structures and retinal pathological clinical
manifestations associated with the most common retinal diseases. In particular, these automated
algorithms were developed on the premise of robustness and efficiency to deal with the difficulties
and complexity inherent in these images. Four objectives were considered in the analysis of
fundus images. Segmentation of exudates, localization of the optic disc, detection of the midline
of blood vessels, segmentation of the vascular network and detection of microaneurysms.
In addition, we also evaluated the detection of diabetic retinopathy on fundus images using the
microaneurysm detection method. An overview of the state of the art is presented to compare the
performance of the developed approaches with the main methods described in the literature for
each of the previously described objectives. To facilitate the comparison of methods, the state of
the art has been divided into rulebased
methods and machine learningbased
methods.
In the research reported in this paper, rulebased
methods based on image processing methods
were preferred over machine learningbased
methods. In particular, scalespace
methods proved
to be effective in achieving the set goals.
Two different approaches to exudate segmentation were developed. The first approach is based on
scalespace
curvature in combination with the local maximum of a scalespace
blob detector and
dynamic thresholds. The second approach is based on the analysis of the distribution function of
the maximum values of the noise map in combination with morphological operators and adaptive
thresholds. Both approaches perform a correct segmentation of the exudates and cope well with
the uneven illumination and contrast variations in the fundus images.
Optic disc localization was achieved using a new technique called cumulative sum fields, which was
combined with a vascular enhancement method. The algorithm proved to be reliable and efficient,
especially for pathological images. The robustness of the method was tested on 8 datasets.
The detection of the midline of the blood vessels was achieved using a modified corner detector
in combination with binary philtres and dynamic thresholding. Segmentation of the vascular network
was achieved using a new scalespace
blood vessels enhancement method. The developed
methods have proven effective in detecting the midline of blood vessels and segmenting vascular
networks.
The microaneurysm detection method relies on a scalespace
microaneurysm detection and labelling
system. A new approach based on the neighbourhood of the microaneurysms was used
for labelling. Microaneurysm detection enabled the assessment of diabetic retinopathy detection.
The microaneurysm detection method proved to be competitive with other methods, especially with highresolution
images. Diabetic retinopathy detection with the developed microaneurysm
detection method showed similar performance to other methods and human experts.
The results of this work show that it is possible to develop reliable and robust scalespace
methods
that can detect various anatomical structures and pathological features of the retina. Furthermore,
the results obtained in this work show that although recent research has focused on machine learning
methods, scalespace
methods can achieve very competitive results and typically have greater
independence from image acquisition. The methods developed in this work may also be relevant
for the future definition of new descriptors and features that can significantly improve the results
of automated methods.As imagens do fundo do olho são hoje um dos principais exames imagiológicos da oftalmologia
moderna, pela sua simplicidade, baixo custo e acima de tudo pelo seu carácter nãoinvasivo.
A
aquisição e armazenamento de imagens do fundo do olho com alta resolução é também relativamente
simples e rápida. Desta forma, as imagens do fundo do olho são um exame fundamental
na identificação de alterações retinianas, monitorização da saúde ocular, e em programas de rastreio.
Considerando o elevado volume e complexidade clínica associada a estas imagens, a análise
e interpretação das mesmas por clínicos treinados tornase
uma tarefa morosa e propensa a erros
humanos. Assim, há um interesse crescente no desenvolvimento de abordagens automatizadas,
acessíveis em custo, e com uma alta sensibilidade e especificidade. Estas devem ser robustas para
serem aplicadas à população em geral no diagnóstico e seguimento de doenças retinianas. Para
serem eficazes, os sistemas de análise têm que conseguir detetar e distinguir estruturas normais
de sinais patológicos.
O objetivo principal da investigação que levou a esta tese de doutoramento é o desenvolvimento
de sistemas automáticos capazes de detetar e segmentar as estruturas anatómicas da retina, e os
sinais patológicos retinianos associados às doenças retinianas mais comuns. Em particular, estes
algoritmos automatizados foram desenvolvidos segundo as premissas de robustez e eficácia para
lidar com as dificuldades e complexidades inerentes a estas imagens.
Foram considerados quatro objetivos de análise de imagens do fundo do olho. São estes, a segmentação
de exsudados, a localização do disco ótico, a deteção da linha central venosa dos vasos
sanguíneos e segmentação da rede vascular, e a deteção de microaneurismas. De acrescentar que
usando o método de deteção de microaneurismas, avaliouse
também a capacidade de deteção da
retinopatia diabética em imagens do fundo do olho.
Para comparar o desempenho das metodologias desenvolvidas neste trabalho, foi realizado um
levantamento do estado da arte, onde foram considerados os métodos mais relevantes descritos na
literatura para cada um dos objetivos descritos anteriormente. Para facilitar a comparação entre
métodos, o estado da arte foi dividido em metodologias de processamento de imagem e baseadas
em aprendizagem máquina.
Optouse
no trabalho de investigação desenvolvido pela utilização de metodologias de análise espacial
de imagem em detrimento de metodologias baseadas em aprendizagem máquina. Em particular,
as metodologias baseadas no espaço de escalas mostraram ser efetivas na obtenção dos
objetivos estabelecidos.
Para a segmentação de exsudados foram usadas duas abordagens distintas. A primeira abordagem
baseiase
na curvatura em espaço de escalas em conjunto com a resposta máxima local de um detetor
de manchas em espaço de escalas e limiares dinâmicos. A segunda abordagem baseiase
na
análise do mapa de distribuição de ruído em conjunto com operadores morfológicos e limiares
adaptativos. Ambas as abordagens fazem uma segmentação dos exsudados de elevada precisão,
além de lidarem eficazmente com a iluminação nãouniforme
e a variação de contraste presente
nas imagens do fundo do olho. A localização do disco ótico foi conseguida com uma nova técnica
designada por campos de soma acumulativos, combinada com métodos de melhoramento da rede
vascular. O algoritmo revela ser fiável e eficiente, particularmente em imagens patológicas. A robustez
do método foi verificada pela sua avaliação em oito bases de dados. A deteção da linha central
dos vasos sanguíneos foi obtida através de um detetor de cantos modificado em conjunto com
filtros binários e limiares dinâmicos. A segmentação da rede vascular foi conseguida com um novo
método de melhoramento de vasos sanguíneos em espaço de escalas. Os métodos desenvolvidos mostraram ser eficazes na deteção da linha central dos vasos sanguíneos e na segmentação da rede
vascular. Finalmente, o método para a deteção de microaneurismas assenta num formalismo de
espaço de escalas na deteção e na rotulagem dos microaneurismas. Para a rotulagem foi utilizada
uma nova abordagem da vizinhança dos candidatos a microaneurismas. A deteção de microaneurismas
permitiu avaliar também a deteção da retinopatia diabética. O método para a deteção
de microaneurismas mostrou ser competitivo quando comparado com outros métodos, em particular
em imagens de alta resolução. A deteção da retinopatia diabética exibiu um desempenho
semelhante a outros métodos e a especialistas humanos.
Os trabalhos descritos nesta tese mostram ser possível desenvolver uma abordagem fiável e robusta
em espaço de escalas capaz de detetar diferentes estruturas anatómicas e sinais patológicos
da retina.
Além disso, os resultados obtidos mostram que apesar de a pesquisa mais recente concentrarse
em metodologias de aprendizagem máquina, as metodologias de análise espacial apresentam
resultados muito competitivos e tipicamente independentes do equipamento de aquisição das imagens.
As metodologias desenvolvidas nesta tese podem ser importantes na definição de novos
descritores e características, que podem melhorar significativamente o resultado de métodos automatizados
Retinal Vessel Segmentation using Tensor Voting
Medical imaging studies generate tremendous amounts of data that are reviewedmanually by physicians every day. Medical image segmentation aims to automate theprocess of extracting (segmenting) “interesting” structures from background structuresin the images, saving physicians time and opening the door to more sophisticatedanalysis such as automatically correlating studies over time. This work focuseson segmenting blood vessels (in particular the retinal vasculature), a task that requiresintegrating both local and global properties of the vasculature to produce goodquality segmentations. We use the Tensor Voting framework as it naturally groupsstructures together based on the consensus of locally voting segments. We investigateseveral ways of encoding the image data as tensors and compare our results quantitativelywith a publically available hand-labeled data set. We demonstrate competitiveperformance versus previously published techniques
Generalizable automated pixel-level structural segmentation of medical and biological data
Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These
solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution.
This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D
structural segmentation in a more generalizable manner, yet has enough adaptability to address
a number of specific image modalities, spanning retinal funduscopy, sequential
fluorescein angiography and two-photon microscopy.
The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based
measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D.
To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective
RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-)
pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations.
Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional
exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this
into consideration, we introduce a 5D orientation mapping to capture these orientation properties.
This mapping is incorporated into the local feature map description prior to a learning
machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods.
For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces
Retinal vessel segmentation using textons
Segmenting vessels from retinal images, like segmentation in many other medical image domains, is a challenging task, as there is no unified way that can be adopted to extract the vessels accurately. However, it is the most critical stage in automatic assessment of various forms of diseases (e.g. Glaucoma, Age-related macular degeneration, diabetic retinopathy and cardiovascular diseases etc.). Our research aims to investigate retinal image segmentation approaches based on textons as they provide a compact description of texture that can be learnt from a training set. This thesis presents a brief review of those diseases and also includes their current situations, future trends and techniques used for their automatic diagnosis in routine clinical applications. The importance of retinal vessel segmentation is
particularly emphasized in such applications. An extensive review of previous work on retinal vessel segmentation and salient texture analysis methods is presented. Five automatic retinal vessel segmentation methods are proposed in this thesis. The first method focuses on addressing the problem of removing pathological anomalies (Drusen, exudates) for retinal vessel segmentation, which have been identified by other researchers as a problem and a common source of error. The results show that the modified method shows some
improvement compared to a previously published method. The second novel supervised segmentation method employs textons. We propose a new filter bank (MR11) that includes bar detectors for vascular feature extraction and other kernels to detect edges and photometric variations in the image. The k-means clustering algorithm is adopted for texton generation based on the vessel and non-vessel elements which are identified by ground truth. The third improved supervised method is developed based on the second one, in which textons are generated by k-means clustering and texton maps representing vessels are derived by back projecting pixel clusters onto hand labelled ground truth. A further step is implemented to ensure that the best combinations of textons are represented in the map and subsequently used to identify vessels in the test set. The experimental results on two benchmark datasets show that our proposed method performs well compared to other published work and the results of human experts. A further test of our system on an independent set of optical fundus images verified its consistent performance. The statistical analysis on experimental results also reveals that it is possible to train unified textons for retinal vessel segmentation. In the fourth method a novel scheme using Gabor filter bank for vessel feature extraction is proposed. The ii method is inspired by the human visual system. Machine learning is used to optimize the
Gabor filter parameters. The experimental results demonstrate that our method significantly enhances the true positive rate while maintaining a level of specificity that is comparable with other approaches. Finally, we proposed a new unsupervised texton based retinal vessel
segmentation method using derivative of SIFT and multi-scale Gabor filers. The lack of sufficient quantities of hand labelled ground truth and the high level of variability in ground truth labels amongst experts provides the motivation for this approach. The evaluation results
reveal that our unsupervised segmentation method is comparable with the best other supervised methods and other best state of the art methods
Segmentation and Characterization of Small Retinal Vessels in Fundus Images Using the Tensor Voting Approach
RÉSUMÉ
La rétine permet de visualiser facilement une partie du réseau vasculaire humain. Elle offre
ainsi un aperçu direct sur le développement et le résultat de certaines maladies liées au réseau
vasculaire dans son entier. Chaque complication visible sur la rétine peut avoir un impact sur
la capacité visuelle du patient. Les plus petits vaisseaux sanguins sont parmi les premières
structures anatomiques affectées par la progression d’une maladie, être capable de les analyser
est donc crucial. Les changements dans l’état, l’aspect, la morphologie, la fonctionnalité, ou
même la croissance des petits vaisseaux indiquent la gravité des maladies.
Le diabète est une maladie métabolique qui affecte des millions de personnes autour
du monde. Cette maladie affecte le taux de glucose dans le sang et cause des changements
pathologiques dans différents organes du corps humain. La rétinopathie diabétique décrit l’en-
semble des conditions et conséquences du diabète au niveau de la rétine. Les petits vaisseaux
jouent un rôle dans le déclenchement, le développement et les conséquences de la rétinopa-
thie. Dans les dernières étapes de cette maladie, la croissance des nouveaux petits vaisseaux,
appelée néovascularisation, présente un risque important de provoquer la cécité. Il est donc
crucial de détecter tous les changements qui ont lieu dans les petits vaisseaux de la rétine
dans le but de caractériser les vaisseaux sains et les vaisseaux anormaux. La caractérisation
en elle-même peut faciliter la détection locale d’une rétinopathie spécifique.
La segmentation automatique des structures anatomiques comme le réseau vasculaire est
une étape cruciale. Ces informations peuvent être fournies à un médecin pour qu’elles soient
considérées lors de son diagnostic. Dans les systèmes automatiques d’aide au diagnostic, le
rôle des petits vaisseaux est significatif. Ne pas réussir à les détecter automatiquement peut
conduire à une sur-segmentation du taux de faux positifs des lésions rouges dans les étapes
ultérieures. Les efforts de recherche se sont concentrés jusqu’à présent sur la localisation
précise des vaisseaux de taille moyenne. Les modèles existants ont beaucoup plus de difficultés
à extraire les petits vaisseaux sanguins. Les modèles existants ne sont pas robustes à la grande
variance d’apparence des vaisseaux ainsi qu’à l’interférence avec l’arrière-plan. Les modèles de
la littérature existante supposent une forme générale qui n’est pas suffisante pour s’adapter
à la largeur étroite et la courbure qui caractérisent les petits vaisseaux sanguins. De plus, le
contraste avec l’arrière-plan dans les régions des petits vaisseaux est très faible. Les méthodes
de segmentation ou de suivi produisent des résultats fragmentés ou discontinus. Par ailleurs,
la segmentation des petits vaisseaux est généralement faite aux dépends de l’amplification
du bruit. Les modèles déformables sont inadéquats pour segmenter les petits vaisseaux. Les
forces utilisées ne sont pas assez flexibles pour compenser le faible contraste, la largeur, et
vii
la variance des vaisseaux. Enfin, les approches de type apprentissage machine nécessitent un
entraînement avec une base de données étiquetée. Il est très difficile d’obtenir ces bases de
données dans le cas des petits vaisseaux.
Cette thèse étend les travaux de recherche antérieurs en fournissant une nouvelle mé-
thode de segmentation des petits vaisseaux rétiniens. La détection de ligne à échelles multiples
(MSLD) est une méthode récente qui démontre une bonne performance de segmentation dans
les images de la rétine, tandis que le vote tensoriel est une méthode proposée pour reconnecter
les pixels. Une approche combinant un algorithme de détection de ligne et de vote tensoriel est
proposée. L’application des détecteurs de lignes a prouvé son efficacité à segmenter les vais-
seaux de tailles moyennes. De plus, les approches d’organisation perceptuelle comme le vote
tensoriel ont démontré une meilleure robustesse en combinant les informations voisines d’une
manière hiérarchique. La méthode de vote tensoriel est plus proche de la perception humain
que d’autres modèles standards. Comme démontré dans ce manuscrit, c’est un outil pour
segmenter les petits vaisseaux plus puissant que les méthodes existantes. Cette combinaison
spécifique nous permet de surmonter les défis de fragmentation éprouvés par les méthodes de
type modèle déformable au niveau des petits vaisseaux. Nous proposons également d’utiliser
un seuil adaptatif sur la réponse de l’algorithme de détection de ligne pour être plus robuste
aux images non-uniformes. Nous illustrons également comment une combinaison des deux
méthodes individuelles, à plusieurs échelles, est capable de reconnecter les vaisseaux sur des
distances variables. Un algorithme de reconstruction des vaisseaux est également proposé.
Cette dernière étape est nécessaire car l’information géométrique complète est requise pour
pouvoir utiliser la segmentation dans un système d’aide au diagnostic.
La segmentation a été validée sur une base de données d’images de fond d’oeil à haute
résolution. Cette base contient des images manifestant une rétinopathie diabétique. La seg-
mentation emploie des mesures de désaccord standards et aussi des mesures basées sur la
perception. En considérant juste les petits vaisseaux dans les images de la base de données,
l’amélioration dans le taux de sensibilité que notre méthode apporte par rapport à la méthode
standard de détection multi-niveaux de lignes est de 6.47%. En utilisant les mesures basées
sur la perception, l’amélioration est de 7.8%.
Dans une seconde partie du manuscrit, nous proposons également une méthode pour
caractériser les rétines saines ou anormales. Certaines images contiennent de la néovascula-
risation. La caractérisation des vaisseaux en bonne santé ou anormale constitue une étape
essentielle pour le développement d’un système d’aide au diagnostic. En plus des défis que
posent les petits vaisseaux sains, les néovaisseaux démontrent eux un degré de complexité
encore plus élevé. Ceux-ci forment en effet des réseaux de vaisseaux à la morphologie com-
plexe et inhabituelle, souvent minces et à fortes courbures. Les travaux existants se limitent
viii
à l’utilisation de caractéristiques de premier ordre extraites des petits vaisseaux segmentés.
Notre contribution est d’utiliser le vote tensoriel pour isoler les jonctions vasculaires et d’uti-
liser ces jonctions comme points d’intérêts. Nous utilisons ensuite une statistique spatiale
de second ordre calculée sur les jonctions pour caractériser les vaisseaux comme étant sains
ou pathologiques. Notre méthode améliore la sensibilité de la caractérisation de 9.09% par
rapport à une méthode de l’état de l’art.
La méthode développée s’est révélée efficace pour la segmentation des vaisseaux réti-
niens. Des tenseurs d’ordre supérieur ainsi que la mise en œuvre d’un vote par tenseur via
un filtrage orientable pourraient être étudiés pour réduire davantage le temps d’exécution et
résoudre les défis encore présents au niveau des jonctions vasculaires. De plus, la caractéri-
sation pourrait être améliorée pour la détection de la rétinopathie proliférative en utilisant
un apprentissage supervisé incluant des cas de rétinopathie diabétique non proliférative ou
d’autres pathologies. Finalement, l’incorporation des méthodes proposées dans des systèmes
d’aide au diagnostic pourrait favoriser le dépistage régulier pour une détection précoce des
rétinopathies et d’autres pathologies oculaires dans le but de réduire la cessité au sein de la
population.----------ABSTRACT
As an easily accessible site for the direct observation of the circulation system, human retina
can offer a unique insight into diseases development or outcome. Retinal vessels are repre-
sentative of the general condition of the whole systematic circulation, and thus can act as
a "window" to the status of the vascular network in the whole body. Each complication on
the retina can have an adverse impact on the patient’s sight. In this direction, small vessels’
relevance is very high as they are among the first anatomical structures that get affected
as diseases progress. Moreover, changes in the small vessels’ state, appearance, morphology,
functionality, or even growth indicate the severity of the diseases.
This thesis will focus on the retinal lesions due to diabetes, a serious metabolic disease
affecting millions of people around the world. This disorder disturbs the natural blood glucose
levels causing various pathophysiological changes in different systems across the human body.
Diabetic retinopathy is the medical term that describes the condition when the fundus and
the retinal vessels are affected by diabetes. As in other diseases, small vessels play a crucial
role in the onset, the development, and the outcome of the retinopathy. More importantly,
at the latest stage, new small vessels, or neovascularizations, growth constitutes a factor of
significant risk for blindness. Therefore, there is a need to detect all the changes that occur
in the small retinal vessels with the aim of characterizing the vessels to healthy or abnormal.
The characterization, in turn, can facilitate the detection of a specific retinopathy locally,
like the sight-threatening proliferative diabetic retinopathy.
Segmentation techniques can automatically isolate important anatomical structures like
the vessels, and provide this information to the physician to assist him in the final decision. In
comprehensive systems for the automatization of DR detection, small vessels role is significant
as missing them early in a CAD pipeline might lead to an increase in the false positive rate
of red lesions in subsequent steps. So far, the efforts have been concentrated mostly on the
accurate localization of the medium range vessels. In contrast, the existing models are weak
in case of the small vessels. The required generalization to adapt an existing model does not
allow the approaches to be flexible, yet robust to compensate for the increased variability in
the appearance as well as the interference with the background. So far, the current template
models (matched filtering, line detection, and morphological processing) assume a general
shape for the vessels that is not enough to approximate the narrow, curved, characteristics
of the small vessels. Additionally, due to the weak contrast in the small vessel regions,
the current segmentation and the tracking methods produce fragmented or discontinued
results. Alternatively, the small vessel segmentation can be accomplished at the expense of
x
background noise magnification, in the case of using thresholding or the image derivatives
methods. Furthermore, the proposed deformable models are not able to propagate a contour
to the full extent of the vasculature in order to enclose all the small vessels. The deformable
model external forces are ineffective to compensate for the low contrast, the low width, the
high variability in the small vessel appearance, as well as the discontinuities. Internal forces,
also, are not able to impose a global shape constraint to the contour that could be able to
approximate the variability in the appearance of the vasculature in different categories of
vessels. Finally, machine learning approaches require the training of a classifier on a labelled
set. Those sets are difficult to be obtained, especially in the case of the smallest vessels. In
the case of the unsupervised methods, the user has to predefine the number of clusters and
perform an effective initialization of the cluster centers in order to converge to the global
minimum.
This dissertation expanded the previous research work and provides a new segmentation
method for the smallest retinal vessels. Multi-scale line detection (MSLD) is a recent method
that demonstrates good segmentation performance in the retinal images, while tensor voting
is a method first proposed for reconnecting pixels. For the first time, we combined the
line detection with the tensor voting framework. The application of the line detectors has
been proved an effective way to segment medium-sized vessels. Additionally, perceptual
organization approaches like tensor voting, demonstrate increased robustness by combining
information coming from the neighborhood in a hierarchical way. Tensor voting is closer than
standard models to the way human perception functions. As we show, it is a more powerful
tool to segment small vessels than the existing methods. This specific combination allows us
to overcome the apparent fragmentation challenge of the template methods at the smallest
vessels. Moreover, we thresholded the line detection response adaptively to compensate for
non-uniform images. We also combined the two individual methods in a multi-scale scheme
in order to reconnect vessels at variable distances. Finally, we reconstructed the vessels
from their extracted centerlines based on pixel painting as complete geometric information
is required to be able to utilize the segmentation in a CAD system.
The segmentation was validated on a high-resolution fundus image database that in-
cludes diabetic retinopathy images of varying stages, using standard discrepancy as well as
perceptual-based measures. When only the smallest vessels are considered, the improve-
ments in the sensitivity rate for the database against the standard multi-scale line detection
method is 6.47%. For the perceptual-based measure, the improvement is 7.8% against the
basic method.
The second objective of the thesis was to implement a method for the characterization of
isolated retinal areas into healthy or abnormal cases. Some of the original images, from which
xi
these patches are extracted, contain neovascularizations. Investigation of image features
for the vessels characterization to healthy or abnormal constitutes an essential step in the
direction of developing CAD system for the automatization of DR screening. Given that the
amount of data will significantly increase under CAD systems, the focus on this category of
vessels can facilitate the referral of sight-threatening cases to early treatment. In addition
to the challenges that small healthy vessels pose, neovessels demonstrate an even higher
degree of complexity as they form networks of convolved, twisted, looped thin vessels. The
existing work is limited to the use of first-order characteristics extracted from the small
segmented vessels that limits the study of patterns. Our contribution is in using the tensor
voting framework to isolate the retinal vascular junctions and in turn using those junctions
as points of interests. Second, we exploited second-order statistics computed on the junction
spatial distribution to characterize the vessels as healthy or neovascularizations. In fact, the
second-order spatial statistics extracted from the junction distribution are combined with
widely used features to improve the characterization sensitivity by 9.09% over the state of
art.
The developed method proved effective for the segmentation of the retinal vessels. Higher
order tensors along with the implementation of tensor voting via steerable filtering could
be employed to further reduce the execution time, and resolve the challenges at vascular
junctions. Moreover, the characterization could be advanced to the detection of prolifera-
tive retinopathy by extending the supervised learning to include non-proliferative diabetic
retinopathy cases or other pathologies. Ultimately, the incorporation of the methods into
CAD systems could facilitate screening for the effective reduction of the vision-threatening
diabetic retinopathy rates, or the early detection of other than ocular pathologies
Diabetic retinopathy diagnosis through multi-agent approaches
Programa Doutoral em Engenharia BiomédicaDiabetic retinopathy has been revealed as a serious public health problem in occidental
world, since it is the most common cause of vision impairment among people
of working age. The early diagnosis and an adequate treatment can prevent loss
of vision. Thus, a regular screening program to detect diabetic retinopathy in the
early stages could be efficient for the prevention of blindness. Due to its characteristics,
digital color fundus photographs have been the preferred eye examination
method adopted in these programs. Nevertheless, due to the growing incidence of
diabetes in population, ophthalmologists have to observe a huge number of images.
Therefore, the development of computational tools that can assist the diagnosis is
of major importance. Several works have been published in the recent past years
for this purpose; but an automatic system for clinical practice has yet to come. In
general, these algorithms are used to normalize, segment and extract information
from images to be utilized by classifiers which aim to classify the regions of the
fundus image. These methods are mostly based on global approaches that cannot
be locally adapted to the image properties and therefore, none of them perform as
needed because of fundus images complexity.
This thesis focuses on the development of new tools based on multi-agent approaches,
to assist the diabetic retinopathy early diagnosis. The fundus image automatic
segmentation concerning the diabetic retinopathy diagnosis should comprise both
pathological (dark and bright lesions) and anatomical features (optic disc, blood
vessels and fovea). In that way, systems for the optic disc detection, bright lesions
segmentation, blood vessels segmentation and dark lesions segmentation were implemented
and, when possible, compared to those approaches already described in
literature. Two kinds of agent based systems were investigated and applied to digital
color fundus photographs: ant colony system and multi-agent system composed of
reactive agents with interaction mechanisms between them. The ant colony system
was used to the optic disc detection and for bright lesion segmentation. Multi-agent
system models were developed for the blood vessel segmentation and for small dark
lesion segmentation. The multi-agent system models created in this study are not
image processing techniques on their own, but they are used as tools to improve
the traditional algorithms results at the micro level. The results of all the proposed approaches are very promising and reveal that the systems created perform better
than other recent methods described in the literature.
Therefore, the main scientific contribution of this thesis is to prove that multi-agent
systems based approaches can be efficient in segmenting structures in retinal images.
Such an approach overcomes the classic image processing algorithms that are limited
to macro results and do not consider the local characteristics of images. Hence,
multi-agent systems based approaches could be a fundamental tool, responsible for
a very efficient system development to be used in screening programs concerning
diabetic retinopathy early diagnosis.A retinopatia diabética tem-se revelado como um problema sério de saúde pública
no mundo ocidental, uma vez que é a principal causa de cegueira entre as pessoas
em idade ativa. Contudo, a perda de visão pode ser prevenida através da deteção
precoce da doença e de um tratamento adequado. Por isso, um programa regular
de rastreio e monitorização da retinopatia diabética pode ser eficiente na prevenção
da deterioração da visão. Devido às suas características, a fotografia digital colorida
do fundo do olho tem sido o exame adotado neste tipo de programas. No entanto,
devido ao aumento da incidência da diabetes na população, o número de imagens
a serem analisadas pelos oftalmologistas é elevado. Assim sendo, é muito importante
o desenvolvimento de ferramentas computacionais para auxiliar no diagnóstico
desta patologia. Nos últimos anos, têm sido vários os trabalhos publicados com
este propósito; porém, não existe ainda um sistema automático (ou recomendável)
para ser usado nas práticas clínicas. No geral, estes algoritmos são usados para
normalizar, segmentar e extrair informação das imagens que vai ser utilizada por
classificadores, cujo objetivo é identificar as regiões da imagem que se procuram.
Estes métodos são maioritariamente baseados em abordagens globais que não podem
ser localmente adaptadas às propriedades das imagens e, portanto, nenhum
apresenta a performance necessária devido à complexidade das imagens do fundo do
olho.
Esta tese foca-se no desenvolvimento de novas ferramentas computacionais baseadas
em sistemas multi-agente, para auxiliar na deteção precoce da retinopatia diabética.
A segmentação automática das imagens do fundo do olho com o objetivo de diagnosticar
a retinopatia diabética, deve englobar características patológicas (lesões claras
e escuras) e anatómicas (disco ótico, vasos sanguíneos e fóvea). Deste modo, foram
criados sistemas para a deteção do disco ótico e para a segmentação das lesões claras,
dos vasos sanguíneos e das lesões escuras e, quando possível, estes foram comparados
com abordagens já descritas na literatura. Dois tipos de sistemas baseados em
agentes foram investigados e aplicados nas imagens digitais coloridas do fundo do
olho: sistema de colónia de formigas e sistema multi-agente constituído por agentes
reativos e com mecanismos de interação entre eles. O sistema de colónia de formigas
foi usado para a deteção do disco ótico e para a segmentação das lesões claras. Modelos de sistemas multi-agente foram desenvolvidos para a segmentação dos vasos
sanguíneos e das lesões escuras. Os modelos multi-agentes criados ao longo deste
estudo não são por si só técnicas de processamento de imagem, mas são sim usados
como ferramentas para melhorar os resultados dos algoritmos tradicionais no baixo
nível. Os resultados de todas as abordagens propostas são muito promissores e revelam
que os sistemas criados apresentam melhor performance que outras abordagens
recentes descritas na literatura.
Posto isto, a maior contribuição científica desta tese é provar que abordagens baseadas
em sistemas multi-agente podem ser eficientes na segmentação de estruturas em imagens
da retina. Uma abordagem deste tipo ultrapassa os algoritmos clássicos de
processamento de imagem, que se limitam aos resultados de alto nível e não têm em
consideração as propriedades locais das imagens. Portanto, as abordagens baseadas
em sistemas multi-agente podem ser uma ferramenta fundamental, responsável pelo
desenvolvimento de um sistema eficiente para ser usado nos programas de rastreio
e monitorização da retinopatia diabética.Work supported by FEDER funds through the "Programa Operacional Factores de Competitividade – COMPETE" and by national funds by FCT- Fundação para a Ciência e a Tecnologia. C. Pereira thanks the FCT for the SFRH / BD / 61829 / 2009 grant
Automatic computation of the arteriovenous ratio and assessment of its effectiveness as a prognostic indicator in hypertension
[Resumen] La retina es la única parte del cuerpo humano en donde se pueden observar los vasos sanguíneos directamente de una forma no invasiva mediante un examen de fondo de ojo. De esta manera, la imagen de la retina mediante las técnicas de procesamiento de imágenes se convirtió en un campo de clave para el diagnóstico precoz de varias enfermedades sistémicas que provocan alteraciones visibles en dicha imagen. Así, alteraciones en el ancho de los vasos retinianos se asocian con patologías tales como diabetes o hipertensión. De hecho, el estrechamiento de las arterias constituye un indicio precoz de la hipertensión arterial sistémica, siendo una característica del grado I de la retinopatía hipertensiva de acuerdo con la clasificación de Keith-Wagener-Barker. En este sentido, se han realizado esfuerzos para desarrollar programas asistidos por ordenador para medir con precisión los cambios en el ancho de los vasos a través del índice arteriovenoso (IAV), es decir, la relación entre los calibres de las arterias y las venas. Sin embargo, aunque estos sistemas se han usado en muchos estudios con fines de investigación, su aplicabilidad en la práctica clínica diaria es todavía discutida. En este trabajo, se propone una nueva metodología para el cálculo del IAV con el fin de estratificar el riesgo cardiovascular de los hipertensos. Por un lado, se ha desarrollado un método completamente automático para estimar el IAV en una imagen de fondo de ojo de un paciente. Por otro lado, se propone un sistema para monitorizar el IAV del paciente a lo largo del tiempo. Para este fin, las mediciones del IAV en las diferentes imágenes adquiridas sobre el mismo ojo del paciente en diferentes fechas se estiman usando el mismo conjunto de vasos medidos en las mismas áreas. Por lo tanto, la mediciones obtenidos de esta manera son comparables y precisas, debido a que son independientes en el conjunto de vasos seleccionados para el cálculo. Las dos técnicas se han integrado en SIRIUS, un sistema web destinado a incluir diferentes servicios en el campo del análisis de la imagen retiniana. El sistema incluye también gestión de pacientes y revisiones, lo que facilita el análisis de las lesiones retinianas causadas por diferentes patologías y su evolución después de un determinado tratamiento. Además al ser una aplicación distribuída a través de la web, proporciona un entorno de colaboración entre diferentes médicos, investigadores y centros.[Resumo] A retina é a única parte do corpo humano onde se poden observar os vasos sanguíneos directamente dunha maneira non invasiva mediante un examen do fondo do ollo. Desta maneira, a imaxe da retina mediante as técnicas de procesamento de imáxenes converteuse nun campo chave para o diagnóstico precoz de varias enfermidades sistémicas que provocan alteracións visibles en dita imaxe. Así, cambios no ancho dos vasos retinianos asócianse con patoloxías tales como a diabetes ou a hipertensión. De feito, o estreitamento das arterias constitúe un indicio prematuro da hipertensión arterial sistémica, sendo unha característica do grado I da retinopatía hipertensiva dacordo coa clasificación de Keith- Wagener-Barker. Neste sentido, fixerónse moitos esforzos para desenvolver programas asistidos por ordenador para medir con precisión os cambios no ancho dos vasos a través do índice arteriovenoso (IAV), é dicir, a relación entre os calibres das arterias e das veas. Nembargantes, aínda que estes sistemas foron usados en moitos estudios con fins investigadores, a sua aplicabilidade na práctica clínica diaria aínda é discutida. Neste traballo, proponse unha nova metodoloxía para o cálculo do IAV co fin de estratificar o risco cardiovascular dos hipertensos. Por un lado, desenvolveuse un método completamente automático para estimar o IAV nunha imaxe de fondo de ollo dun doente. Por outra banda, proponse un sistema para monitorizar o IAV dun doente a lo longo do tempo. Para isto, as medicións do IAV nas diferentes imaxes adquiridas sobre o mesmo ollo do doente en diferentes datas fanse usando o mesmo conxunto de vasos medidos nas mesmas áreas. Polo tanto, as medicións obtidas desta maneira son comparables e precisas, debido a que son independentes do conxunto de vasos seleccionados para o cálculo. As dúas técnicas foron integradas no SIRIUS, un sistema web destinado a incluir diferentes servicios no campo da análise da imaxe retiniana. O sistema inclúe tamén xestión de doentes e revisións, facilitando a análise e estudo das lesións retinianas causadas por diferentes patoloxías e a súa evolución despois dun determinado tratamento. Ademais ao ser unha aplicación distribuída a través da web, proporciona un entorno de colaboración entre diferentes médicos, investigadores e centros.[Abstract] Retina is the only part in the human body where blood vessels can be directly observed in a non-invasive way through an eye fundus examination. In this manner, the retinal imaging assisted by image processing techniques became a key field for the early diagnosis of several systemic diseases which cause visible alterations in the fundus image. Thus, changes in the retinal vessel widths are associated with pathologies such as diabetes or hypertension. In fact, arteriolar narrowing constitutes an early sign of systemic hypertension, being a feature for the grade I of hypertension retinopathy according to Keith-Wagener-Barker classification. In this sense, some efforts have been made to develop computer-assisted programs to measure accurately abnormalities in the vessel widths through the arteriovenous ratio (AVR), that is, the relation between arteriolar and venular vessel widths. However, although these systems have been used in many studies for research purposes, their applicability to daily clinical practice is yet discussed. In this work, a new methodology for the AVR computation is proposed in order to stratify the cardiovascular risk of hypertension. On one hand, a fully automatic method to estimate the AVR in a sample patient's image is developed. On the other hand, an AVR monitoring system to compute the patient's AVR over time was implemented. To this end, the AVR measurements computed in the different patient's images acquired from the same eye at different dates, uses the same set of vessels measured at the same areas. Thus, the measurements achieved in this manner are comparable and precise due to they are independent on the set of vessels selected for the calculus. The two approaches have been integrated in SIRIUS, a web-based system aimed to include different services in the field of retinal image analysis. It includes patient and checkup management, making easier to analyze the retinal lesions caused by different pathologies and their evolution after a specific treatment. Moreover, being a application distributed via the web, it provides a collaborative environment among different physicians, researchers and medical centers
Modeling, Pattern Analysis and Feature-Based Retrieval on Retinal Images
Inexpensive high quality fundus camera systems enable imaging of retina for vision related health management and diagnosis at large scale. A computer based analysis system can help establish the general baseline of normal conditions vs. anomalous ones, so that different classes of retinal conditions can be classified. Advanced applications, ranging from disease screening algorithms, aging vs. disease trend modeling and prediction, and content-based retrieval systems can be developed.
In this dissertation, I propose an analytical framework for the modeling of retina blood vessels to capture their statistical properties, so that based on these properties one can develop blood vessel mapping algorithms with self-optimized parameters. Then, other image objects can be registered based on vascular topology modeling techniques. On the basis of these low level analytical models and algorithms, the third major element of this dissertation is a high level population statistics application, in which texture classification of macular patterns is correlated with vessel structures, which can also be used for retinal image retrieval. The analytical models have been implemented and tested based on various image sources. Some of the algorithms have been used for clinical tests.
The major contributions of this dissertation are summarized as follows: (1) A concise, accurate feature representation of retinal blood vessel on retinal images by proposing two feature descriptors Sp and Ep derived from radial contrast transform. (2) A new statistical model of lognormal distribution, which captures the underlying physical property of the levels of generations of the vascular network on retinal images. (3) Fast and accurate detection algorithms for retinal objects, which include retinal blood vessel, macular-fovea area and optic disc, and (4) A novel population statistics based modeling technique for correlation analysis of blood vessels and other image objects that only exhibit subtle texture changes
Automatic analysis of retinal images to aid in the diagnosis and grading of diabetic retinopathy
Diabetic retinopathy (DR) is the most common complication of diabetes mellitus and one of the leading causes of preventable blindness in the adult working population. Visual loss can be prevented from the early stages of DR, when the treatments are effective. Therefore, early diagnosis is paramount. However, DR may be clinically asymptomatic until the advanced stage, when vision is already affected and treatment may become difficult. For this reason, diabetic patients should undergo regular eye examinations through screening programs. Traditionally, DR screening programs are run by trained specialists through visual inspection of the retinal images. However, this manual analysis is time consuming and expensive. With the increasing incidence of diabetes and the limited number of clinicians and sanitary resources, the early detection of DR becomes non-viable. For this reason, computed-aided diagnosis (CAD) systems are required to assist specialists for a fast, reliable diagnosis, allowing to reduce the workload and the associated costs.
We hypothesize that the application of novel, automatic algorithms for fundus image analysis could contribute to the early diagnosis of DR. Consequently, the main objective of the present Doctoral Thesis is to study, design and develop novel methods based on the automatic analysis of fundus images to aid in the screening, diagnosis, and treatment of DR.
In order to achieve the main goal, we built a private database and used five retinal public databases: DRIMDB, DIARETDB1, DRIVE, Messidor and Kaggle. The stages of fundus image processing covered in this Thesis are: retinal image quality assessment (RIQA), the location of the optic disc (OD) and the fovea, the segmentation of RLs and EXs, and the DR severity grading.
RIQA was studied with two different approaches. The first approach was based on the combination of novel, global features. Results achieved 91.46% accuracy, 92.04% sensitivity, and 87.92% specificity using the private database. We developed a second approach aimed at RIQA based on deep learning. We achieved 95.29% accuracy with the private database and 99.48% accuracy with the DRIMDB database.
The location of the OD and the fovea was performed using a combination of saliency maps. The proposed methods were evaluated over the private database and the public databases DRIVE, DIARETDB1 and Messidor. For the OD, we achieved 100% accuracy for all databases except Messidor (99.50%). As for the fovea location, we also reached 100% accuracy for all databases except Messidor (99.67%).
The joint segmentation of RLs and EXs was accomplished by decomposing the fundus image into layers. Results were computed per pixel and per image. Using the private database, 88.34% per-image accuracy (ACCi) was reached for the RL detection and 95.41% ACCi for EX detection. An additional method was proposed for the segmentation of RLs based on superpixels. Evaluating this method with the private database, we obtained 84.45% ACCi. Results were validated using the DIARETDB1 database.
Finally, we proposed a deep learning framework for the automatic DR severity grading. The method was based on a novel attention mechanism which performs a separate attention of the dark and the bright structures of the retina. The Kaggle DR detection dataset was used for development and validation. The International Clinical DR Scale was considered, which is made up of 5 DR severity levels. Classification results for all classes achieved 83.70% accuracy and a Quadratic Weighted Kappa of 0.78.
The methods proposed in this Doctoral Thesis form a complete, automatic DR screening system, contributing to aid in the early detection of DR. In this way, diabetic patients could receive better attention for their ocular health avoiding vision loss. In addition, the workload of specialists could be relieved while healthcare costs are reduced.La retinopatía diabética (RD) es la complicación más común de la diabetes mellitus y una de las principales causas de ceguera prevenible en la población activa adulta. El diagnóstico precoz es primordial para prevenir la pérdida visual. Sin embargo, la RD es clínicamente asintomática hasta etapas avanzadas, cuando la visión ya está afectada. Por eso, los pacientes diabéticos deben someterse a exámenes oftalmológicos periódicos a través de programas de cribado. Tradicionalmente, estos programas están a cargo de especialistas y se basan de la inspección visual de retinografías. Sin embargo, este análisis manual requiere mucho tiempo y es costoso. Con la creciente incidencia de la diabetes y la escasez de recursos sanitarios, la detección precoz de la RD se hace inviable. Por esta razón, se necesitan sistemas de diagnóstico asistido por ordenador (CAD) que ayuden a los especialistas a realizar un diagnóstico rápido y fiable, que permita reducir la carga de trabajo y los costes asociados.
El objetivo principal de la presente Tesis Doctoral es estudiar, diseñar y desarrollar nuevos métodos basados en el análisis automático de retinografías para ayudar en el cribado, diagnóstico y tratamiento de la RD. Las etapas estudiadas fueron: la evaluación de la calidad de la imagen retiniana (RIQA), la localización del disco óptico (OD) y la fóvea, la segmentación de RL y EX y la graduación de la severidad de la RD.
RIQA se estudió con dos enfoques diferentes. El primer enfoque se basó en la combinación de características globales. Los resultados lograron una precisión del 91,46% utilizando la base de datos privada. El segundo enfoque se basó en aprendizaje profundo. Logramos un 95,29% de precisión con la base de datos privada y un 99,48% con la base de datos DRIMDB.
La localización del OD y la fóvea se realizó mediante una combinación de mapas de saliencia. Los métodos propuestos fueron evaluados sobre la base de datos privada y las bases de datos públicas DRIVE, DIARETDB1 y Messidor. Para el OD, logramos una precisión del 100% para todas las bases de datos excepto Messidor (99,50%). En cuanto a la ubicación de la fóvea, también alcanzamos un 100% de precisión para todas las bases de datos excepto Messidor (99,67%).
La segmentación conjunta de RL y EX se logró descomponiendo la imagen del fondo de ojo en capas. Utilizando la base de datos privada, se alcanzó un 88,34% de precisión por imagen (ACCi) para la detección de RL y un 95,41% de ACCi para la detección de EX. Se propuso un método adicional para la segmentación de RL basado en superpíxeles. Evaluando este método con la base de datos privada, obtuvimos 84.45% ACCi. Los resultados se validaron utilizando la base de datos DIARETDB1.
Finalmente, propusimos un método de aprendizaje profundo para la graduación automática de la gravedad de la DR. El método se basó en un mecanismo de atención. Se utilizó la base de datos Kaggle y la Escala Clínica Internacional de RD (5 niveles de severidad). Los resultados de clasificación para todas las clases alcanzaron una precisión del 83,70% y un Kappa ponderado cuadrático de 0,78.
Los métodos propuestos en esta Tesis Doctoral forman un sistema completo y automático de cribado de RD, contribuyendo a ayudar en la detección precoz de la RD. De esta forma, los pacientes diabéticos podrían recibir una mejor atención para su salud ocular evitando la pérdida de visión. Además, se podría aliviar la carga de trabajo de los especialistas al mismo tiempo que se reducen los costes sanitarios.Escuela de DoctoradoDoctorado en Tecnologías de la Información y las Telecomunicacione
- …