Multi-constraints based deep learning model for automated segmentation and diagnosis of coronary artery disease in X-ray angiographic images

Background: The detection of coronary artery disease (CAD) from the X-ray coronary angiography is a crucial process which is hindered by various issues such as presence of noise, insufficient contrast of the input images along with the uncertainties caused by the motion due to respiration and variation of angles of vessels. Methods: In this article, an Automated Segmentation and Diagnosis of Coronary Artery Disease (ASCARIS) model is proposed in order to overcome the prevailing challenges in detection of CAD from the X-ray images. Initially, the preprocessing of the input images was carried out by using the modified wiener filter for the removal of both internal and external noise pixels from the images. Then, the enhancement of contrast was carried out by utilizing the optimized maximum principal curvature to preserve the edge information thereby contributing to increasing the segmentation accuracy. Further, the binarization of enhanced images was executed by the means of OTSU thresholding. The segmentation of coronary arteries was performed by implementing the Attention-based Nested U-Net, in which the attention estimator was incorporated to overcome the difficulties caused by intersections and overlapped arteries. The increased segmentation accuracy was achieved by performing angle estimation. Finally, the VGG-16 based architecture was implemented to extract threefold features from the segmented image to perform classification of X-ray images into normal and abnormal classes. Results: The experimentation of the proposed ASCARIS model was carried out in the MATLAB R2020a simulation tool and the evaluation of the proposed model was compared with several existing approaches in terms of accuracy, sensitivity, specificity, revised contrast to noise ratio, mean square error, dice coefficient, Jaccard similarity, Hausdorff distance, Peak signal-to-noise ratio (PSNR), segmentation accuracy and ROC curve. Discussion: The results obtained conclude that the proposed model outperforms the existing approaches in all the evaluation metrics thereby achieving optimized classification of CAD. The proposed method removes the large number of background artifacts and obtains a better vascular structure

Automatic Spatiotemporal Analysis of Cardiac Image Series

RÉSUMÉ À ce jour, les maladies cardiovasculaires demeurent au premier rang des principales causes de décès en Amérique du Nord. Chez l’adulte et au sein de populations de plus en plus jeunes, la soi-disant épidémie d’obésité entraînée par certaines habitudes de vie tels que la mauvaise alimentation, le manque d’exercice et le tabagisme est lourde de conséquences pour les personnes affectées, mais aussi sur le système de santé. La principale cause de morbidité et de mortalité chez ces patients est l’athérosclérose, une accumulation de plaque à l’intérieur des vaisseaux sanguins à hautes pressions telles que les artères coronaires. Les lésions athérosclérotiques peuvent entraîner l’ischémie en bloquant la circulation sanguine et/ou en provoquant une thrombose. Cela mène souvent à de graves conséquences telles qu’un infarctus. Outre les problèmes liés à la sténose, les parois artérielles des régions criblées de plaque augmentent la rigidité des parois vasculaires, ce qui peut aggraver la condition du patient. Dans la population pédiatrique, la pathologie cardiovasculaire acquise la plus fréquente est la maladie de Kawasaki. Il s’agit d’une vasculite aigüe pouvant affecter l’intégrité structurale des parois des artères coronaires et mener à la formation d’anévrismes. Dans certains cas, ceux-ci entravent l’hémodynamie artérielle en engendrant une perfusion myocardique insuffisante et en activant la formation de thromboses. Le diagnostic de ces deux maladies coronariennes sont traditionnellement effectués à l’aide d’angiographies par fluoroscopie. Pendant ces examens paracliniques, plusieurs centaines de projections radiographiques sont acquises en séries suite à l’infusion artérielle d’un agent de contraste. Ces images révèlent la lumière des vaisseaux sanguins et la présence de lésions potentiellement pathologiques, s’il y a lieu. Parce que les séries acquises contiennent de l’information très dynamique en termes de mouvement du patient volontaire et involontaire (ex. battements cardiaques, respiration et déplacement d’organes), le clinicien base généralement son interprétation sur une seule image angiographique où des mesures géométriques sont effectuées manuellement ou semi-automatiquement par un technicien en radiologie. Bien que l’angiographie par fluoroscopie soit fréquemment utilisé partout dans le monde et souvent considéré comme l’outil de diagnostic “gold-standard” pour de nombreuses maladies vasculaires, la nature bidimensionnelle de cette modalité d’imagerie est malheureusement très limitante en termes de spécification géométrique des différentes régions pathologiques. En effet, la structure tridimensionnelle des sténoses et des anévrismes ne peut pas être pleinement appréciée en 2D car les caractéristiques observées varient selon la configuration angulaire de l’imageur. De plus, la présence de lésions affectant les artères coronaires peut ne pas refléter la véritable santé du myocarde, car des mécanismes compensatoires naturels (ex. vaisseaux----------ABSTRACT Cardiovascular disease continues to be the leading cause of death in North America. In adult and, alarmingly, ever younger populations, the so-called obesity epidemic largely driven by lifestyle factors that include poor diet, lack of exercise and smoking, incurs enormous stresses on the healthcare system. The primary cause of serious morbidity and mortality for these patients is atherosclerosis, the build up of plaque inside high pressure vessels like the coronary arteries. These lesions can lead to ischemic disease and may progress to precarious blood flow blockage or thrombosis, often with infarction or other severe consequences. Besides the stenosis-related outcomes, the arterial walls of plaque-ridden regions manifest increased stiffness, which may exacerbate negative patient prognosis. In pediatric populations, the most prevalent acquired cardiovascular pathology is Kawasaki disease. This acute vasculitis may affect the structural integrity of coronary artery walls and progress to aneurysmal lesions. These can hinder the blood flow’s hemodynamics, leading to inadequate downstream perfusion, and may activate thrombus formation which may lead to precarious prognosis. Diagnosing these two prominent coronary artery diseases is traditionally performed using fluoroscopic angiography. Several hundred serial x-ray projections are acquired during selective arterial infusion of a radiodense contrast agent, which reveals the vessels’ luminal area and possible pathological lesions. The acquired series contain highly dynamic information on voluntary and involuntary patient movement: respiration, organ displacement and heartbeat, for example. Current clinical analysis is largely limited to a single angiographic image where geometrical measures will be performed manually or semi-automatically by a radiological technician. Although widely used around the world and generally considered the gold-standard diagnosis tool for many vascular diseases, the two-dimensional nature of this imaging modality is limiting in terms of specifying the geometry of various pathological regions. Indeed, the 3D structures of stenotic or aneurysmal lesions may not be fully appreciated in 2D because their observable features are dependent on the angular configuration of the imaging gantry. Furthermore, the presence of lesions in the coronary arteries may not reflect the true health of the myocardium, as natural compensatory mechanisms may obviate the need for further intervention. In light of this, cardiac magnetic resonance perfusion imaging is increasingly gaining attention and clinical implementation, as it offers a direct assessment of myocardial tissue viability following infarction or suspected coronary artery disease. This type of modality is plagued, however, by motion similar to that present in fluoroscopic imaging. This issue predisposes clinicians to laborious manual intervention in order to align anatomical structures in sequential perfusion frames, thus hindering automation o

Automatic Spatiotemporal Analysis of Cardiac Image Series

RÉSUMÉ À ce jour, les maladies cardiovasculaires demeurent au premier rang des principales causes de décès en Amérique du Nord. Chez l’adulte et au sein de populations de plus en plus jeunes, la soi-disant épidémie d’obésité entraînée par certaines habitudes de vie tels que la mauvaise alimentation, le manque d’exercice et le tabagisme est lourde de conséquences pour les personnes affectées, mais aussi sur le système de santé. La principale cause de morbidité et de mortalité chez ces patients est l’athérosclérose, une accumulation de plaque à l’intérieur des vaisseaux sanguins à hautes pressions telles que les artères coronaires. Les lésions athérosclérotiques peuvent entraîner l’ischémie en bloquant la circulation sanguine et/ou en provoquant une thrombose. Cela mène souvent à de graves conséquences telles qu’un infarctus. Outre les problèmes liés à la sténose, les parois artérielles des régions criblées de plaque augmentent la rigidité des parois vasculaires, ce qui peut aggraver la condition du patient. Dans la population pédiatrique, la pathologie cardiovasculaire acquise la plus fréquente est la maladie de Kawasaki. Il s’agit d’une vasculite aigüe pouvant affecter l’intégrité structurale des parois des artères coronaires et mener à la formation d’anévrismes. Dans certains cas, ceux-ci entravent l’hémodynamie artérielle en engendrant une perfusion myocardique insuffisante et en activant la formation de thromboses. Le diagnostic de ces deux maladies coronariennes sont traditionnellement effectués à l’aide d’angiographies par fluoroscopie. Pendant ces examens paracliniques, plusieurs centaines de projections radiographiques sont acquises en séries suite à l’infusion artérielle d’un agent de contraste. Ces images révèlent la lumière des vaisseaux sanguins et la présence de lésions potentiellement pathologiques, s’il y a lieu. Parce que les séries acquises contiennent de l’information très dynamique en termes de mouvement du patient volontaire et involontaire (ex. battements cardiaques, respiration et déplacement d’organes), le clinicien base généralement son interprétation sur une seule image angiographique où des mesures géométriques sont effectuées manuellement ou semi-automatiquement par un technicien en radiologie. Bien que l’angiographie par fluoroscopie soit fréquemment utilisé partout dans le monde et souvent considéré comme l’outil de diagnostic “gold-standard” pour de nombreuses maladies vasculaires, la nature bidimensionnelle de cette modalité d’imagerie est malheureusement très limitante en termes de spécification géométrique des différentes régions pathologiques. En effet, la structure tridimensionnelle des sténoses et des anévrismes ne peut pas être pleinement appréciée en 2D car les caractéristiques observées varient selon la configuration angulaire de l’imageur. De plus, la présence de lésions affectant les artères coronaires peut ne pas refléter la véritable santé du myocarde, car des mécanismes compensatoires naturels (ex. vaisseaux----------ABSTRACT Cardiovascular disease continues to be the leading cause of death in North America. In adult and, alarmingly, ever younger populations, the so-called obesity epidemic largely driven by lifestyle factors that include poor diet, lack of exercise and smoking, incurs enormous stresses on the healthcare system. The primary cause of serious morbidity and mortality for these patients is atherosclerosis, the build up of plaque inside high pressure vessels like the coronary arteries. These lesions can lead to ischemic disease and may progress to precarious blood flow blockage or thrombosis, often with infarction or other severe consequences. Besides the stenosis-related outcomes, the arterial walls of plaque-ridden regions manifest increased stiffness, which may exacerbate negative patient prognosis. In pediatric populations, the most prevalent acquired cardiovascular pathology is Kawasaki disease. This acute vasculitis may affect the structural integrity of coronary artery walls and progress to aneurysmal lesions. These can hinder the blood flow’s hemodynamics, leading to inadequate downstream perfusion, and may activate thrombus formation which may lead to precarious prognosis. Diagnosing these two prominent coronary artery diseases is traditionally performed using fluoroscopic angiography. Several hundred serial x-ray projections are acquired during selective arterial infusion of a radiodense contrast agent, which reveals the vessels’ luminal area and possible pathological lesions. The acquired series contain highly dynamic information on voluntary and involuntary patient movement: respiration, organ displacement and heartbeat, for example. Current clinical analysis is largely limited to a single angiographic image where geometrical measures will be performed manually or semi-automatically by a radiological technician. Although widely used around the world and generally considered the gold-standard diagnosis tool for many vascular diseases, the two-dimensional nature of this imaging modality is limiting in terms of specifying the geometry of various pathological regions. Indeed, the 3D structures of stenotic or aneurysmal lesions may not be fully appreciated in 2D because their observable features are dependent on the angular configuration of the imaging gantry. Furthermore, the presence of lesions in the coronary arteries may not reflect the true health of the myocardium, as natural compensatory mechanisms may obviate the need for further intervention. In light of this, cardiac magnetic resonance perfusion imaging is increasingly gaining attention and clinical implementation, as it offers a direct assessment of myocardial tissue viability following infarction or suspected coronary artery disease. This type of modality is plagued, however, by motion similar to that present in fluoroscopic imaging. This issue predisposes clinicians to laborious manual intervention in order to align anatomical structures in sequential perfusion frames, thus hindering automation o

Human treelike tubular structure segmentation: A comprehensive review and future perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed

An improved classification approach for echocardiograms embedding temporal information

Cardiovascular disease is an umbrella term for all diseases of the heart. At present, computer-aided echocardiogram diagnosis is becoming increasingly beneficial. For echocardiography, different cardiac views can be acquired depending on the location and angulations of the ultrasound transducer. Hence, the automatic echocardiogram view classification is the first step for echocardiogram diagnosis, especially for computer-aided system and even for automatic diagnosis in the future. In addition, heart views classification makes it possible to label images especially for large-scale echo videos, provide a facility for database management and collection. This thesis presents a framework for automatic cardiac viewpoints classification of echocardiogram video data. In this research, we aim to overcome the challenges facing this investigation while analyzing, recognizing and classifying echocardiogram videos from 3D (2D spatial and 1D temporal) space. Specifically, we extend 2D KAZE approach into 3D space for feature detection and propose a histogram of acceleration as feature descriptor. Subsequently, feature encoding follows before the application of SVM to classify echo videos. In addition, comparison with the state of the art methodologies also takes place, including 2D SIFT, 3D SIFT, and optical flow technique to extract temporal information sustained in the video images. As a result, the performance of 2D KAZE, 2D KAZE with Optical Flow, 3D KAZE, Optical Flow, 2D SIFT and 3D SIFT delivers accuracy rate of 89.4%, 84.3%, 87.9%, 79.4%, 83.8% and 73.8% respectively for the eight view classes of echo videos

Learning to extract features for 2D – 3D multimodal registration

The ability to capture depth information form an scene has greatly increased in the recent years. 3D sensors, traditionally high cost and low resolution sensors, are being democratized and 3D scans of indoor and outdoor scenes are becoming more and more common. However, there is still a great data gap between the amount of captures being performed with 2D and 3D sensors. Although the 3D sensors provide more information about the scene, 2D sensors are still more accessible and widely used. This trade-off between availability and information between sensors brings us to a multimodal scenario of mixed 2D and 3D data. This thesis explores the fundamental block of this multimodal scenario: the registration between a single 2D image and a single unorganized point cloud. An unorganized 3D point cloud is the basic representation of a 3D capture. In this representation the surveyed points are represented only by their real word coordinates and, optionally, by their colour information. This simplistic representation brings multiple challenges to the registration, since most of the state of the art works leverage the existence of metadata about the scene or prior knowledges. Two different techniques are explored to perform the registration: a keypoint-based technique and an edge-based technique. The keypoint-based technique estimates the transformation by means of correspondences detected using Deep Learning, whilst the edge-based technique refines a transformation using a multimodal edge detection to establish anchor points to perform the estimation. An extensive evaluation of the proposed methodologies is performed. Albeit further research is needed to achieve adequate performances, the obtained results show the potential of the usage of deep learning techniques to learn 2D and 3D similarities. The results also show the good performance of the proposed 2D-3D iterative refinement, up to the state of the art on 3D-3D registration.La capacitat de captar informació de profunditat d’una escena ha augmentat molt els darrers anys. Els sensors 3D, tradicionalment d’alt cost i baixa resolució, s’estan democratitzant i escànners 3D d’escents interiors i exteriors són cada vegada més comuns. Tot i això, encara hi ha una gran bretxa entre la quantitat de captures que s’estan realitzant amb sensors 2D i 3D. Tot i que els sensors 3D proporcionen més informació sobre l’escena, els sensors 2D encara són més accessibles i àmpliament utilitzats. Aquesta diferència entre la disponibilitat i la informació entre els sensors ens porta a un escenari multimodal de dades mixtes 2D i 3D. Aquesta tesi explora el bloc fonamental d’aquest escenari multimodal: el registre entre una sola imatge 2D i un sol núvol de punts no organitzat. Un núvol de punts 3D no organitzat és la representació bàsica d’una captura en 3D. En aquesta representació, els punts mesurats es representen només per les seves coordenades i, opcionalment, per la informació de color. Aquesta representació simplista aporta múltiples reptes al registre, ja que la majoria dels algoritmes aprofiten l’existència de metadades sobre l’escena o coneixements previs. Per realitzar el registre s’exploren dues tècniques diferents: una tècnica basada en punts clau i una tècnica basada en contorns. La tècnica basada en punts clau estima la transformació mitjançant correspondències detectades mitjançant Deep Learning, mentre que la tècnica basada en contorns refina una transformació mitjançant una detecció multimodal de la vora per establir punts d’ancoratge per realitzar l’estimació. Es fa una avaluació àmplia de les metodologies proposades. Tot i que es necessita més investigació per obtenir un rendiment adequat, els resultats obtinguts mostren el potencial de l’ús de tècniques d’aprenentatge profund per aprendre similituds 2D i 3D. Els resultats també mostren l’excel·lent rendiment del perfeccionament iteratiu 2D-3D proposat, similar al dels algoritmes de registre 3D-3D.La capacidad de captar información de profundidad de una escena ha aumentado mucho en los últimos años. Los sensores 3D, tradicionalmente de alto costo y baja resolución, se están democratizando y escáneres 3D de escents interiores y exteriores son cada vez más comunes. Sin embargo, todavía hay una gran brecha entre la cantidad de capturas que se están realizando con sensores 2D y 3D. Aunque los sensores 3D proporcionan más información sobre la escena, los sensores 2D todavía son más accesibles y ampliamente utilizados. Esta diferencia entre la disponibilidad y la información entre los sensores nos lleva a un escenario multimodal de datos mixtos 2D y 3D. Esta tesis explora el bloque fundamental de este escenario multimodal: el registro entre una sola imagen 2D y una sola nube de puntos no organizado. Una nube de puntos 3D no organizado es la representación básica de una captura en 3D. En esta representación, los puntos medidos se representan sólo por sus coordenadas y, opcionalmente, por la información de color. Esta representación simplista aporta múltiples retos en el registro, ya que la mayoría de los algoritmos aprovechan la existencia de metadatos sobre la escena o conocimientos previos. Para realizar el registro se exploran dos técnicas diferentes: una técnica basada en puntos clave y una técnica basada en contornos. La técnica basada en puntos clave estima la transformación mediante correspondencias detectadas mediante Deep Learning, mientras que la técnica basada en contornos refina una transformación mediante una detección multimodal del borde para establecer puntos de anclaje para realizar la estimación. Se hace una evaluación amplia de las metodologías propuestas. Aunque se necesita más investigación para obtener un rendimiento adecuado, los resultados obtenidos muestran el potencial del uso de técnicas de aprendizaje profundo para aprender similitudes 2D y 3D. Los resultados también muestran el excelente rendimiento del perfeccionamiento iterativo 2D-3D propuesto, similar al de los algoritmos de registro 3D-3D

Learning to extract features for 2D – 3D multimodal registration

The ability to capture depth information form an scene has greatly increased in the recent years. 3D sensors, traditionally high cost and low resolution sensors, are being democratized and 3D scans of indoor and outdoor scenes are becoming more and more common. However, there is still a great data gap between the amount of captures being performed with 2D and 3D sensors. Although the 3D sensors provide more information about the scene, 2D sensors are still more accessible and widely used. This trade-off between availability and information between sensors brings us to a multimodal scenario of mixed 2D and 3D data. This thesis explores the fundamental block of this multimodal scenario: the registration between a single 2D image and a single unorganized point cloud. An unorganized 3D point cloud is the basic representation of a 3D capture. In this representation the surveyed points are represented only by their real word coordinates and, optionally, by their colour information. This simplistic representation brings multiple challenges to the registration, since most of the state of the art works leverage the existence of metadata about the scene or prior knowledges. Two different techniques are explored to perform the registration: a keypoint-based technique and an edge-based technique. The keypoint-based technique estimates the transformation by means of correspondences detected using Deep Learning, whilst the edge-based technique refines a transformation using a multimodal edge detection to establish anchor points to perform the estimation. An extensive evaluation of the proposed methodologies is performed. Albeit further research is needed to achieve adequate performances, the obtained results show the potential of the usage of deep learning techniques to learn 2D and 3D similarities. The results also show the good performance of the proposed 2D-3D iterative refinement, up to the state of the art on 3D-3D registration.La capacitat de captar informació de profunditat d’una escena ha augmentat molt els darrers anys. Els sensors 3D, tradicionalment d’alt cost i baixa resolució, s’estan democratitzant i escànners 3D d’escents interiors i exteriors són cada vegada més comuns. Tot i això, encara hi ha una gran bretxa entre la quantitat de captures que s’estan realitzant amb sensors 2D i 3D. Tot i que els sensors 3D proporcionen més informació sobre l’escena, els sensors 2D encara són més accessibles i àmpliament utilitzats. Aquesta diferència entre la disponibilitat i la informació entre els sensors ens porta a un escenari multimodal de dades mixtes 2D i 3D. Aquesta tesi explora el bloc fonamental d’aquest escenari multimodal: el registre entre una sola imatge 2D i un sol núvol de punts no organitzat. Un núvol de punts 3D no organitzat és la representació bàsica d’una captura en 3D. En aquesta representació, els punts mesurats es representen només per les seves coordenades i, opcionalment, per la informació de color. Aquesta representació simplista aporta múltiples reptes al registre, ja que la majoria dels algoritmes aprofiten l’existència de metadades sobre l’escena o coneixements previs. Per realitzar el registre s’exploren dues tècniques diferents: una tècnica basada en punts clau i una tècnica basada en contorns. La tècnica basada en punts clau estima la transformació mitjançant correspondències detectades mitjançant Deep Learning, mentre que la tècnica basada en contorns refina una transformació mitjançant una detecció multimodal de la vora per establir punts d’ancoratge per realitzar l’estimació. Es fa una avaluació àmplia de les metodologies proposades. Tot i que es necessita més investigació per obtenir un rendiment adequat, els resultats obtinguts mostren el potencial de l’ús de tècniques d’aprenentatge profund per aprendre similituds 2D i 3D. Els resultats també mostren l’excel·lent rendiment del perfeccionament iteratiu 2D-3D proposat, similar al dels algoritmes de registre 3D-3D.La capacidad de captar información de profundidad de una escena ha aumentado mucho en los últimos años. Los sensores 3D, tradicionalmente de alto costo y baja resolución, se están democratizando y escáneres 3D de escents interiores y exteriores son cada vez más comunes. Sin embargo, todavía hay una gran brecha entre la cantidad de capturas que se están realizando con sensores 2D y 3D. Aunque los sensores 3D proporcionan más información sobre la escena, los sensores 2D todavía son más accesibles y ampliamente utilizados. Esta diferencia entre la disponibilidad y la información entre los sensores nos lleva a un escenario multimodal de datos mixtos 2D y 3D. Esta tesis explora el bloque fundamental de este escenario multimodal: el registro entre una sola imagen 2D y una sola nube de puntos no organizado. Una nube de puntos 3D no organizado es la representación básica de una captura en 3D. En esta representación, los puntos medidos se representan sólo por sus coordenadas y, opcionalmente, por la información de color. Esta representación simplista aporta múltiples retos en el registro, ya que la mayoría de los algoritmos aprovechan la existencia de metadatos sobre la escena o conocimientos previos. Para realizar el registro se exploran dos técnicas diferentes: una técnica basada en puntos clave y una técnica basada en contornos. La técnica basada en puntos clave estima la transformación mediante correspondencias detectadas mediante Deep Learning, mientras que la técnica basada en contornos refina una transformación mediante una detección multimodal del borde para establecer puntos de anclaje para realizar la estimación. Se hace una evaluación amplia de las metodologías propuestas. Aunque se necesita más investigación para obtener un rendimiento adecuado, los resultados obtenidos muestran el potencial del uso de técnicas de aprendizaje profundo para aprender similitudes 2D y 3D. Los resultados también muestran el excelente rendimiento del perfeccionamiento iterativo 2D-3D propuesto, similar al de los algoritmos de registro 3D-3D.Postprint (published version

Human Treelike Tubular Structure Segmentation: A Comprehensive Review and Future Perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed.Comment: 30 pages, 19 figures, submitted to CBM journa

Fast catheter segmentation and tracking based on x-ray fluoroscopic and echocardiographic modalities for catheter-based cardiac minimally invasive interventions

X-ray fluoroscopy and echocardiography imaging (ultrasound, US) are two imaging modalities that are widely used in cardiac catheterization. For these modalities, a fast, accurate and stable algorithm for the detection and tracking of catheters is required to allow clinicians to observe the catheter location in real-time. Currently X-ray fluoroscopy is routinely used as the standard modality in catheter ablation interventions. However, it lacks the ability to visualize soft tissue and uses harmful radiation. US does not have these limitations but often contains acoustic artifacts and has a small field of view. These make the detection and tracking of the catheter in US very challenging. The first contribution in this thesis is a framework which combines Kalman filter and discrete optimization for multiple catheter segmentation and tracking in X-ray images. Kalman filter is used to identify the whole catheter from a single point detected on the catheter in the first frame of a sequence of x-ray images. An energy-based formulation is developed that can be used to track the catheters in the following frames. We also propose a discrete optimization for minimizing the energy function in each frame of the X-ray image sequence. Our approach is robust to tangential motion of the catheter and combines the tubular and salient feature measurements into a single robust and efficient framework. The second contribution is an algorithm for catheter extraction in 3D ultrasound images based on (a) the registration between the X-ray and ultrasound images and (b) the segmentation of the catheter in X-ray images. The search space for the catheter extraction in the ultrasound images is constrained to lie on or close to a curved surface in the ultrasound volume. The curved surface corresponds to the back-projection of the extracted catheter from the X-ray image to the ultrasound volume. Blob-like features are detected in the US images and organized in a graphical model. The extracted catheter is modelled as the optimal path in this graphical model. Both contributions allow the use of ultrasound imaging for the improved visualization of soft tissue. However, X-ray imaging is still required for each ultrasound frame and the amount of X-ray exposure has not been reduced. The final contribution in this thesis is a system that can track the catheter in ultrasound volumes automatically without the need for X-ray imaging during the tracking. Instead X-ray imaging is only required for the system initialization and for recovery from tracking failures. This allows a significant reduction in the amount of X-ray exposure for patient and clinicians.Open Acces