Automating Carotid Intima-Media Thickness Video Interpretation with Convolutional Neural Networks

Cardiovascular disease (CVD) is the leading cause of mortality yet largely preventable, but the key to prevention is to identify at-risk individuals before adverse events. For predicting individual CVD risk, carotid intima-media thickness (CIMT), a noninvasive ultrasound method, has proven to be valuable, offering several advantages over CT coronary artery calcium score. However, each CIMT examination includes several ultrasound videos, and interpreting each of these CIMT videos involves three operations: (1) select three end-diastolic ultrasound frames (EUF) in the video, (2) localize a region of interest (ROI) in each selected frame, and (3) trace the lumen-intima interface and the media-adventitia interface in each ROI to measure CIMT. These operations are tedious, laborious, and time consuming, a serious limitation that hinders the widespread utilization of CIMT in clinical practice. To overcome this limitation, this paper presents a new system to automate CIMT video interpretation. Our extensive experiments demonstrate that the suggested system significantly outperforms the state-of-the-art methods. The superior performance is attributable to our unified framework based on convolutional neural networks (CNNs) coupled with our informative image representation and effective post-processing of the CNN outputs, which are uniquely designed for each of the above three operations.Comment: J. Y. Shin, N. Tajbakhsh, R. T. Hurst, C. B. Kendall, and J. Liang. Automating carotid intima-media thickness video interpretation with convolutional neural networks. CVPR 2016, pp 2526-2535; N. Tajbakhsh, J. Y. Shin, R. T. Hurst, C. B. Kendall, and J. Liang. Automatic interpretation of CIMT videos using convolutional neural networks. Deep Learning for Medical Image Analysis, Academic Press, 201

Active skeleton for bacteria modeling

The investigation of spatio-temporal dynamics of bacterial cells and their molecular components requires automated image analysis tools to track cell shape properties and molecular component locations inside the cells. In the study of bacteria aging, the molecular components of interest are protein aggregates accumulated near bacteria boundaries. This particular location makes very ambiguous the correspondence between aggregates and cells, since computing accurately bacteria boundaries in phase-contrast time-lapse imaging is a challenging task. This paper proposes an active skeleton formulation for bacteria modeling which provides several advantages: an easy computation of shape properties (perimeter, length, thickness, orientation), an improved boundary accuracy in noisy images, and a natural bacteria-centered coordinate system that permits the intrinsic location of molecular components inside the cell. Starting from an initial skeleton estimate, the medial axis of the bacterium is obtained by minimizing an energy function which incorporates bacteria shape constraints. Experimental results on biological images and comparative evaluation of the performances validate the proposed approach for modeling cigar-shaped bacteria like Escherichia coli. The Image-J plugin of the proposed method can be found online at http://fluobactracker.inrialpes.fr.Comment: Published in Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualizationto appear i

Object Detection at the Optimal Scale with Hidden State Shape Models

Hidden State Shape Models (HSSMs) [2], a variant of Hidden Markov Models (HMMs) [9], were proposed to detect shape classes of variable structure in cluttered images. In this paper, we formulate a probabilistic framework for HSSMs which provides two major improvements in comparison to the previous method [2]. First, while the method in [2] required the scale of the object to be passed as an input, the method proposed here estimates the scale of the object automatically. This is achieved by introducing a new term for the observation probability that is based on a object-clutter feature model. Second, a segmental HMM [6, 8] is applied to model the "duration probability" of each HMM state, which is learned from the shape statistics in a training set and helps obtain meaningful registration results. Using a segmental HMM provides a principled way to model dependencies between the scales of different parts of the object. In object localization experiments on a dataset of real hand images, the proposed method significantly outperforms the method of [2], reducing the incorrect localization rate from 40% to 15%. The improvement in accuracy becomes more significant if we consider that the method proposed here is scale-independent, whereas the method of [2] takes as input the scale of the object we want to localize

Fully automated segmentation and tracking of the intima media thickness in ultrasound video sequences of the common carotid artery

Abstract—The robust identification and measurement of the intima media thickness (IMT) has a high clinical relevance because it represents one of the most precise predictors used in the assessment of potential future cardiovascular events. To facilitate the analysis of arterial wall thickening in serial clinical investigations, in this paper we have developed a novel fully automatic algorithm for the segmentation, measurement, and tracking of the intima media complex (IMC) in B-mode ultrasound video sequences. The proposed algorithm entails a two-stage image analysis process that initially addresses the segmentation of the IMC in the first frame of the ultrasound video sequence using a model-based approach; in the second step, a novel customized tracking procedure is applied to robustly detect the IMC in the subsequent frames. For the video tracking procedure, we introduce a spatially coherent algorithm called adaptive normalized correlation that prevents the tracking process from converging to wrong arterial interfaces. This represents the main contribution of this paper and was developed to deal with inconsistencies in the appearance of the IMC over the cardiac cycle. The quantitative evaluation has been carried out on 40 ultrasound video sequences of the common carotid artery (CCA) by comparing the results returned by the developed algorithm with respect to ground truth data that has been manually annotated by clinical experts. The measured IMTmean ± standard deviation recorded by the proposed algorithm is 0.60 mm ± 0.10, with a mean coefficient of variation (CV) of 2.05%, whereas the corresponding result obtained for the manually annotated ground truth data is 0.60 mm ± 0.11 with a mean CV equal to 5.60%. The numerical results reported in this paper indicate that the proposed algorithm is able to correctly segment and track the IMC in ultrasound CCA video sequences, and we were encouraged by the stability of our technique when applied to data captured under different imaging conditions. Future clinical studies will focus on the evaluation of patients that are affected by advanced cardiovascular conditions such as focal thickening and arterial plaques

Contour extraction from HVEM image of microvessel using active contour models

This thesis reports the research results on automatic contour extraction from high voltage electron microscope (HVEM) image of thick cross section montages of small blood vessels. The previous work on this subject, which was based on the conventional edge detection operations combined with edge linking, has proven inadequate to describe the inner structural compartments of microvessels. In this thesis, an active contour model (commonly referred to as Snakes ) has been applied to advance the previous work. Active contour models have proven themselves to be a powerful and flexible paradigm for many problems in image understanding, especially in contour extraction from medical images. With the developed energy functions, the active contour is attracted towards the edges under the action of internal forces (describing some elasticity properties of the contour), image forces and external forces by means of minimization of the energy functions. Based on this active model, an effective algorithm is implemented as a powerful tool for 2-D contour extraction in our problem for the first time. The results thus obtained turn out to be encouraging

Achieving the Way for Automated Segmentation of Nuclei in Cancer Tissue Images through Morphology-Based Approach: a Quantitative Evaluation

In this paper we address the problem of nuclear segmentation in cancer tissue images, that is critical for specific protein activity quantification and for cancer diagnosis and therapy. We present a fully automated morphology-based technique able to perform accurate nuclear segmentations in images with heterogeneous staining and multiple tissue layers and we compare it with an alternate semi-automated method based on a well established segmentation approach, namely active contours. We discuss active contours’ limitations in the segmentation of immunohistochemical images and we demonstrate and motivate through extensive experiments the better accuracy of our fully automated approach compared to various active contours implementations

An Improved Multi-Level Edge-Based Stereo Correspondence Technique for Snake Based Object Segmentation

Disparity maps generated by stereo correspondence are very useful for stereo object segmentation because based on disparity background clutter can be effectively removed from the image. This enables conventional methods such as snake-based to efficiently detect the object of interest contour. In this research I propose two main enhancements on Alattar’s method first I increased the number of edge levels, and utilized the color information in the matching process. Besides a few minor modifications, these enhancements achieve a more accurate disparity map which eventually helps achieve higher segmentation accuracy by the snake. Experiments were performed in various indoor and outdoor image conditions to evaluate the matching performance of the proposed method compared to the previous work

Intima-Media Thickness: Setting a Standard for a Completely Automated Method of Ultrasound Measurement

The intima - media thickness (IMT) of the common carotid artery is a widely used clinical marker of severe cardiovascular diseases. IMT is usually manually measured on longitudinal B-Mode ultrasound images. Many computer-based techniques for IMT measurement have been proposed to overcome the limits of manual segmentation. Most of these, however, require a certain degree of user interaction. In this paper we describe a new completely automated layers extraction (CALEXia) technique for the segmentation and IMT measurement of carotid wall in ultrasound images. CALEXia is based on an integrated approach consisting of feature extraction, line fitting, and classification that enables the automated tracing of the carotid adventitial walls. IMT is then measured by relying on a fuzzy K-means classifier. We tested CALEXia on a database of 200 images. We compared CALEXia performances to those of a previously developed methodology that was based on signal analysis (CULEXsa). Three trained operators manually segmented the images and the average profiles were considered as the ground truth. The average error from CALEXia for lumen - intima (LI) and media - adventitia (MA) interface tracings were 1.46 ± 1.51 pixel (0.091 ± 0.093 mm) and 0.40 ± 0.87 pixel (0.025 ± 0.055 mm), respectively. The corresponding errors for CULEXsa were 0.55 ± 0.51 pixels (0.035 ± 0.032 mm) and 0.59 ± 0.46 pixels (0.037 ± 0.029 mm). The IMT measurement error was equal to 0.87 ± 0.56 pixel (0.054 ± 0.035 mm) for CALEXia and 0.12 ± 0.14 pixel (0.01 ± 0.01 mm) for CULEXsa. Thus, CALEXia showed limited performance in segmenting the LI interface, but outperformed CULEXsa in the MA interface and in the number of images correctly processed (10 for CALEXia and 16 for CULEXsa). Based on two complementary strategies, we anticipate fusing them for further IMT improvement

Construction of boundary element models in bioelectromagnetism

Multisensor electro- and magnetoencephalographic (EEG and MEG) as well as electro- and magnetocardiographic (ECG and MCG) recordings have been proved useful in noninvasively extracting information on bioelectric excitation. The anatomy of the patient needs to be taken into account, when excitation sites are localized by solving the inverse problem. In this work, a methodology has been developed to construct patient specific boundary element models for bioelectromagnetic inverse problems from magnetic resonance (MR) data volumes as well as from two orthogonal X-ray projections. The process consists of three main steps: reconstruction of 3-D geometry, triangulation of reconstructed geometry, and registration of the model with a bioelectromagnetic measurement system. The 3-D geometry is reconstructed from MR data by matching a 3-D deformable boundary element template to images. The deformation is accomplished as an energy minimization process consisting of image and model based terms. The robustness of the matching is improved by multi-resolution and global-to-local approaches as well as using oriented distance maps. A boundary element template is also used when 3-D geometry is reconstructed from X-ray projections. The deformation is first accomplished in 2-D for the contours of simulated, built from the template, and real X-ray projections. The produced 2-D vector field is back-projected and interpolated on the 3-D template surface. A marching cube triangulation is computed for the reconstructed 3-D geometry. Thereafter, a non-iterative mesh-simplification method is applied. The method is based on the Voronoi-Delaunay duality on a 3-D surface with discrete distance measures. Finally, the triangulated surfaces are registered with a bioelectromagnetic measurement utilizing markers. More than fifty boundary element models have been successfully constructed from MR images using the methods developed in this work. A simulation demonstrated the feasibility of X-ray reconstruction; some practical problems of X-ray imaging need to be solved to begin tests with real data.reviewe