Automating Carotid Intima-Media Thickness Video Interpretation with Convolutional Neural Networks

Cardiovascular disease (CVD) is the leading cause of mortality yet largely preventable, but the key to prevention is to identify at-risk individuals before adverse events. For predicting individual CVD risk, carotid intima-media thickness (CIMT), a noninvasive ultrasound method, has proven to be valuable, offering several advantages over CT coronary artery calcium score. However, each CIMT examination includes several ultrasound videos, and interpreting each of these CIMT videos involves three operations: (1) select three end-diastolic ultrasound frames (EUF) in the video, (2) localize a region of interest (ROI) in each selected frame, and (3) trace the lumen-intima interface and the media-adventitia interface in each ROI to measure CIMT. These operations are tedious, laborious, and time consuming, a serious limitation that hinders the widespread utilization of CIMT in clinical practice. To overcome this limitation, this paper presents a new system to automate CIMT video interpretation. Our extensive experiments demonstrate that the suggested system significantly outperforms the state-of-the-art methods. The superior performance is attributable to our unified framework based on convolutional neural networks (CNNs) coupled with our informative image representation and effective post-processing of the CNN outputs, which are uniquely designed for each of the above three operations.Comment: J. Y. Shin, N. Tajbakhsh, R. T. Hurst, C. B. Kendall, and J. Liang. Automating carotid intima-media thickness video interpretation with convolutional neural networks. CVPR 2016, pp 2526-2535; N. Tajbakhsh, J. Y. Shin, R. T. Hurst, C. B. Kendall, and J. Liang. Automatic interpretation of CIMT videos using convolutional neural networks. Deep Learning for Medical Image Analysis, Academic Press, 201

Localizing Region-Based Level-set Contouring for Common Carotid Artery in Ultrasonography

 This work developed a fully-automated and efficient method for detecting contour of common carotid artery in the cross section view of two-dimensional B-mode sonography. First, we applied a preprocessing filter to the ultrasound image for the sake of reducing speckle. An adaptive initial contouring method was then performed to obtain the initial contour for level set segmentation. Finally, the localizing region-based level set segmentation automatically extracted the precise contours of common carotid artery. The proposed method evaluated 130 ultrasound images from three healthy volunteers and the segmentation results were compared to the boundaries outlined by an expert. Preliminary results showed that the method described here could identify the contour of common carotid artery with satisfactory accuracy in this dataset

Constrained snake vs. conventional snake for carotid ultrasound automated IMT measurements on multi-center data sets

Accurate intima-media thickness (IMT) measurement of the carotid artery from minimal plaque ultrasound images is a relevant clinical need, since IMT increase is related to the progression of atherosclerosis. In this paper, we describe a novel dual snake-based model for the high-performance carotid IMT measurement, called Carotid Measurement Using Dual Snakes (CMUDS). Snakes (which are deformable contours) adapt to the lumen-intima (LI) and media-adventitia (MA) interfaces, thus enabling the IMT computation as distance between the LI and MA snakes. However, traditional snakes might be unable to maintain a correct distance and in some spatial location along the artery, it might even collapse between them or diverge. The technical improvement of this work is the definition of a dual snake-based constrained system, which prevents the LI and MA snakes from collapsing or bleeding, thus optimizing the IMT estimation. The CMUDS system consists of two parametric models automatically initialized using the far adventitia border which we automatically traced by using a previously developed multi-resolution approach. The dual snakes evolve simultaneously and are constrained by the distances between them, ensuring the regularization of LI/MA topology. We benchmarked our automated CMUDS with the previous conventional semi-automated snake system called Carotid Measurement Using Single Snake (CMUSS). Two independent readers manually traced the LIMA boundaries of a multi-institutional, multi-ethnic, and multi-scanner database of 665 CCA longitudinal 2D images. We evaluated our system performance by comparing it with the gold standard as traced by clinical readers. CMUDS and CMUSS correctly processed 100% of the 665 images. Comparing the performance with respect to the two readers, our automatically measured IMT was on average very close to that of the two readers (IMT measurement biases for CMUSS was equal to −0.011 ± 0.329 mm and −0.045 ± 0.317 mm, respectively, while for CMUDS, it was 0.030 ± 0.284 mm and −0.004 ± 0.273 mm, respectively). The Figure-of-Merit of the system was 98.5% and 94.4% for CMUSS, while 96.0% and 99.6% for CMUDS, respectively. Results showed that the dual-snake system CMUDS reduced the IMT measurement error accuracy (Wilcoxon, p < 0.02) and the IMT error variability (Fisher, p < 3 × 10−2). We propose the CMUDS technique for use in large multi-centric studies, where the need for a standard, accurate, and automated IMT measurement technique is require

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Vascular Segmentation Algorithms for Generating 3D Atherosclerotic Measurements

Atherosclerosis manifests as plaques within large arteries of the body and remains as a leading cause of mortality and morbidity in the world. Major cardiovascular events may occur in patients without known preexisting symptoms, thus it is important to monitor progression and regression of the plaque burden in the arteries for evaluating patient\u27s response to therapy. In this dissertation, our main focus is quantification of plaque burden from the carotid and femoral arteries, which are major sites for plaque formation, and are straight forward to image noninvasively due to their superficial location. Recently, 3D measurements of plaque burden have shown to be more sensitive to the changes of plaque burden than one-/two-dimensional measurements. However, despite the advancements of 3D noninvasive imaging technology with rapid acquisition capabilities, and the high sensitivity of the 3D plaque measurements of plaque burden, they are still not widely used due to the inordinate amount of time and effort required to delineate artery walls plus plaque boundaries to obtain 3D measurements from the images. Therefore, the objective of this dissertation is developing novel semi-automated segmentation methods to alleviate measurement burden from the observer for segmentation of the outer wall and lumen boundaries from: (1) 3D carotid ultrasound (US) images, (2) 3D carotid black-blood magnetic resonance (MR) images, and (3) 3D femoral black-blood MR images. Segmentation of the carotid lumen and outer wall from 3DUS images is a challenging task due to low image contrast, for which no method has been previously reported. Initially, we developed a 2D slice-wise segmentation algorithm based on the level set method, which was then extended to 3D. The 3D algorithm required fewer user interactions than manual delineation and the 2D method. The algorithm reduced user time by ≈79% (1.72 vs. 8.3 min) compared to manual segmentation for generating 3D-based measurements with high accuracy (Dice similarity coefficient (DSC)\u3e90%). Secondly, we developed a novel 3D multi-region segmentation algorithm, which simultaneously delineates both the carotid lumen and outer wall surfaces from MR images by evolving two coupled surfaces using a convex max-flow-based technique. The algorithm required user interaction only on a single transverse slice of the 3D image for generating 3D surfaces of the lumen and outer wall. The algorithm was parallelized using graphics processing units (GPU) to increase computational speed, thus reducing user time by 93% (0.78 vs. 12 min) compared to manual segmentation. Moreover, the algorithm yielded high accuracy (DSC \u3e 90%) and high precision (intra-observer CV \u3c 5.6% and inter-observer CV \u3c 6.6%). Finally, we developed and validated an algorithm based on convex max-flow formulation to segment the femoral arteries that enforces a tubular shape prior and an inter-surface consistency of the outer wall and lumen to maintain a minimum separation distance between the two surfaces. The algorithm required the observer to choose only about 11 points on its medial axis of the artery to yield the 3D surfaces of the lumen and outer wall, which reduced the operator time by 97% (1.8 vs. 70-80 min) compared to manual segmentation. Furthermore, the proposed algorithm reported DSC greater than 85% and small intra-observer variability (CV ≈ 6.69%). In conclusion, the development of robust semi-automated algorithms for generating 3D measurements of plaque burden may accelerate translation of 3D measurements to clinical trials and subsequently to clinical care

Image segmentation and reconstruction of 3D surfaces from carotid ultrasound images

Tese de doutoramento. Engenharia Electrotécnica e de Computadores. Faculdade de Engenharia. Universidade do Porto. 200

Basic Science to Clinical Research: Segmentation of Ultrasound and Modelling in Clinical Informatics

The world of basic science is a world of minutia; it boils down to improving even a fraction of a percent over the baseline standard. It is a domain of peer reviewed fractions of seconds and the world of squeezing every last ounce of efficiency from a processor, a storage medium, or an algorithm. The field of health data is based on extracting knowledge from segments of data that may improve some clinical process or practice guideline to improve the time and quality of care. Clinical informatics and knowledge translation provide this information in order to reveal insights to the world of improving patient treatments, regimens, and overall outcomes. In my world of minutia, or basic science, the movement of blood served an integral role. The novel detection of sound reverberations map out the landscape for my research. I have applied my algorithms to the various anatomical structures of the heart and artery system. This serves as a basis for segmentation, active contouring, and shape priors. The algorithms presented, leverage novel applications in segmentation by using anatomical features of the heart for shape priors and the integration of optical flow models to improve tracking. The presented techniques show improvements over traditional methods in the estimation of left ventricular size and function, along with plaque estimation in the carotid artery. In my clinical world of data understanding, I have endeavoured to decipher trends in Alzheimer’s disease, Sepsis of hospital patients, and the burden of Melanoma using mathematical modelling methods. The use of decision trees, Markov models, and various clustering techniques provide insights into data sets that are otherwise hidden. Finally, I demonstrate how efficient data capture from providers can achieve rapid results and actionable information on patient medical records. This culminated in generating studies on the burden of illness and their associated costs. A selection of published works from my research in the world of basic sciences to clinical informatics has been included in this thesis to detail my transition. This is my journey from one contented realm to a turbulent one