Automated interpretation of systolic and diastolic function on the echocardiogram:a multicohort study

Background: Echocardiography is the diagnostic modality for assessing cardiac systolic and diastolic function to diagnose and manage heart failure. However, manual interpretation of echocardiograms can be time consuming and subject to human error. Therefore, we developed a fully automated deep learning workflow to classify, segment, and annotate two-dimensional (2D) videos and Doppler modalities in echocardiograms. Methods: We developed the workflow using a training dataset of 1145 echocardiograms and an internal test set of 406 echocardiograms from the prospective heart failure research platform (Asian Network for Translational Research and Cardiovascular Trials; ATTRaCT) in Asia, with previous manual tracings by expert sonographers. We validated the workflow against manual measurements in a curated dataset from Canada (Alberta Heart Failure Etiology and Analysis Research Team; HEART; n=1029 echocardiograms), a real-world dataset from Taiwan (n=31 241), the US-based EchoNet-Dynamic dataset (n=10 030), and in an independent prospective assessment of the Asian (ATTRaCT) and Canadian (Alberta HEART) datasets (n=142) with repeated independent measurements by two expert sonographers. Findings: In the ATTRaCT test set, the automated workflow classified 2D videos and Doppler modalities with accuracies (number of correct predictions divided by the total number of predictions) ranging from 0·91 to 0·99. Segmentations of the left ventricle and left atrium were accurate, with a mean Dice similarity coefficient greater than 93% for all. In the external datasets (n=1029 to 10 030 echocardiograms used as input), automated measurements showed good agreement with locally measured values, with a mean absolute error range of 9–25 mL for left ventricular volumes, 6–10% for left ventricular ejection fraction (LVEF), and 1·8–2·2 for the ratio of the mitral inflow E wave to the tissue Doppler e' wave (E/e' ratio); and reliably classified systolic dysfunction (LVEF <40%, area under the receiver operating characteristic curve [AUC] range 0·90–0·92) and diastolic dysfunction (E/e' ratio ≥13, AUC range 0·91–0·91), with narrow 95% CIs for AUC values. Independent prospective evaluation confirmed less variance of automated compared with human expert measurements, with all individual equivalence coefficients being less than 0 for all measurements. Interpretation: Deep learning algorithms can automatically annotate 2D videos and Doppler modalities with similar accuracy to manual measurements by expert sonographers. Use of an automated workflow might accelerate access, improve quality, and reduce costs in diagnosing and managing heart failure globally. Funding: A*STAR Biomedical Research Council and A*STAR Exploit Technologies

Applying novel machine learning technology to optimize computer-aided detection and diagnosis of medical images

The purpose of developing Computer-Aided Detection (CAD) schemes is to assist physicians (i.e., radiologists) in interpreting medical imaging findings and reducing inter-reader variability more accurately. In developing CAD schemes, Machine Learning (ML) plays an essential role because it is widely used to identify effective image features from complex datasets and optimally integrate them with the classifiers, which aims to assist the clinicians to more accurately detect early disease, classify disease types and predict disease treatment outcome. In my dissertation, in different studies, I assess the feasibility of developing several novel CAD systems in the area of medical imaging for different purposes. The first study aims to develop and evaluate a new computer-aided diagnosis (CADx) scheme based on analysis of global mammographic image features to predict the likelihood of cases being malignant. CADx scheme is applied to pre-process mammograms, generate two image maps in the frequency domain using discrete cosine transform and fast Fourier transform, compute bilateral image feature differences from left and right breasts, and apply a support vector machine (SVM) method to predict the likelihood of the case being malignant. This study demonstrates the feasibility of developing a new global image feature analysis based CADx scheme of mammograms with high performance. This new CADx approach is more efficient in development and potentially more robust in future applications by avoiding difficulty and possible errors in breast lesion segmentation. In the second study, to automatically identify a set of effective mammographic image features and build an optimal breast cancer risk stratification model, I investigate advantages of applying a machine learning approach embedded with a locally preserving projection (LPP) based feature combination and regeneration algorithm to predict short-term breast cancer risk. To this purpose, a computer-aided image processing scheme is applied to segment fibro-glandular tissue depicted on mammograms and initially compute 44 features related to the bilateral asymmetry of mammographic tissue density distribution between left and right breasts. Next, an embedded LLP algorithm optimizes the feature space and regenerates a new operational vector with 4 features using a maximal variance approach. This study demonstrates that applying the LPP algorithm effectively reduces feature dimensionality, and yields higher and potentially more robust performance in predicting short-term breast cancer risk. In the third study, to more precisely classify malignant lesions, I investigate the feasibility of applying a random projection algorithm to build an optimal feature vector from the initially CAD-generated large feature pool and improve the performance of the machine learning model. In this process, a CAD scheme is first applied to segment mass regions and initially compute 181 features. An SVM model embedded with the feature dimensionality reduction method is then built to predict the likelihood of lesions being malignant. This study demonstrates that the random project algorithm is a promising method to generate optimal feature vectors to improve the performance of machine learning models of medical images. The last study aims to develop and test a new CAD scheme of chest X-ray images to detect coronavirus (COVID-19) infected pneumonia. To this purpose, the CAD scheme first applies two image preprocessing steps to remove the majority of diaphragm regions, process the original image using a histogram equalization algorithm, and a bilateral low-pass filter. Then, the original image and two filtered images are used to form a pseudo color image. This image is fed into three input channels of a transfer learning-based convolutional neural network (CNN) model to classify chest X-ray images into 3 classes of COVID-19 infected pneumonia, other community-acquired no-COVID-19 infected pneumonia, and normal (non-pneumonia) cases. This study demonstrates that adding two image preprocessing steps and generating a pseudo color image plays an essential role in developing a deep learning CAD scheme of chest X-ray images to improve accuracy in detecting COVID-19 infected pneumonia. In summary, I developed and presented several image pre-processing algorithms, feature extraction methods, and data optimization techniques to present innovative approaches for quantitative imaging markers based on machine learning systems in all these studies. The studies' simulation and results show the discriminative performance of the proposed CAD schemes on different application fields helpful to assist radiologists on their assessments in diagnosing disease and improve their overall performance

Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 [electronic resource] : 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II /

The six-volume set LNCS 11764, 11765, 11766, 11767, 11768, and 11769 constitutes the refereed proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, held in Shenzhen, China, in October 2019. The 539 revised full papers presented were carefully reviewed and selected from 1730 submissions in a double-blind review process. The papers are organized in the following topical sections: Part I: optical imaging; endoscopy; microscopy. Part II: image segmentation; image registration; cardiovascular imaging; growth, development, atrophy and progression. Part III: neuroimage reconstruction and synthesis; neuroimage segmentation; diffusion weighted magnetic resonance imaging; functional neuroimaging (fMRI); miscellaneous neuroimaging. Part IV: shape; prediction; detection and localization; machine learning; computer-aided diagnosis; image reconstruction and synthesis. Part V: computer assisted interventions; MIC meets CAI. Part VI: computed tomography; X-ray imaging.Image Segmentation -- Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation -- Comparative Evaluation of Hand-Engineered and Deep-Learned Features for Neonatal Hip Bone Segmentation in Ultrasound -- Unsupervised Quality Control of Image Segmentation based on Bayesian Learning -- One Network To Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation -- 'Project & Excite' Modules for Segmentation of Volumetric Medical Scans -- Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation -- Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation -- Extreme Points Derived Confidence Map as a Cue For Class-Agnostic Segmentation Using Deep Neural Network -- Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation -- Instance Segmentation from Volumetric Biomedical Images without Voxel-Wise Labeling -- Optimizing the Dice Score and Jaccard Index for Medical Image Segmentation: Theory & Practice -- Dual Adaptive Pyramid Network for Cross-Stain Histopathology Image Segmentation -- HD-Net: Hybrid Discriminative Network for Prostate Segmentation in MR Images -- PHiSeg: Capturing Uncertainty in Medical Image Segmentation -- Neural Style Transfer Improves 3D Cardiovascular MR Image Segmentation on Inconsistent Data -- Supervised Uncertainty Quantification for Segmentation with Multiple Annotations -- 3D Tiled Convolution for Effective Segmentation of Volumetric Medical Images -- Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation -- Statistical intensity- and shape-modeling to automate cerebrovascular segmentation from TOF-MRA data -- Segmentation of Vessels in Ultra High Frequency Ultrasound Sequences using Contextual Memory -- Accurate Esophageal Gross Tumor Volume Segmentation in PET/CT using Two-Stream Chained 3D Deep Network Fusion -- Mixed-Supervised Dual-Network for Medical Image Segmentation -- Fully Automated Pancreas Segmentation with Two-stage 3D Convolutional Neural Networks -- Globally Guided Progressive Fusion Network for 3D Pancreas Segmentation -- Automatic Segmentation of Muscle Tissue and Inter-muscular Fat in Thigh and Calf MRI Images -- Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation -- Radiomics-guided GAN for Segmentation of Liver Tumor without Contrast Agents -- Liver Segmentation in Magnetic Resonance Imaging via Mean Shape Fitting with Fully Convolutional Neural Networks -- Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation -- Automatic Segmentation of Vestibular Schwannoma from T2-Weighted MRI by Deep Spatial Attention with Hardness-Weighted Loss -- Learning Shape Representation on Sparse Point Clouds for Volumetric Image Segmentation -- Collaborative Multi-agent Learning for MR Knee Articular Cartilage Segmentation -- 3D U2-Net: A 3D Universal U-Net for Multi-Domain Medical Image Segmentation -- Impact of Adversarial Examples on Deep Learning Segmentation Models -- Multi-Resolution Path CNN with Deep Supervision for Intervertebral Disc Localization and Segmentation -- Automatic paraspinal muscle segmentation in patients with lumbar pathology using deep convolutional neural network -- Constrained Domain Adaptation for Segmentation -- Image Registration -- Image-and-Spatial Transformer Networks for Structure-Guided Image Registration -- Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration -- A deep learning approach to MR-less spatial normalization for tau PET images -- TopAwaRe: Topology-Aware Registration -- Multimodal Data Registration for Brain Structural Association Networks -- Dual-Stream Pyramid Registration Network -- A Cooperative Autoencoder for Population-Based Regularization of CNN Image Registration -- Conditional Segmentation in Lieu of Image Registration -- On the applicability of registration uncertainty -- DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation -- Linear Time Invariant Model based Motion Correction (LiMo-Moco) of Dynamic Radial Contrast Enhanced MRI -- Incompressible image registration using divergence-conforming B-splines -- Cardiovascular Imaging -- Direct Quantification for Coronary Artery Stenosis Using Multiview Learning -- Bayesian Optimization on Large Graphs via a Graph Convolutional Generative Model: Application in Cardiac Model Personalization -- Discriminative Coronary Artery Tracking via 3D CNN in Cardiac CT Angiography -- Multi-modality Whole-Heart and Great Vessel Segmentation in Congenital Heart Disease using Deep Neural Networks and Graph Matching -- Harmonic Balance Techniques in Cardiovascular Fluid Mechanics -- Deep learning within a priori temporal feature spaces for large-scale dynamic MR image reconstruction: Application to 5-D cardiac MR Multitasking -- k-t NEXT: Dynamic MR Image Reconstruction Exploiting Spatio-temporal Correlations -- Model-based reconstruction for highly accelerated first-pass perfusion cardiac MRI -- Learning Shape Priors for Robust Cardiac MR Segmentation from Multi-view images -- Right Ventricle Segmentation in Short-Axis MRI Using A Shape Constrained Dense Connected U-net -- Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction -- A Fine-Grain Error Map Prediction and Segmentation Quality Assessment Framework for Whole-Heart Segmentation -- Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors -- Curriculum semi-supervised segmentation -- A Multi-modal Network for Cardiomyopathy Death Risk Prediction with CMR Images and Clinical Information -- 3D Cardiac Shape Prediction with Deep Neural Networks: Simultaneous Use of Images and Patient Metadata -- Discriminative Consistent Domain Generation for Semi-supervised Learning -- Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation -- MSU-Net: Multiscale Statistical U-Net for Real-time 3D Cardiac MRI Video Segmentation -- The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN -- Cardiac MRI Segmentation with Strong Anatomical Guarantees -- Decompose-and-Integrate Learning for Multi-class Segmentation in Medical Images -- Missing Slice Imputation in Population CMR Imaging via Conditional Generative Adversarial Nets -- Unsupervised Standard Plane Synthesis in Population Cine MRI via Cycle-Consistent Adversarial Networks -- Data Efficient Unsupervised Domain Adaptation for Cross-Modality Image Segmentation -- Recurrent Aggregation Learning for Multi-View Echocardiographic Sequences Segmentation -- Echocardiography View Classification Using Quality Transfer Star Generative Adversarial Networks -- Dual-view Joint Estimation of Left Ventricular Ejection Fraction with Uncertainty Modelling in Echocardiograms -- Frame Rate Up-Conversion in Echocardiography Using a Conditioned Variational Autoencoder and Generative Adversarial Model -- Annotation-Free Cardiac Vessel Segmentation via Knowledge Transfer from Retinal Images -- DeepAAA: clinically applicable and generalizable detection of abdominal aortic aneurysm using deep learning -- Texture-based classification of significant stenosis in CCTA multi-view images of coronary arteries -- Fourier Spectral Dynamic Data Assimilation: Interlacing CFD with 4D flow MRI -- Quality Control-Driven Image Segmentation Towards Reliable Automatic Image Analysis in Large-Scale Cardiovascular Magnetic Resonance Aortic Cine Imaging -- HFA-Net: 3D Cardiovascular Image Segmentation with Asymmetrical Pooling and Content-Aware Fusion -- Spectral CT based training dataset generation and augmentation for conventional CT vascular segmentation -- Context-Aware Inductive Bias Learning for Vessel Border Detection in Multi-modal Intracoronary Imaging -- Growth, Development, Atrophy and Progression -- Neural parameters estimation for brain tumor growth modeling -- Learning-Guided Infinite Network Atlas Selection for Predicting Longitudinal Brain Network Evolution from a Single Observation -- Deep Probabilistic Modeling of Glioma Growth -- Surface-Volume Consistent Construction of Longitudinal Atlases for the Early Developing Brains -- Variational Autoencoder for Regression: Application to Brain Aging Analysis -- Early Development of Infant Brain Complex Network -- Revealing Developmental Regionalization of Infant Cerebral Cortex Based on Multiple Cortical Properties -- Continually Modeling Alzheimer's Disease Progression via Deep Multi-Order Preserving Weight Consolidation -- Disease Knowledge Transfer across Neurodegenerative Diseases.The six-volume set LNCS 11764, 11765, 11766, 11767, 11768, and 11769 constitutes the refereed proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, held in Shenzhen, China, in October 2019. The 539 revised full papers presented were carefully reviewed and selected from 1730 submissions in a double-blind review process. The papers are organized in the following topical sections: Part I: optical imaging; endoscopy; microscopy. Part II: image segmentation; image registration; cardiovascular imaging; growth, development, atrophy and progression. Part III: neuroimage reconstruction and synthesis; neuroimage segmentation; diffusion weighted magnetic resonance imaging; functional neuroimaging (fMRI); miscellaneous neuroimaging. Part IV: shape; prediction; detection and localization; machine learning; computer-aided diagnosis; image reconstruction and synthesis. Part V: computer assisted interventions; MIC meets CAI. Part VI: computed tomography; X-ray imaging