MyoPS A Benchmark of Myocardial Pathology Segmentation Combining Three-Sequence Cardiac Magnetic Resonance Images

Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on myocardium is the key to this assessment. This work defines a new task of medical image analysis, i.e., to perform myocardial pathology segmentation (MyoPS) combining three-sequence cardiac magnetic resonance (CMR) images, which was first proposed in the MyoPS challenge, in conjunction with MICCAI 2020. The challenge provided 45 paired and pre-aligned CMR images, allowing algorithms to combine the complementary information from the three CMR sequences for pathology segmentation. In this article, we provide details of the challenge, survey the works from fifteen participants and interpret their methods according to five aspects, i.e., preprocessing, data augmentation, learning strategy, model architecture and post-processing. In addition, we analyze the results with respect to different factors, in order to examine the key obstacles and explore potential of solutions, as well as to provide a benchmark for future research. We conclude that while promising results have been reported, the research is still in the early stage, and more in-depth exploration is needed before a successful application to the clinics. Note that MyoPS data and evaluation tool continue to be publicly available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/)

Méthodes d'apprentissage profond basées sur l'incertitude pour une segmentation et une analyse robustes et fiables de l'IRM cardiaque

Deep learning-based segmentation methods have shown promise in automating the seg-mentation of cardiac MRI images, but they still face challenges in robustly segmentingsmall, and ambiguous regions with irregular shapes like myocardial scars. Additionally,these models struggle with domain shifts and out-of-distribution (OOD) samples, whichmakes them unreliable and limits their usage in clinical practice. The main objective ofthis thesis is to enhance the robustness and reliability of deep learning models for cardiacMRI segmentation and analysis by leveraging uncertainty estimates.To improve the segmentation of myocardial scars, a segmentation model is proposed thatintegrates uncertainty information into the learning process. Uncertainty estimation isachieved by utilizing Monte-Carlo dropout-based Bayesian neural networks during train-ing, which are then incorporated into the loss function. This approach yields improvedsegmentation accuracy and probability calibration, achieving state-of-the-art performanceon publicly available datasets focused on scar segmentation from Late Gadolinium En-hancement (LGE) MRI. The method demonstrates superior performance, particularly forvisually challenging images with higher epistemic uncertainty.To enhance the reliability of segmentation models, an uncertainty-based quality control(QC) framework is introduced to identify failed segmentations before further analysis. TheQC framework utilizes a Bayesian Swin transformer-based U-Net for the segmentation ofT1 mapping images and employs image-level uncertainty features to detect poorly seg-mented images. Experimental results on private and public datasets demonstrate that theproposed QC method significantly outperforms other state-of-the-art uncertainty-basedQC methods, particularly in challenging scenarios. After rejecting inaccurate segmenta-tions, T1 mapping, and Extracellular volume (ECV) values are automatically computed,enabling reliable characterization of myocardial tissues in healthy and pathological cases.Furthermore, a post-hoc OOD detection method is proposed to identify and reject outlierimages. This method utilizes the encoder features of the segmentation model and similar-ity metrics to enhance the trustworthiness of segmentation models. Experimental resultsdemonstrate that the proposed method outperforms state-of-the-art feature space-basedand uncertainty-based OOD detection methods across the various OOD datasets. Thisfurther safeguards performance by rejecting unsuitable outliers.Les méthodes de segmentation basées sur l'apprentissage profond se sont révélées prometteuses pour automatiser la segmentation des images IRM cardiaques, mais elles sont toujours confrontées à des défis pour segmenter de manière robuste des régions petites et ambiguës aux formes irrégulières comme les cicatrices myocardiques. De plus, ces modèles sont confrontés aux changements de domaine et aux échantillons hors distribution (OOD), ce qui les rend peu fiables et limite leur utilisation dans la pratique clinique. L'objectif principal de cette thèse est d'améliorer la robustesse et la fiabilité des modèles d'apprentissage profond pour la segmentation et l'analyse d'IRM cardiaque en exploitant les estimations d'incertitude.Pour améliorer la segmentation des cicatrices myocardiques, un modèle de segmentation est proposé qui intègre les informations d'incertitude dans le processus d'apprentissage. L'estimation de l'incertitude est obtenue en utilisant des réseaux neuronaux bayésiens basés sur une méthode Monté Carlo Drop out pendant la formation, qui sont ensuite incorporés dans la fonction de perte. Cette approche permet d'améliorer la précision de la segmentation et l'étalonnage des probabilités, obtenant ainsi des performances de l'état de l'art sur des ensembles de données accessibles au public axés sur la segmentation des cicatrices à partir de l'IRM avec rehaussement tardif au gadolinium (LGE). La méthode démontre des performances supérieures, en particulier pour les images visuellement difficiles avec une incertitude épistémique plus élevée.Pour améliorer la fiabilité des modèles de segmentation, un cadre de contrôle qualité (CQ) basé sur l'incertitude est introduit pour identifier les segmentations ayant échoué avant une analyse plus approfondie. Le cadre CQ utilise un U-Net basé sur un Transformer bayésien Swin pour la segmentation des images cartographiques T1 et utilise des caractéristiques d'incertitude au niveau de l'image pour détecter les images mal segmentées. Les résultats expérimentaux sur des ensembles de données privés et publics démontrent que la méthode de CQ proposée surpasse considérablement les autres méthodes de CQ de l'état de l'art basées sur l'incertitude, en particulier dans des scénarios difficiles. Après avoir rejeté les segmentations inexactes, la cartographie T1 et les valeurs du volume extracellulaire (ECV) sont automatiquement calculées, permettant une caractérisation fiable des tissus myocardiques dans les cas sains et pathologiques.De plus, une méthode de détection OOD post-hoc est proposée pour identifier et rejeter les images aberrantes. Cette méthode utilise les fonctionnalités d'encodeur du modèle de segmentation et les métriques de similarité pour améliorer la fiabilité des modèles de segmentation. Les résultats expérimentaux démontrent que la méthode proposée surpasse les méthodes de détection OOD de l'état de l'art basées sur l'espace des caractéristiques et l'incertitude dans les différents ensembles de données OOD. Cela garantit davantage les performances en rejetant les valeurs aberrantes inappropriées

Automatic Quality Assessment of Cardiac MR Images with Motion Artefacts using Multi-task Learning and K-Space Motion Artefact Augmentation

International audienceThe movement of patients and respiratory motion during MRI acquisition produce image artefacts that reduce the image quality and its diagnostic value. Quality assessment of the images is essential to minimize segmentation errors and avoid wrong clinical decisions in the downstream tasks. In this paper, we propose automatic multi-task learning (MTL) based classification model to detect cardiac MR images with different levels of motion artefact. We also develop an automatic segmentation model that leverages k-space based motion artefact augmentation (MAA) and a novel compound loss that utilizes Dice loss with a polynomial version of cross-entropy loss (PolyLoss) to robustly segment cardiac structures from cardiac MRIs with respiratory motion artefacts. We evaluate the proposed method on Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion 2022) challenge dataset. For the detection task, the multi-task learning based model that simultaneously learns both image artefact prediction and breath-hold type prediction achieved significantly better results compared to the single-task model, showing the benefits of MTL. In addition, we utilized test-time augmentation (TTA) to enhance the classification accuracy and study aleatoric uncertainty of the images. Using TTA further improved the classification result as it achieved an accuracy of 0.65 and Cohen's kappa of 0.413. From the estimated aleatoric uncertainty, we observe that images with higher aleatoric uncertainty are more difficult to classify than the ones with lower uncertainty. For the segmentation task, the k-space based MAA enhanced the segmentation accuracy of the baseline model. From the results, we also observe that using a hybrid loss of Dice and PolyLoss can be advantageous to robustly segment cardiac MRIs with motion artefact, leading to a mean Dice of 0.9204, 0.8315, and 0.8906 and mean HD95 of 8.09 mm, 3.60 mm and 6.07 mm for LV, MYO and RV respectively on the official validation set. On the test set, the proposed segmentation method was ranked in second place in the segmentation task of CMRxMotion 2022 challenge

Using Polynomial Loss and Uncertainty Information for Robust Left Atrial and Scar Quantification and Segmentation

International audienceAutomatic and accurate segmentation of the left atrial (LA) cavity and scar can be helpful for the diagnosis and prognosis of patients with atrial fibrillation. However, automating the segmentation can be difficult due to the poor image quality, variable LA shapes, and small discrete regions of LA scars. In this paper, we proposed a fully-automatic method to segment LA cavity and scar from Late Gadolinium Enhancement (LGE) MRIs. For the loss functions, we propose two different losses for each task. To enhance the segmentation of LA cavity from the multicenter dataset, we present a hybrid loss that leverages Dice loss with a polynomial version of cross-entropy loss (PolyCE). We also utilize different data augmentations that include histogram matching to increase the variety of the dataset. For the more difficult LA scar segmentation, we propose a loss function that uses uncertainty information to improve the uncertain and inaccurate scar segmentation results. We evaluate the proposed method on the Left Atrial and Scar Quantification and Segmentation (LAScarQS 2022) Challenge dataset. It achieves a Dice score of 0.8897 and a Hausdorff distance (HD) of 16.91mm for LA cavity and a Dice score of 0.6406 and sensitivity of 0.5853 for LA scar. From the results, we notice that for LA scar segmentation, which has small and irregular shapes, the proposed loss that utilizes the uncertainty estimates generated by the scar yields the best result compared to the other loss functions. For the multi-center LA cavity segmentation, we observe that combining the region-based Dice loss with the pixelwise PolyCE can achieve a good result by enhancing the segmentation result in terms of both Dice score and HD. Furthermore, using moderate-level data augmentation with histogram matching improves the model's generalization capability. Our proposed method won the Left Atrial and Scar Quantification and Segmentation (LAScarQS 2022) Challenge

Leveraging Uncertainty Estimates to Improve Segmentation Performance in Cardiac MR

International audienceIn medical image segmentation, several studies have used Bayesian neural networks to segment and quantify the uncertainty of the images. These studies show that there might be an increased epistemic uncertainty in areas where there are semantically and visually challenging pixels. The uncertain areas of the image can be of a great interest as they can possibly indicate the regions of incorrect segmentation. To leverage the uncertainty information, we propose a segmentation model that incorporates the uncertainty into its learning process. Firstly, we generate the uncertainty estimate (sample variance) using Monte-Carlo dropout during training. Then we incorporate it into the loss function to improve the segmentation accuracy and probability calibration. The proposed method is validated on the publicly available EMIDEC MIC-CAI 2020 dataset that mainly focuses on segmentation of healthy and infarcted myocardium. Our method achieves the state of the art results outperforming the top ranked methods of the challenge. The experimental results show that adding the uncertainty information to the loss function improves the segmentation results by enhancing the geometrical and clinical segmentation metrics of both the scar and myocardium. These improvements are particularly significant at the visually challenging and difficult images which have higher epistemic uncertainty. The proposed system also produces more calibrated probabilities

Using MRI-specific Data Augmentation to Enhance the Segmentation of Right Ventricle in Multi-disease, Multi-center and Multi-view Cardiac MRI

International audienc

Automatic uncertainty-based quality controlled T1 mapping and ECV analysis from native and post-contrast cardiac T1 mapping images using Bayesian vision transformer

International audienc

An Improved 3D Deep Learning-Based Segmentation of Left Ventricular Myocardial Diseases from Delayed-Enhancement MRI with Inclusion and Classification Prior Information U-Net (ICPIU-Net)

International audienceAccurate segmentation of the myocardial scar may supply relevant advancements in predicting and controlling deadly ventricular arrhythmias in subjects with cardiovascular disease. In this paper, we propose the architecture of inclusion and classification of prior information U-Net (ICPIU-Net) to efficiently segment the left ventricle (LV) myocardium, myocardial infarction (MI), and microvascular-obstructed (MVO) tissues from late gadolinium enhancement magnetic resonance (LGE-MR) images. Our approach was developed using two subnets cascaded to first segment the LV cavity and myocardium. Then, we used inclusion and classification constraint networks to improve the resulting segmentation of the diseased regions within the pre-segmented LV myocardium. This network incorporates the inclusion and classification information of the LGE-MRI to maintain topological constraints of pathological areas. In the testing stage, the outputs of each segmentation network obtained with specific estimated parameters from training were fused using the majority voting technique for the final label prediction of each voxel in the LGE-MR image. The proposed method was validated by comparing its results to manual drawings by experts from 50 LGE-MR images. Importantly, compared to various deep learning-based methods participating in the EMIDEC challenge, the results of our approach have a more significant agreement with manual contouring in segmenting myocardial diseases

Deep Learning Segmentation of the Right Ventricle in Cardiac MRI:The M&Ms challenge

In recent years, several deep learning models have been proposed to accurately quantify and diagnose cardiac pathologies. These automated tools heavily rely on the accurate segmentation of cardiac structures in MRI images. However, segmentation of the right ventricle is challenging due to its highly complex shape and ill-defined borders. Hence, there is a need for new methods to handle such structure's geometrical and textural complexities, notably in the presence of pathologies such as Dilated Right Ventricle, Tricuspid Regurgitation, Arrhythmogenesis, Tetralogy of Fallot, and Inter-atrial Communication. The last MICCAI challenge on right ventricle segmentation was held in 2012 and included only 48 cases from a single clinical center. As part of the 12th Workshop on Statistical Atlases and Computational Models of the Heart (STACOM 2021), the M&Ms-2 challenge was organized to promote the interest of the research community around right ventricle segmentation in multi-disease, multi-view, and multi-center cardiac MRI. Three hundred sixty CMR cases, including short-axis and long-axis 4-chamber views, were collected from three Spanish hospitals using nine different scanners from three different vendors, and included a diverse set of right and left ventricle pathologies. The solutions provided by the participants show that nnU-Net achieved the best results overall. However, multi-view approaches were able to capture additional information, highlighting the need to integrate multiple cardiac diseases, views, scanners, and acquisition protocols to produce reliable automatic cardiac segmentation algorithms