Segmentation of fetal 2D images with deep learning: a review

Image segmentation plays a vital role in providing sustainable medical care in this evolving biomedical image processing technology. Nowadays, it is considered one of the most important research directions in the computer vision field. Since the last decade, deep learning-based medical image processing has become a research hotspot due to its exceptional performance. In this paper, we present a review of different deep learning techniques used to segment fetal 2D images. First, we explain the basic ideas of each approach and then thoroughly investigate the methods used for the segmentation of fetal images. Secondly, the results and accuracy of different approaches are also discussed. The dataset details used for assessing the performance of the respective method are also documented. Based on the review studies, the challenges and future work are also pointed out at the end. As a result, it is shown that deep learning techniques are very effective in the segmentation of fetal 2D images.info:eu-repo/semantics/publishedVersio

3D T2w fetal body MRI:automated organ volumetry, growth charts and population-averaged atlas

Structural fetal body MRI provides true 3D information required for volumetry of fetal organs. However, current clinical and research practice primarily relies on manual slice-wise segmentation of raw T2-weighted stacks, which is time consuming, subject to inter- and intra-observer bias and affected by motion-corruption. Furthermore, there are no existing standard guidelines defining a universal approach to parcellation of fetal organs. This work produces the first parcellation protocol of the fetal body organs for motion-corrected 3D fetal body MRI. It includes 10 organ ROIs relevant to fetal quantitative volumetry studies. We also introduce the first population-averaged T2w MRI atlas of the fetal body. The protocol was used as a basis for training of a neural network for automated organ segmentation. It showed robust performance for different gestational ages. This solution minimises the need for manual editing and significantly reduces time. The general feasibility of the proposed pipeline was also assessed by analysis of organ growth charts created from automated parcellations of 91 normal control 3T MRI datasets that showed expected increase in volumetry during 22-38 weeks gestational age range. In addition, the results of comparison between 60 normal and 12 fetal growth restriction datasets revealed significant differences in organ volumes.</p

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Multi-task learning for joint weakly-supervised segmentation and aortic arch anomaly classification in fetal cardiac MRI

Congenital Heart Disease (CHD) is a group of cardiac malformations present already during fetal life, representing the prevailing category of birth defects globally. Our aim in this study is to aid 3D fetal vessel topology visualisation in aortic arch anomalies, a group which encompasses a range of conditions with significant anatomical heterogeneity. We present a multi-task framework for automated multi-class fetal vessel segmentation from 3D black blood T2w MRI and anomaly classification. Our training data consists of binary manual segmentation masks of the cardiac vessels' region in individual subjects and fully-labelled anomaly-specific population atlases. Our framework combines deep learning label propagation using VoxelMorph with 3D Attention U-Net segmentation and DenseNet121 anomaly classification. We target 11 cardiac vessels and three distinct aortic arch anomalies, including double aortic arch, right aortic arch, and suspected coarctation of the aorta. We incorporate an anomaly classifier into our segmentation pipeline, delivering a multi-task framework with the primary motivation of correcting topological inaccuracies of the segmentation. The hypothesis is that the multi-task approach will encourage the segmenter network to learn anomaly-specific features. As a secondary motivation, an automated diagnosis tool may have the potential to enhance diagnostic confidence in a decision support setting. Our results showcase that our proposed training strategy significantly outperforms label propagation and a network trained exclusively on propagated labels. Our classifier outperforms a classifier trained exclusively on T2w volume images, with an average balanced accuracy of 0.99 (0.01) after joint training. Adding a classifier improves the anatomical and topological accuracy of all correctly classified double aortic arch subjects.Comment: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:01

Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

Test-time augmentation-based active learning and self-training for label-efficient segmentation

Deep learning techniques depend on large datasets whose annotation is time-consuming. To reduce annotation burden, the self-training (ST) and active-learning (AL) methods have been developed as well as methods that combine them in an iterative fashion. However, it remains unclear when each method is the most useful, and when it is advantageous to combine them. In this paper, we propose a new method that combines ST with AL using Test-Time Augmentations (TTA). First, TTA is performed on an initial teacher network. Then, cases for annotation are selected based on the lowest estimated Dice score. Cases with high estimated scores are used as soft pseudo-labels for ST. The selected annotated cases are trained with existing annotated cases and ST cases with border slices annotations. We demonstrate the method on MRI fetal body and placenta segmentation tasks with different data variability characteristics. Our results indicate that ST is highly effective for both tasks, boosting performance for in-distribution (ID) and out-of-distribution (OOD) data. However, while self-training improved the performance of single-sequence fetal body segmentation when combined with AL, it slightly deteriorated performance of multi-sequence placenta segmentation on ID data. AL was helpful for the high variability placenta data, but did not improve upon random selection for the single-sequence body data. For fetal body segmentation sequence transfer, combining AL with ST following ST iteration yielded a Dice of 0.961 with only 6 original scans and 2 new sequence scans. Results using only 15 high-variability placenta cases were similar to those using 50 cases. Code is available at: https://github.com/Bella31/TTA-quality-estimation-ST-ALComment: Accepted to MICCAI MILLanD workshop 202

Trustworthy Deep Learning for Medical Image Segmentation

Despite the recent success of deep learning methods at achieving new state-of-the-art accuracy for medical image segmentation, some major limitations are still restricting their deployment into clinics. One major limitation of deep learning-based segmentation methods is their lack of robustness to variability in the image acquisition protocol and in the imaged anatomy that were not represented or were underrepresented in the training dataset. This suggests adding new manually segmented images to the training dataset to better cover the image variability. However, in most cases, the manual segmentation of medical images requires highly skilled raters and is time-consuming, making this solution prohibitively expensive. Even when manually segmented images from different sources are available, they are rarely annotated for exactly the same regions of interest. This poses an additional challenge for current state-of-the-art deep learning segmentation methods that rely on supervised learning and therefore require all the regions of interest to be segmented for all the images to be used for training. This thesis introduces new mathematical and optimization methods to mitigate those limitations.Comment: PhD thesis successfully defended on 1st July 2022. Examiners: Prof Sotirios Tsaftaris and Dr Wenjia Ba

Differences in callosal and subcortical volumes and associated neurobehavioural deficits in children with prenatal alcohol exposure

Certain high-risk communities in the Western Cape Province of South Africa where heavy maternal prenatal alcohol consumption is perpetuated by historical and societal challenges, have some of the highest prevalence rates of fetal alcohol syndrome (FAS) in the world. FAS has lifelong behavioural and cognitive consequences. Neuroimaging research aims to link deficits in brain structure and function to behavioural outcomes. Manual tracing is considered the gold standard of neuroanatomical volumetric analysis. Combined with neurobehavioural testing it can provide links between structure and function, but is time consuming and labour intensive. Automated segmentation programmes, such as FreeSurfer, are a faster alternative. The challenge is creating automated programmes that can provide results that are comparable to manual tracing, especially in a clinical sample. The aims of this thesis were to investigate (1) the effects of prenatal alcohol exposure (PAE) on the sizes of the caudate nucleus, nucleus accumbens, hippocampus and corpus callosum (CC) and potential relations of regional volumes with IQ and verbal learning, (2) to compare the performance of manual and automated segmentation methods in identifying alcohol-related changes in brain morphometry, and (3) to examine the effects of PAE on inter-hemispheric transfer during adolescence and potential relations of CC size with inter-hemispheric transfer deficits. Participants for this project were recruited from the Cape Town Longitudinal Cohort for whom alcohol exposure data were gathered prospectively from the mothers during pregnancy using the timeline follow-back approach. Participants had been diagnosed previously by two expert dysmorphologists as either control, non-syndromal heavily exposed (HE), partial FAS (PFAS) or FAS. High-resolution T1-weighted images were acquired using a sequence optimized for morphometric neuroanatomical analysis on a Siemens 3T Allegra MRI scanner for 71 right-handed children (9 FAS, 19 PFAS, 24 HE and 19 non-exposed controls) from this cohort at ages 9-11 years. Bilateral caudate nuclei, nucleus accumbens and hippocampi and the CC were manually traced using Multitracer. FreeSurfer was used for automated segmentation. All structures were segmented with both FreeSurfer versions 5.1 and 6.0 to compare progress within development of automated segmentation algorithms. Associations of volumes from manual tracing with IQ and performance on the California Verbal Learning Test-Children’s Version (CVLT-C) were also examined. Inter-hemispheric transfer was assessed using a finger localization task (FLT) administered to 74 participants (12 FAS, 16 PFAS, 14 HE, and 32 controls) from the same cohort at ages 16-17 years. Of these, 34 participants had completed MRI at 9-11 years. Higher levels of PAE were associated with reductions in CC area, as well as bilateral volume reductions in caudate nuclei and hippocampi, effects that remained significant after controlling for alcohol-related reductions in TIV (total intracranial volume). Amongst dysmorphic children (FAS/PFAS), poorer performance on the CVLT-C was related to larger hippocampi and smaller CC. Smaller CC was also associated with lower IQ and partially mediated the effect of PAE on IQ. Manual and automated comparisons showed good agreement in the caudate nuclei, which are simpler to segment, moderate to good agreement in the smaller, more complex nucleus accumbens and hippocampi, and poor agreement in the CC. The latter is not surprising, however, in view of the fact that manual tracing measured the average area of the CC on a mid-sagittal slice, while FreeSurfer measures CC volume over a number of contiguous slices. After controlling for confounders and adjustment for smaller TIV, the latest FreeSurfer version 6.0 provided evidence of alcohol-related volumetric brain reductions comparable to manual segmentation. Only the most severely affected children with FAS demonstrated inter-hemispheric transfer deficits, with the number of transfer-related errors tending to increase with decreasing CC volume among children with PAE. This study confirms and extends evidence of PAE-related decreases in subcortical and CC size and that callosal volume partially mediates alcohol-related impairment in IQ. Although FreeSurfer v 6.0 achieves automated segmentations that are comparable to manual tracing, even in a paediatric clinical sample, performance is more reliable in some structures than others. Improvement and standardization of CC segmentation is especially important given the vulnerability of the CC and its critical role in domains affected by PAE, including verbal learning, IQ and inter-hemispheric transfer of information

Volumetric MRI Reconstruction from 2D Slices in the Presence of Motion

Despite recent advances in acquisition techniques and reconstruction algorithms, magnetic resonance imaging (MRI) remains challenging in the presence of motion. To mitigate this, ultra-fast two-dimensional (2D) MRI sequences are often used in clinical practice to acquire thick, low-resolution (LR) 2D slices to reduce in-plane motion. The resulting stacks of thick 2D slices typically provide high-quality visualizations when viewed in the in-plane direction. However, the low spatial resolution in the through-plane direction in combination with motion commonly occurring between individual slice acquisitions gives rise to stacks with overall limited geometric integrity. In further consequence, an accurate and reliable diagnosis may be compromised when using such motion-corrupted, thick-slice MRI data. This thesis presents methods to volumetrically reconstruct geometrically consistent, high-resolution (HR) three-dimensional (3D) images from motion-corrupted, possibly sparse, low-resolution 2D MR slices. It focuses on volumetric reconstructions techniques using inverse problem formulations applicable to a broad field of clinical applications in which associated motion patterns are inherently different, but the use of thick-slice MR data is current clinical practice. In particular, volumetric reconstruction frameworks are developed based on slice-to-volume registration with inter-slice transformation regularization and robust, complete-outlier rejection for the reconstruction step that can either avoid or efficiently deal with potential slice-misregistrations. Additionally, this thesis describes efficient Forward-Backward Splitting schemes for image registration for any combination of differentiable (not necessarily convex) similarity measure and convex (not necessarily smooth) regularization with a tractable proximal operator. Experiments are performed on fetal and upper abdominal MRI, and on historical, printed brain MR films associated with a uniquely long-term study dating back to the 1980s. The results demonstrate the broad applicability of the presented frameworks to achieve robust reconstructions with the potential to improve disease diagnosis and patient management in clinical practice