Deep learning for image-based liver analysis — A comprehensive review focusing on malignant lesions

Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.publishedVersio

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

Incorporating Cardiac Substructures Into Radiation Therapy For Improved Cardiac Sparing

Growing evidence suggests that radiation therapy (RT) doses to the heart and cardiac substructures (CS) are strongly linked to cardiac toxicities, though only the heart is considered clinically. This work aimed to utilize the superior soft-tissue contrast of magnetic resonance (MR) to segment CS, quantify uncertainties in their position, assess their effect on treatment planning and an MR-guided environment. Automatic substructure segmentation of 12 CS was completed using a novel hybrid MR/computed tomography (CT) atlas method and was improved upon using a 3-dimensional neural network (U-Net) from deep learning. Intra-fraction motion due to respiration was then quantified. The inter-fraction setup uncertainties utilizing a novel MR-linear accelerator were also quantified. Treatment planning comparisons were performed with and without substructure inclusions and methods to reduce radiation dose to sensitive CS were evaluated. Lastly, these described technologies (deep learning U-Net) were translated to an MR-linear accelerator and a segmentation pipeline was created. Automatic segmentations from the hybrid MR/CT atlas was able to generate accurate segmentations for the chambers and great vessels (Dice similarity coefficient (DSC) \u3e 0.75) but coronary artery segmentations were unsuccessful (DSC\u3c0.3). After implementing deep learning, DSC for the chambers and great vessels was ≥0.85 along with an improvement in the coronary arteries (DSC\u3e0.5). Similar accuracy was achieved when implementing deep learning for MR-guided RT. On average, automatic segmentations required ~10 minutes to generate per patient and deep learning only required 14 seconds. The inclusion of CS in the treatment planning process did not yield statistically significant changes in plan complexity, PTV, or OAR dose. Automatic segmentation results from deep learning pose major efficiency and accuracy gains for CS segmentation offering high potential for rapid implementation into radiation therapy planning for improved cardiac sparing. Introducing CS into RT planning for MR-guided RT presented an opportunity for more effective sparing with limited increase in plan complexity

Improving Dose-Response Correlations for Locally Advanced NSCLC Patients Treated with IMRT or PSPT

The standard of care for locally advanced non-small cell lung cancer (NSCLC) is concurrent chemo-radiotherapy. Despite recent advancements in radiation delivery methods, the median survival time of NSCLC patients remains below 28 months. Higher tumor dose has been found to increase survival but also a higher rate of radiation pneumonitis (RP) that affects breathing capability. In fear of such toxicity, less-aggressive treatment plans are often clinically preferred, leading to metastasis and recurrence. Therefore, accurate RP prediction is crucial to ensure tumor coverage to improve treatment outcome. Current models have associated RP with increased dose but with limited accuracy as they lack spatial correlation between accurate dose representation and quantitative RP representation. These models represent lung tissue damage with radiation dose distribution planned pre-treatment, which assumes a fixed patient geometry and inevitably renders imprecise dose delivery due to intra-fractional breathing motion and inter-fractional anatomy response. Additionally, current models employ whole-lung dose metrics as the contributing factor to RP as a qualitative, binary outcome but these global dose metrics discard microscopic, voxel-(3D pixel)-level information and prevent spatial correlations with quantitative RP representation. To tackle these limitations, we developed advanced deformable image registration (DIR) techniques that registered corresponding anatomical voxels between images for tracking and accumulating dose throughout treatment. DIR also enabled voxel-level dose-response correlation when CT image density change (IDC) was used to quantify RP. We hypothesized that more accurate estimates of biologically effective dose distributions actually delivered, achieved through (a) dose accumulation using deformable registration of weekly 4DCT images acquired over the course or radiotherapy and (b) the incorporation of variable relative biological effectiveness (RBE), would lead to statistically and clinically significant improvement in the correlation of RP with biologically effective dose distributions. Our work resulted in a robust intra-4DCT and inter-4DCT DIR workflow, with the accuracy meeting AAPM TG-132 recommendations for clinical implementation of DIR. The automated DIR workflow allowed us to develop a fully automated 4DCT-based dose accumulation pipeline in RayStation (RaySearch Laboratories, Stockholm, Sweden). With a sample of 67 IMRT patients, our results showed that the accumulated dose was statistically different than the planned dose across the entire cohort with an average MLD increase of ~1 Gy and clinically different for individual patients where 16% resulted in difference in the score of the normal tissue complication probability (NTCP) using an established, clinically used model, which could qualify the patients for treatment planning re-evaluation. Lastly, we associated dose difference with accuracy difference by establishing and comparing voxel-level dose-IDC correlations and concluded that the accumulated dose better described the localized damage, thereby a closer representation of the delivered dose. Using the same dose-response correlation strategy, we plotted the dose-IDC relationships for both photon patients (N = 51) and proton patients (N = 67), we measured the variable proton RBE values to be 3.07–1.27 from 9–52 Gy proton voxels. With the measured RBE values, we fitted an established variable proton RBE model with pseudo-R2 of 0.98. Therefore, our results led to statistically and clinically significant improvement in the correlation of RP with accumulated and biologically effective dose distributions and demonstrated the potential of incorporating the effect of anatomical change and biological damage in RP prediction models

CT liver tumor segmentation hybrid approach using neutrosophic sets, fast fuzzy c-means and adaptive watershed algorithm

Liver tumor segmentation from computed tomography (CT) images is a critical and challenging task. Due to the fuzziness in the liver pixel range, the neighboring organs of the liver with the same intensity, high noise and large variance of tumors. The segmentation process is necessary for the detection, identification, and measurement of objects in CT images. We perform an extensive review of the CT liver segmentation literature

Automated liver tissues delineation based on machine learning techniques: A survey, current trends and future orientations

There is no denying how machine learning and computer vision have grown in the recent years. Their highest advantages lie within their automation, suitability, and ability to generate astounding results in a matter of seconds in a reproducible manner. This is aided by the ubiquitous advancements reached in the computing capabilities of current graphical processing units and the highly efficient implementation of such techniques. Hence, in this paper, we survey the key studies that are published between 2014 and 2020, showcasing the different machine learning algorithms researchers have used to segment the liver, hepatic-tumors, and hepatic-vasculature structures. We divide the surveyed studies based on the tissue of interest (hepatic-parenchyma, hepatic-tumors, or hepatic-vessels), highlighting the studies that tackle more than one task simultaneously. Additionally, the machine learning algorithms are classified as either supervised or unsupervised, and further partitioned if the amount of works that fall under a certain scheme is significant. Moreover, different datasets and challenges found in literature and websites, containing masks of the aforementioned tissues, are thoroughly discussed, highlighting the organizers original contributions, and those of other researchers. Also, the metrics that are used excessively in literature are mentioned in our review stressing their relevancy to the task at hand. Finally, critical challenges and future directions are emphasized for innovative researchers to tackle, exposing gaps that need addressing such as the scarcity of many studies on the vessels segmentation challenge, and why their absence needs to be dealt with in an accelerated manner.Comment: 41 pages, 4 figures, 13 equations, 1 table. A review paper on liver tissues segmentation based on automated ML-based technique

Respiratory organ motion in interventional MRI : tracking, guiding and modeling

Respiratory organ motion is one of the major challenges in interventional MRI, particularly in interventions with therapeutic ultrasound in the abdominal region. High-intensity focused ultrasound found an application in interventional MRI for noninvasive treatments of different abnormalities. In order to guide surgical and treatment interventions, organ motion imaging and modeling is commonly required before a treatment start. Accurate tracking of organ motion during various interventional MRI procedures is prerequisite for a successful outcome and safe therapy. In this thesis, an attempt has been made to develop approaches using focused ultrasound which could be used in future clinically for the treatment of abdominal organs, such as the liver and the kidney. Two distinct methods have been presented with its ex vivo and in vivo treatment results. In the first method, an MR-based pencil-beam navigator has been used to track organ motion and provide the motion information for acoustic focal point steering, while in the second approach a hybrid imaging using both ultrasound and magnetic resonance imaging was combined for advanced guiding capabilities. Organ motion modeling and four-dimensional imaging of organ motion is increasingly required before the surgical interventions. However, due to the current safety limitations and hardware restrictions, the MR acquisition of a time-resolved sequence of volumetric images is not possible with high temporal and spatial resolution. A novel multislice acquisition scheme that is based on a two-dimensional navigator, instead of a commonly used pencil-beam navigator, was devised to acquire the data slices and the corresponding navigator simultaneously using a CAIPIRINHA parallel imaging method. The acquisition duration for four-dimensional dataset sampling is reduced compared to the existing approaches, while the image contrast and quality are improved as well. Tracking respiratory organ motion is required in interventional procedures and during MR imaging of moving organs. An MR-based navigator is commonly used, however, it is usually associated with image artifacts, such as signal voids. Spectrally selective navigators can come in handy in cases where the imaging organ is surrounding with an adipose tissue, because it can provide an indirect measure of organ motion. A novel spectrally selective navigator based on a crossed-pair navigator has been developed. Experiments show the advantages of the application of this novel navigator for the volumetric imaging of the liver in vivo, where this navigator was used to gate the gradient-recalled echo sequence