Deep learning for image-based liver analysis — A comprehensive review focusing on malignant lesions

Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.publishedVersio

Automated liver tissues delineation based on machine learning techniques: A survey, current trends and future orientations

There is no denying how machine learning and computer vision have grown in the recent years. Their highest advantages lie within their automation, suitability, and ability to generate astounding results in a matter of seconds in a reproducible manner. This is aided by the ubiquitous advancements reached in the computing capabilities of current graphical processing units and the highly efficient implementation of such techniques. Hence, in this paper, we survey the key studies that are published between 2014 and 2020, showcasing the different machine learning algorithms researchers have used to segment the liver, hepatic-tumors, and hepatic-vasculature structures. We divide the surveyed studies based on the tissue of interest (hepatic-parenchyma, hepatic-tumors, or hepatic-vessels), highlighting the studies that tackle more than one task simultaneously. Additionally, the machine learning algorithms are classified as either supervised or unsupervised, and further partitioned if the amount of works that fall under a certain scheme is significant. Moreover, different datasets and challenges found in literature and websites, containing masks of the aforementioned tissues, are thoroughly discussed, highlighting the organizers original contributions, and those of other researchers. Also, the metrics that are used excessively in literature are mentioned in our review stressing their relevancy to the task at hand. Finally, critical challenges and future directions are emphasized for innovative researchers to tackle, exposing gaps that need addressing such as the scarcity of many studies on the vessels segmentation challenge, and why their absence needs to be dealt with in an accelerated manner.Comment: 41 pages, 4 figures, 13 equations, 1 table. A review paper on liver tissues segmentation based on automated ML-based technique

PET/MRI 및 MR-IGRT를 위한 MRI 기반 합성 CT 생성의 타당성 연구

학위논문 (박사) -- 서울대학교 대학원 : 의과대학 의과학과, 2020. 8. 이재성.Over the past decade, the application of magnetic resonance imaging (MRI) in the field of diagnosis and treatment has increased. MRI provides higher soft-tissue contrast, especially in the brain, abdominal organ, and bone marrow without the expose of ionizing radiation. Hence, simultaneous positron emission tomography/MR (PET/MR) system and MR-image guided radiation therapy (MR-IGRT) system has recently been emerged and currently available for clinical study. One major issue in PET/MR system is attenuation correction from MRI scans for PET quantification and a similar need for the assignment of electron densities to MRI scans for dose calculation can be found in MR-IGRT system. Because the MR signals are related to the proton density and relaxation properties of tissue, not to electron density. To overcome this problem, the method called synthetic CT (sCT), a pseudo CT derived from MR images, has been proposed. In this thesis, studies on generating synthetic CT and investigating the feasibility of using a MR-based synthetic CT for diagnostic and radiotherapy application were presented. Firstly, MR image-based attenuation correction (MR-AC) method using level-set segmentation for brain PET/MRI was developed. To resolve conventional inaccuracy MR-AC problem, we proposed an improved ultrashort echo time MR-AC method that was based on a multiphase level-set algorithm with main magnetic field inhomogeneity correction. We also assessed the feasibility of level-set based MR-AC method, compared with CT-AC and MR-AC provided by the manufacturer of the PET/MRI scanner. Secondly, we proposed sCT generation from the low field MR images using 2D convolution neural network model for MR-IGRT system. This sCT images were compared to the deformed CT generated using the deformable registration being used in the current system. We assessed the feasibility of using sCT for radiation treatment planning from each of the patients with pelvic, thoraic and abdominal region through geometric and dosimetric evaluation.지난 10년간 진단 및 치료분야에서 자기공명영상(Magnetic resonance imaging; MRI) 의 적용이 증가하였다. MRI는 CT와 비교해 추가적인 전리방사선의 피폭없이 뇌, 복부 기관 및 골수 등에서 더 높은 연조직 대비를 제공한다. 따라서 MRI를 적용한 양전자방출단층촬영(Positron emission tomography; PET)/MR 시스템과 MR 영상 유도 방사선 치료 시스템(MR-image guided radiation therapy; MR-IGRT)이 진단 및 치료 방사선분야에 등장하여 임상에 사용되고 있다. PET/MR 시스템의 한 가지 주요 문제는 PET 정량화를 위한 MRI 스캔으로부터의 감쇠 보정이며, MR-IGRT 시스템에서 선량 계산을 위해 MR 영상에 전자 밀도를 할당하는 것과 비슷한 필요성을 찾을 수 있다. 이는 MR 신호가 전자 밀도가 아닌 조직의 양성자 밀도 및 T1, T2 이완 특성과 관련이 있기 때문이다. 이 문제를 극복하기 위해, MR 이미지로부터 유래된 가상의 CT인 합성 CT라 불리는 방법이 제안되었다. 본 학위논문에서는 합성 CT 생성 방법 및 진단 및 방사선 치료에 적용을 위한 MR 영상 기반 합성 CT 사용의 임상적 타당성을 조사하였다. 첫째로, 뇌 PET/MR를 위한 레벨셋 분할을 이용한 MR 이미지 기반 감쇠 보정 방법을 개발하였다. MR 이미지 기반 감쇠 보정의 부정확성은 정량화 오류와 뇌 PET/MRI 연구에서 병변의 잘못된 판독으로 이어진다. 이 문제를 해결하기 위해, 자기장 불균일 보정을 포함한 다상 레벨셋 알고리즘에 기초한 개선된 초단파 에코 시간 MR-AC 방법을 제안하였다. 또한 CT-AC 및 PET/MRI 스캐너 제조업체가 제공한 MR-AC와 비교하여 레벨셋 기반 MR-AC 방법의 임상적 사용가능성을 평가하였다. 둘째로, MR-IGRT 시스템을 위한 심층 컨볼루션 신경망 모델을 사용하여 저필드 MR 이미지에서 생성된 합성 CT 방법를 제안하였다. 이 합성 CT 이미지를 변형 정합을 사용하여 생성된 변형 CT와 비교 하였다. 또한 골반, 흉부 및 복부 환자에서의 기하학적, 선량적 분석을 통해 방사선 치료계획에서의 합성 CT를 사용가능성을 평가하였다.Chapter 1. Introduction 1 1.1. Background 1 1.1.1. The Integration of MRI into Other Medical Devices 1 1.1.2. Chanllenges in the MRI Integrated System 4 1.1.3. Synthetic CT Generation 5 1.2. Purpose of Research 6 Chapter 2. MRI-based Attenuation Correction for PET/MRI 8 2.1. Background 8 2.2. Materials and Methods 10 2.2.1. Brain PET Dataset 19 2.2.2. MR-Based Attenuation Map using Level-Set Algorithm 12 2.2.3. Image Processing and Reconstruction 18 2.3. Results 20 2.4. Discussion 28 Chapter 3. MRI-based synthetic CT generation for MR-IGRT 30 3.1. Background 30 3.2. Materials and Methods 32 3.2.1. MR-dCT Paired DataSet 32 3.2.2. Synthetic CT Generation using 2D CNN 36 3.2.3. Data Analysis 38 3.3. Results 41 3.3.1. Image Comparison 41 3.3.2. Geometric Analysis 49 3.3.3. Dosimetric Analysis 49 3.4. Discussion 56 Chapter 4. Conclusions 59 Bibliography 60 Abstract in Korean (국문 초록) 64Docto

Impact of Image Denoising Techniques on CNN-based Liver Vessel Segmentation using Synthesis Low-dose Contrast Enhanced CT Images

Liver vessel segmentation in contrast enhanced CT (CECT) image is relevant for several clinical applications. However, the liver segmentation on noisy images obtain incorrect liver vessel segmentation which may lead to distortion in the simulation of cooling effect near the vessels during the planning. In this study, we present a framework that consists of three well-known and state-of-the-art denoising techniques, Vesssel enhancing diffusion (VED), RED-CNN, and MAP-NN and using a state-of-the-art Convolution Neural Networks (nn-Unet) to segment the liver vessels from the CECT images. The impact of denoising methods on the vessel segmentation are ablated using with multi-level simulated low-dose CECT of the liver. The experiment is carried on CECT images of the liver from two public and one private datasets. We evaluate the performance of the framework using Dice score and sensitivity criteria. Furthermore, we investigate the efficient of denoising on roughness of the surface of liver vessel segmentation. The results from our experiment suggest that denoising methods can improve the liver vessel segmentation quality in the CECT image with high low-dose noise while they degrade the liver vessel segmentation accuracy for low-noise-level CECT images

Automated liver tissues delineation techniques: A systematic survey on machine learning current trends and future orientations

Machine learning and computer vision techniques have grown rapidly in recent years due to their automation, suitability, and ability to generate astounding results. Hence, in this paper, we survey the key studies that are published between 2014 and 2022, showcasing the different machine learning algorithms researchers have used to segment the liver, hepatic tumors, and hepatic-vasculature structures. We divide the surveyed studies based on the tissue of interest (hepatic-parenchyma, hepatic-tumors, or hepatic-vessels), highlighting the studies that tackle more than one task simultaneously. Additionally, the machine learning algorithms are classified as either supervised or unsupervised, and they are further partitioned if the amount of work that falls under a certain scheme is significant. Moreover, different datasets and challenges found in literature and websites containing masks of the aforementioned tissues are thoroughly discussed, highlighting the organizers' original contributions and those of other researchers. Also, the metrics used excessively in the literature are mentioned in our review, stressing their relevance to the task at hand. Finally, critical challenges and future directions are emphasized for innovative researchers to tackle, exposing gaps that need addressing, such as the scarcity of many studies on the vessels' segmentation challenge and why their absence needs to be dealt with sooner than later. 2022 The Author(s)This publication was made possible by an Award [GSRA6-2-0521-19034] from Qatar National Research Fund (a member of Qatar Foundation). The contents herein are solely the responsibility of the authors. Open Access funding provided by the Qatar National LibraryScopu

Deep Semantic Segmentation of Natural and Medical Images: A Review

The semantic image segmentation task consists of classifying each pixel of an image into an instance, where each instance corresponds to a class. This task is a part of the concept of scene understanding or better explaining the global context of an image. In the medical image analysis domain, image segmentation can be used for image-guided interventions, radiotherapy, or improved radiological diagnostics. In this review, we categorize the leading deep learning-based medical and non-medical image segmentation solutions into six main groups of deep architectural, data synthesis-based, loss function-based, sequenced models, weakly supervised, and multi-task methods and provide a comprehensive review of the contributions in each of these groups. Further, for each group, we analyze each variant of these groups and discuss the limitations of the current approaches and present potential future research directions for semantic image segmentation.Comment: 45 pages, 16 figures. Accepted for publication in Springer Artificial Intelligence Revie