State of the Art in Artificial Intelligence and Radiomics in Hepatocellular Carcinoma

The most common liver malignancy is hepatocellular carcinoma (HCC), which is also associated with high mortality. Often HCC develops in a chronic liver disease setting, and early diagnosis as well as accurate screening of high-risk patients is crucial for appropriate and effective management of these patients. While imaging characteristics of HCC are well-defined in the diagnostic phase, challenging cases still occur, and current prognostic and predictive models are limited in their accuracy. Radiomics and machine learning (ML) offer new tools to address these issues and may lead to scientific breakthroughs with the potential to impact clinical practice and improve patient outcomes. In this review, we will present an overview of these technologies in the setting of HCC imaging across different modalities and a range of applications. These include lesion segmentation, diagnosis, prognostic modeling and prediction of treatment response. Finally, limitations preventing clinical application of radiomics and ML at the present time are discussed, together with necessary future developments to bring the field forward and outside of a purely academic endeavor

Dynamic And Quantitative Radiomics Analysis In Interventional Radiology

Interventional Radiology (IR) is a subspecialty of radiology that performs invasive procedures driven by diagnostic imaging for predictive and therapeutic purpose. The development of artificial intelligence (AI) has revolutionized the industry of IR. Researchers have created sophisticated models backed by machine learning algorithms and optimization methodologies for image registration, cellular structure detection and computer-aided disease diagnosis and prognosis predictions. However, due to the incapacity of the human eye to detect tiny structural characteristics and inter-radiologist heterogeneity, conventional experience-based IR visual evaluations may have drawbacks. Radiomics, a technique that utilizes machine learning, offers a practical and quantifiable solution to this issue. This technology has been used to evaluate the heterogeneity of malignancies that are difficult to detect by the human eye by creating an automated pipeline for the extraction and analysis of high throughput computational imaging characteristics from radiological medical pictures. However, it is a demanding task to directly put radiomics into applications in IR because of the heterogeneity and complexity of medical imaging data. Furthermore, recent radiomics studies are based on static images, while many clinical applications (such as detecting the occurrence and development of tumors and assessing patient response to chemotherapy and immunotherapy) is a dynamic process. Merely incorporating static features cannot comprehensively reflect the metabolic characteristics and dynamic processes of tumors or soft tissues. To address these issues, we proposed a robust feature selection framework to manage the high-dimensional small-size data. Apart from that, we explore and propose a descriptor in the view of computer vision and physiology by integrating static radiomics features with time-varying information in tumor dynamics. The major contributions to this study include: Firstly, we construct a result-driven feature selection framework, which could efficiently reduce the dimension of the original feature set. The framework integrates different feature selection techniques to ensure the distinctiveness, uniqueness, and generalization ability of the output feature set. In the task of classification hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC) in primary liver cancer, only three radiomics features (chosen from more than 1, 800 features of the proposed framework) can obtain an AUC of 0.83 in the independent dataset. Besides, we also analyze features’ pattern and contributions to the results, enhancing clinical interpretability of radiomics biomarkers. Secondly, we explore and build a pulmonary perfusion descriptor based on 18F-FDG whole-body dynamic PET images. Our major novelties include: 1) propose a physiology-and-computer-vision-interpretable descriptor construction framework by the decomposition of spatiotemporal information into three dimensions: shades of grey levels, textures, and dynamics. 2) The spatio-temporal comparison of pulmonary descriptor intra and inter patients is feasible, making it possible to be an auxiliary diagnostic tool in pulmonary function assessment. 3) Compared with traditional PET metabolic biomarker analysis, the proposed descriptor incorporates image’s temporal information, which enables a better understanding of the time-various mechanisms and detection of visual perfusion abnormalities among different patients. 4) The proposed descriptor eliminates the impact of vascular branching structure and gravity effect by utilizing time warping algorithms. Our experimental results showed that our proposed framework and descriptor are promising tools to medical imaging analysis

Prediction of Chemotherapy Response of Liver Metastases from Baseline CT-Images Using Deep Neural Networks

Dans les pays développés, le cancer colorectal est reconnu comme étant la deuxième cause la plus importante de mortalité liée au cancer. La chimiothérapie est considérée comme un traitement standard pour les métastases colorectales du foie (MCF). Parmi les patients qui développent des MCF, l’évaluation de la réponse du patient au traitement de chimiothérapie est souvent requise pour déterminer le besoin d’une chimiothérapie de seconde ligne, ainsi qu’une éligibilité à la chirurgie. Toutefois, tandis que les régimes basés sur un régime dénommé FOLFOX sont typiquement utilisés pour le traitement de la MCF, l’identification de la sensibilité du patient reste difficile. Les systèmes de diagnostic assistés par ordinateur peuvent fournir de l’information supplémentaire sur la classification des métastases du foie identifiées au niveau des images de diagnostic. Du aux quelques difficultés que rencontrent les radiologues pour distinguer, à l’oeil nu, les lésions traitées des lésions non-traitées, nous proposons dans cette étude un système automatisé basé sur les réseaux profonds convolutifs (RPC). Dans un premier lieu, ces réseaux profonds différencient les lésions traitées des lésions non-traitées, pour ensuite identifier les nouvelles lésions apparaissant sur les tomodensitométries. Ensuite, un réseau de neurones dense émet une prédiction, à partir des lésions non traitées visibles sur les tomodensitométries de prétraitement pour les patients à MCF sous chimiothérapie, sur leur réponse au régime spécifique de chimiothérapie. Dans ce contexte, la référence pour l’évaluation de la réponse au traitement pour le régime approprié de chimiothérapie était le degré de régression de la tumeur en histopathologie. La méthode adoptée dans cette étude nous a aidé à adresser les trois grands objectifs de ce projet de recherche. La première étape était de développer un système automatique de classification des tumeurs traitées et non-traitées à partir des tomodensitométries de patients. La deuxième étape a été de concevoir une nouvelle approche pour prédire la réponse au traitement FOLFOX qui utilise le médicament Bevacizumab en tant que traitement de première ligne. La troisième étape était de prédire le pourcentage de changement volumétrique de la tumeur suivant deux moments temporels consécutifs. Les algorithmes d’intelligence artificielle (IA), et les approches d’apprentissage profond en particulier, ont montré des progrès prometteurs en vision par ordinateur ainsi qu’en traitement d’images. Il existe de nombreuses applications en analyse d’images médicales qui utilisent des réseaux profonds convolutifs pour propulser ces progrès en avant le plus rapidement possible. En pratique, les radiologues et lesmédecins évaluent les images médicales visuellement pour le diagnostic, la détection, la récurrence, le suivi et la caractérisation des maladies. Les méthodes d’apprentissage profond sont souvent supérieures pour la reconnaissance automatique de structures complexes à partir d’images, et quantifier l’évaluation de propriétés et caractéristiques radiographiques.----------ABSTRACT: In developed countries, colorectal cancer is known as the second cause of cancer-related mortality. Chemotherapy is considered a standard treatment for colorectal liver metastases (CLM). Among patients who develop CLM, the assessment of patient response to chemotherapy is often required to determine the need for second-line chemotherapy, and eligibility for surgery. However, a drug regimen known as FOLFOX are typically used for CLM treatment, the identification of responsive patients remains elusive. Computer-aided diagnosis systems may provide insight into the classification of liver metastases identified on diagnostic images. Due to some difficulties for radiologists to distinguish between treated and untreated lesions from the naked eyes, in this study, we propose a fully automated framework based on deep convolutional neural networks (DCNN) which first differentiates treated and untreated lesions to identify new lesions appearing on CT scans, followed by a fully connected neural network to predict from untreated lesions in pre-treatment computed tomography (CT) for patients with CLMundergoing chemotherapy, their response to the specific chemotherapy regimen. In this respect, the ground truth for assessment of treatment response for proper chemotherapy regimen was histopathology to determine the tumor regression grade (TRG). The adopted method in this study helped us to address the three main research objectives. The first step is to develop an automated framework for the classification of treated and untreated tumors based on the patient’s CT images. The second step is to design a new approach for the prediction of response to FOLFOX regimens with Bevacizumab agents as the first-line of treatment. The third step is to predict the percentage of the tumor volume change following two consecutive exams. Artificial intelligence (AI) algorithms, particularly deep learning approach, have shown very astonishing progress in computer vision and image processing tasks. There are several applications in the medical image analysis area which use DCNN to propel these methods forward as quickly as possible. In practice, radiologists and physicians attempt to assess visually medical images for diagnosis, detection, recurrence, monitoring, and characterization of diseases. Deep learning methods surpass at the recognition of complicated patterns from imaging data automatically and quantify the assessment of radiographic features and characteristics

Medical Image Analytics (Radiomics) with Machine/Deeping Learning for Outcome Modeling in Radiation Oncology

Image-based quantitative analysis (radiomics) has gained great attention recently. Radiomics possesses promising potentials to be applied in the clinical practice of radiotherapy and to provide personalized healthcare for cancer patients. However, there are several challenges along the way that this thesis will attempt to address. Specifically, this thesis focuses on the investigation of repeatability and reproducibility of radiomics features, the development of new machine/deep learning models, and combining these for robust outcomes modeling and their applications in radiotherapy. Radiomics features suffer from robustness issues when applied to outcome modeling problems, especially in head and neck computed tomography (CT) images. These images tend to contain streak artifacts due to patients’ dental implants. To investigate the influence of artifacts for radiomics modeling performance, we firstly developed an automatic artifact detection algorithm using gradient-based hand-crafted features. Then, comparing the radiomics models trained on ‘clean’ and ‘contaminated’ datasets. The second project focused on using hand-crafted radiomics features and conventional machine learning methods for the prediction of overall response and progression-free survival for Y90 treated liver cancer patients. By identifying robust features and embedding prior knowledge in the engineered radiomics features and using bootstrapped LASSO to select robust features, we trained imaging and dose based models for the desired clinical endpoints, highlighting the complementary nature of this information in Y90 outcomes prediction. Combining hand-crafted and machine learnt features can take advantage of both expert domain knowledge and advanced data-driven approaches (e.g., deep learning). Thus, we proposed a new variational autoencoder network framework that modeled radiomics features, clinical factors, and raw CT images for the prediction of intrahepatic recurrence-free and overall survival for hepatocellular carcinoma (HCC) patients in this third project. The proposed approach was compared with widely used Cox proportional hazard model for survival analysis. Our proposed methods achieved significant improvement in terms of the prediction using the c-index metric highlighting the value of advanced modeling techniques in learning from limited and heterogeneous information in actuarial prediction of outcomes. Advances in stereotactic radiation therapy (SBRT) has led to excellent local tumor control with limited toxicities for HCC patients, but intrahepatic recurrence still remains prevalent. As an extension of the third project, we not only hope to predict the time to intrahepatic recurrence, but also the location where the tumor might recur. This will be clinically beneficial for better intervention and optimizing decision making during the process of radiotherapy treatment planning. To address this challenging task, firstly, we proposed an unsupervised registration neural network to register atlas CT to patient simulation CT and obtain the liver’s Couinaud segments for the entire patient cohort. Secondly, a new attention convolutional neural network has been applied to utilize multimodality images (CT, MR and 3D dose distribution) for the prediction of high-risk segments. The results showed much improved efficiency for obtaining segments compared with conventional registration methods and the prediction performance showed promising accuracy for anticipating the recurrence location as well. Overall, this thesis contributed new methods and techniques to improve the utilization of radiomics for personalized radiotherapy. These contributions included new algorithm for detecting artifacts, a joint model of dose with image heterogeneity, combining hand-crafted features with machine learnt features for actuarial radiomics modeling, and a novel approach for predicting location of treatment failure.PHDApplied PhysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163092/1/liswei_1.pd

Investigation of intra-tumour heterogeneity to identify texture features to characterise and quantify neoplastic lesions on imaging

The aim of this work was to further our knowledge of using imaging data to discover image derived biomarkers and other information about the imaged tumour. Using scans obtained from multiple centres to discover and validate the models has advanced earlier research and provided a platform for further larger centre prospective studies. This work consists of two major studies which are describe separately: STUDY 1: NSCLC Purpose The aim of this multi-center study was to discover and validate radiomics classifiers as image-derived biomarkers for risk stratification of non-small-cell lung cancer (NSCLC). Patients and methods Pre-therapy PET scans from 358 Stage I–III NSCLC patients scheduled for radical radiotherapy/chemoradiotherapy acquired between October 2008 and December 2013 were included in this seven-institution study. Using a semiautomatic threshold method to segment the primary tumors, radiomics predictive classifiers were derived from a training set of 133 scans using TexLAB v2. Least absolute shrinkage and selection operator (LASSO) regression analysis allowed data dimension reduction and radiomics feature vector (FV) discovery. Multivariable analysis was performed to establish the relationship between FV, stage and overall survival (OS). Performance of the optimal FV was tested in an independent validation set of 204 patients, and a further independent set of 21 (TESTI) patients. Results Of 358 patients, 249 died within the follow-up period [median 22 (range 0–85) months]. From each primary tumor, 665 three-dimensional radiomics features from each of seven gray levels were extracted. The most predictive feature vector discovered (FVX) was independent of known prognostic factors, such as stage and tumor volume, and of interest to multi-center studies, invariant to the type of PET/CT manufacturer. Using the median cut-off, FVX predicted a 14-month survival difference in the validation cohort (N = 204, p = 0.00465; HR = 1.61, 95% CI 1.16–2.24). In the TESTI cohort, a smaller cohort that presented with unusually poor survival of stage I cancers, FVX correctly indicated a lack of survival difference (N = 21, p = 0.501). In contrast to the radiomics classifier, clinically routine PET variables including SUVmax, SUVmean and SUVpeak lacked any prognostic information. Conclusion PET-based radiomics classifiers derived from routine pre-treatment imaging possess intrinsic prognostic information for risk stratification of NSCLC patients to radiotherapy/chemo-radiotherapy. STUDY 2: Ovarian Cancer Purpose The 5-year survival of epithelial ovarian cancer is approximately 35-40%, prompting the need to develop additional methods such as biomarkers for personalised treatment. Patient and Methods 657 texture features were extracted from the CT scans of 364 untreated EOC patients. A 4-texture feature ‘Radiomic Prognostic Vector (RPV)’ was developed using machine learning methods on the training set. Results The RPV was able to identify the 5% of patients with the worst prognosis, significantly improving established prognostic methods and was further validated in two independent, multi-centre cohorts. In addition, the genetic, transcriptomic and proteomic analysis from two independent datasets demonstrated that stromal and DNA damage response pathways are activated in RPV-stratified tumours. Conclusion RPV could be used to guide personalised therapy of EOC. Overall, the two large datasets of different imaging modalities have increased our knowledge of texture analysis, improving the models currently available and provided us with more areas with which to implement these tools in the clinical setting.Open Acces

A histopathological image classification method for cholangiocarcinoma based on spatial-channel feature fusion convolution neural network

Histopathological image analysis plays an important role in the diagnosis and treatment of cholangiocarcinoma. This time-consuming and complex process is currently performed manually by pathologists. To reduce the burden on pathologists, this paper proposes a histopathological image classification method for cholangiocarcinoma based on spatial-channel feature fusion convolutional neural networks. Specifically, the proposed model consists of a spatial branch and a channel branch. In the spatial branch, residual structural blocks are used to extract deep spatial features. In the channel branch, a multi-scale feature extraction module and some multi-level feature extraction modules are designed to extract channel features in order to increase the representational ability of the model. The experimental results of the Multidimensional Choledoch Database show that the proposed method performs better than other classical CNN classification methods

Deep learning-based diagnostic system for malignant liver detection

Cancer is the second most common cause of death of human beings, whereas liver cancer is the fifth most common cause of mortality. The prevention of deadly diseases in living beings requires timely, independent, accurate, and robust detection of ailment by a computer-aided diagnostic (CAD) system. Executing such intelligent CAD requires some preliminary steps, including preprocessing, attribute analysis, and identification. In recent studies, conventional techniques have been used to develop computer-aided diagnosis algorithms. However, such traditional methods could immensely affect the structural properties of processed images with inconsistent performance due to variable shape and size of region-of-interest. Moreover, the unavailability of sufficient datasets makes the performance of the proposed methods doubtful for commercial use. To address these limitations, I propose novel methodologies in this dissertation. First, I modified a generative adversarial network to perform deblurring and contrast adjustment on computed tomography (CT) scans. Second, I designed a deep neural network with a novel loss function for fully automatic precise segmentation of liver and lesions from CT scans. Third, I developed a multi-modal deep neural network to integrate pathological data with imaging data to perform computer-aided diagnosis for malignant liver detection. The dissertation starts with background information that discusses the proposed study objectives and the workflow. Afterward, Chapter 2 reviews a general schematic for developing a computer-aided algorithm, including image acquisition techniques, preprocessing steps, feature extraction approaches, and machine learning-based prediction methods. The first study proposed in Chapter 3 discusses blurred images and their possible effects on classification. A novel multi-scale GAN network with residual image learning is proposed to deblur images. The second method in Chapter 4 addresses the issue of low-contrast CT scan images. A multi-level GAN is utilized to enhance images with well-contrast regions. Thus, the enhanced images improve the cancer diagnosis performance. Chapter 5 proposes a deep neural network for the segmentation of liver and lesions from abdominal CT scan images. A modified Unet with a novel loss function can precisely segment minute lesions. Similarly, Chapter 6 introduces a multi-modal approach for liver cancer variants diagnosis. The pathological data are integrated with CT scan images to diagnose liver cancer variants. In summary, this dissertation presents novel algorithms for preprocessing and disease detection. Furthermore, the comparative analysis validates the effectiveness of proposed methods in computer-aided diagnosis

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.

Fast 4D Ultrasound Registration for Image Guided Liver Interventions

Liver problems are a serious health issue. The common liver problems are hepatitis, fatty liver, liver cancer and liver damage caused by alcohol abuse. Continuous, long term disease may cause a condition of the liver known as the Liver Cirrhosis. Liver cirrhosis makes the liver scarred and hardened up causing portal hypertension. In such a situation the collateral vessels try to bypass the liver as blood cannot freely flow through the liver; causing internal bleeding. One of the treatments of portal hypertension is Transjugular intrahepatic portosystemic shunt (TIPS). In a TIPS procedure a tract in the liver is created that shortcuts two veins in the liver, reducing the portal hypertension. Radiofrequency ablation (RFA) is use for the treatment of liver cancer. In RFA, a needle electrode is placed through the skin into the liver tumor. High-frequency electrical currents are passed through the electrode, creating heat that destroys the cancer cells, without damaging the surrounding liver tissues. TIPS and RFA are minimally invasive procedures, where small incisions are made to perform the surgery and are alternative to open surgery. A minimally invasive alternative has large potential in reducing complication rates, minimizing surgical trauma and reducing hospital stay. However, in these procedures, due to lack of direct eyesight, three-dimensional imaging information about the anatomy and instruments during the intervention is required. The most difficult part of these procedures is the interpretation and selection of oblique views for needle/instrument insertion and target visualization. In our work we develop and evaluate techniques that enable the effective use of 3D ultrasound for image guided interventions. Ultrasound is low cost, mobile and unlike CT and X-rays does not use any harmful radiation in the imaging process. During these procedures, breathing shifts the region of interest and makes it difficult to constantly focus on a region of interest. We provide an approach to correct for the motion due to breathing. Additionally, we propose a method for image fusion of interventional ultrasound and preoperative imaging modalities such as CT for cases where the lesions are visible in CT but not visible in ultrasound. Incorporating CT data during intervention additionally adds greater definition and precision to the ultrasound based navigation system. Concluding, in this thesis, we presented methods and evaluated their accuracies that demonstrate the use of real-time 3D US and its fusion with CT in potentially improving image guidance in minimally invasive US guided liver interventions