Deep learning for clinical decision support in oncology

In den letzten Jahrzehnten sind medizinische Bildgebungsverfahren wie die Computertomographie (CT) zu einem unersetzbaren Werkzeug moderner Medizin geworden, welche eine zeitnahe, nicht-invasive Begutachtung von Organen und Geweben ermöglichen. Die Menge an anfallenden Daten ist dabei rapide gestiegen, allein innerhalb der letzten Jahre um den Faktor 15, und aktuell verantwortlich für 30 % des weltweiten Datenvolumens. Die Anzahl ausgebildeter Radiologen ist weitestgehend stabil, wodurch die medizinische Bildanalyse, angesiedelt zwischen Medizin und Ingenieurwissenschaften, zu einem schnell wachsenden Feld geworden ist. Eine erfolgreiche Anwendung verspricht Zeitersparnisse, und kann zu einer höheren diagnostischen Qualität beitragen. Viele Arbeiten fokussieren sich auf „Radiomics“, die Extraktion und Analyse von manuell konstruierten Features. Diese sind jedoch anfällig gegenüber externen Faktoren wie dem Bildgebungsprotokoll, woraus Implikationen für Reproduzierbarkeit und klinische Anwendbarkeit resultieren. In jüngster Zeit sind Methoden des „Deep Learning“ zu einer häufig verwendeten Lösung algorithmischer Problemstellungen geworden. Durch Anwendungen in Bereichen wie Robotik, Physik, Mathematik und Wirtschaft, wurde die Forschung im Bereich maschinellen Lernens wesentlich verändert. Ein Kriterium für den Erfolg stellt die Verfügbarkeit großer Datenmengen dar. Diese sind im medizinischen Bereich rar, da die Bilddaten strengen Anforderungen bezüglich Datenschutz und Datensicherheit unterliegen, und oft heterogene Qualität, sowie ungleichmäßige oder fehlerhafte Annotationen aufweisen, wodurch ein bedeutender Teil der Methoden keine Anwendung finden kann. Angesiedelt im Bereich onkologischer Bildgebung zeigt diese Arbeit Wege zur erfolgreichen Nutzung von Deep Learning für medizinische Bilddaten auf. Mittels neuer Methoden für klinisch relevante Anwendungen wie die Schätzung von Läsionswachtum, Überleben, und Entscheidungkonfidenz, sowie Meta-Learning, Klassifikator-Ensembling, und Entscheidungsvisualisierung, werden Wege zur Verbesserungen gegenüber State-of-the-Art-Algorithmen aufgezeigt, welche ein breites Anwendungsfeld haben. Hierdurch leistet die Arbeit einen wesentlichen Beitrag in Richtung einer klinischen Anwendung von Deep Learning, zielt auf eine verbesserte Diagnose, und damit letztlich eine verbesserte Gesundheitsversorgung insgesamt.Over the last decades, medical imaging methods, such as computed tomography (CT), have become an indispensable tool of modern medicine, allowing for a fast, non-invasive inspection of organs and tissue. Thus, the amount of acquired healthcare data has rapidly grown, increased 15-fold within the last years, and accounts for more than 30 % of the world's generated data volume. In contrast, the number of trained radiologists remains largely stable. Thus, medical image analysis, settled between medicine and engineering, has become a rapidly growing research field. Its successful application may result in remarkable time savings and lead to a significantly improved diagnostic performance. Many of the work within medical image analysis focuses on radiomics, i. e. the extraction and analysis of hand-crafted imaging features. Radiomics, however, has been shown to be highly sensitive to external factors, such as the acquisition protocol, having major implications for reproducibility and clinical applicability. Lately, deep learning has become one of the most employed methods for solving computational problems. With successful applications in diverse fields, such as robotics, physics, mathematics, and economy, deep learning has revolutionized the process of machine learning research. Having large amounts of training data is a key criterion for its successful application. These data, however, are rare within medicine, as medical imaging is subject to a variety of data security and data privacy regulations. Moreover, medical imaging data often suffer from heterogeneous quality, label imbalance, and label noise, rendering a considerable fraction of deep learning-based algorithms inapplicable. Settled in the field of CT oncology, this work addresses these issues, showing up ways to successfully handle medical imaging data using deep learning. It proposes novel methods for clinically relevant tasks, such as lesion growth and patient survival prediction, confidence estimation, meta-learning and classifier ensembling, and finally deep decision explanation, yielding superior performance in comparison to state-of-the-art approaches, and being applicable to a wide variety of applications. With this, the work contributes towards a clinical translation of deep learning-based algorithms, aiming for an improved diagnosis, and ultimately overall improved patient healthcare

Artificial Intelligence Predictive Models of Response to Cytotoxic Chemotherapy Alone or Combined to Targeted Therapy for Metastatic Colorectal Cancer Patients: A Systematic Review and Meta-Analysis

Simple Summary Metastatic colorectal cancer (mCRC) has high incidence and mortality. Nevertheless, innovative biomarkers have been developed for predicting the response to therapy. We have examined the ability of learning methods to build prognostic and predictive models to predict response to chemotherapy, alone or combined with targeted therapy in mCRC patients, by targeting specific narrative publications. After a literature search, 26 original articles met inclusion and exclusion criteria and were included in the study. We showed that all investigations conducted in this field provided generally promising results in predicting the response to therapy or toxic side-effects, using a meta-analytic approach. We found that radiomics and molecular biomarker signatures were able to discriminate response vs. non-response by correctly identifying up to 99% of mCRC patients who were responders and up to 100% of patients who were non-responders. Our study supports the use of computer science for developing personalized treatment decision processes for mCRC patients. Tailored treatments for metastatic colorectal cancer (mCRC) have not yet completely evolved due to the variety in response to drugs. Therefore, artificial intelligence has been recently used to develop prognostic and predictive models of treatment response (either activity/efficacy or toxicity) to aid in clinical decision making. In this systematic review, we have examined the ability of learning methods to predict response to chemotherapy alone or combined with targeted therapy in mCRC patients by targeting specific narrative publications in Medline up to April 2022 to identify appropriate original scientific articles. After the literature search, 26 original articles met inclusion and exclusion criteria and were included in the study. Our results show that all investigations conducted on this field have provided generally promising results in predicting the response to therapy or toxic side-effects. By a meta-analytic approach we found that the overall weighted means of the area under the receiver operating characteristic (ROC) curve (AUC) were 0.90, 95% C.I. 0.80-0.95 and 0.83, 95% C.I. 0.74-0.89 in training and validation sets, respectively, indicating a good classification performance in discriminating response vs. non-response. The calculation of overall HR indicates that learning models have strong ability to predict improved survival. Lastly, the delta-radiomics and the 74 gene signatures were able to discriminate response vs. non-response by correctly identifying up to 99% of mCRC patients who were responders and up to 100% of patients who were non-responders. Specifically, when we evaluated the predictive models with tests reaching 80% sensitivity (SE) and 90% specificity (SP), the delta radiomics showed an SE of 99% and an SP of 94% in the training set and an SE of 85% and SP of 92 in the test set, whereas for the 74 gene signatures the SE was 97.6% and the SP 100% in the training set

The Era of Radiogenomics in Precision Medicine: An Emerging Approach to Support Diagnosis, Treatment Decisions, and Prognostication in Oncology

With the rapid development of new technologies, including artificial intelligence and genome sequencing, radiogenomics has emerged as a state-of-the-art science in the field of individualized medicine. Radiogenomics combines a large volume of quantitative data extracted from medical images with individual genomic phenotypes and constructs a prediction model through deep learning to stratify patients, guide therapeutic strategies, and evaluate clinical outcomes. Recent studies of various types of tumors demonstrate the predictive value of radiogenomics. And some of the issues in the radiogenomic analysis and the solutions from prior works are presented. Although the workflow criteria and international agreed guidelines for statistical methods need to be confirmed, radiogenomics represents a repeatable and cost-effective approach for the detection of continuous changes and is a promising surrogate for invasive interventions. Therefore, radiogenomics could facilitate computer-aided diagnosis, treatment, and prediction of the prognosis in patients with tumors in the routine clinical setting. Here, we summarize the integrated process of radiogenomics and introduce the crucial strategies and statistical algorithms involved in current studies

Standardization of body composition status in patients with advanced urothelial tumors: the role of a CT-based aI-powered software for the assessment of sarcopenia and patient outcome correlation

Background: Sarcopenia is a well know prognostic factor in oncology, influencing patients' quality of life and survival. We aimed to investigate the role of sarcopenia, assessed by a Computed Tomography (CT)-based artificial intelligence (AI)-powered-software, as a predictor of objective clinical benefit in advanced urothelial tumors and its correlations with oncological outcomes. Methods: We retrospectively searched patients with advanced urothelial tumors, treated with systemic platinum-based chemotherapy and an available total body CT, performed before and after therapy. An AI-powered software was applied to CT to obtain the Skeletal Muscle Index (SMI-L3), derived from the area of the psoas, long spine, and abdominal muscles, at the level of L3 on CT axial images. Logistic and Cox-regression modeling was implemented to explore the association of sarcopenic status and anthropometric features to the clinical benefit rate and survival endpoints. Results: 97 patients were included, 66 with bladder cancer and 31 with upper-tract urothelial carcinoma. Clinical benefit outcomes showed a linear positive association with all the observed body composition variables variations. The chances of not experiencing disease progression were positively associated with ∆_SMI-L3, ∆_psoas, and ∆_long spine muscle when they ranged from ~10-20% up to ~45-55%. Greater survival chances were matched by patients achieving a wider ∆_SMI-L3, ∆_abdominal and ∆_long spine muscle. Conclusions: A CT-based AI-powered software body composition and sarcopenia analysis provide prognostic assessments for objective clinical benefits and oncological outcomes

Radiomics using computed tomography to predict CD73 expression and prognosis of colorectal cancer liver metastases

ABSTRACT: Background Finding a noninvasive radiomic surrogate of tumor immune features could help identify patients more likely to respond to novel immune checkpoint inhibitors. Particularly, CD73 is an ectonucleotidase that cata- lyzes the breakdown of extracellular AMP into immunosuppressive adenosine, which can be blocked by therapeutic antibodies. High CD73 expression in colorectal cancer liver metastasis (CRLM) resected with curative intent is associ- ated with early recurrence and shorter patient survival. The aim of this study was hence to evaluate whether machine learning analysis of preoperative liver CT-scan could estimate high vs low CD73 expression in CRLM and whether such radiomic score would have a prognostic significance. Methods We trained an Attentive Interpretable Tabular Learning (TabNet) model to predict, from preoperative CT images, stratified expression levels of CD73 (CD73High vs. CD73Low ) assessed by immunofluorescence (IF) on tissue microarrays. Radiomic features were extracted from 160 segmented CRLM of 122 patients with matched IF data, preprocessed and used to train the predictive model. We applied a five-fold cross-validation and validated the perfor- mance on a hold-out test set. Results TabNet provided areas under the receiver operating characteristic curve of 0.95 (95% CI 0.87 to 1.0) and 0.79 (0.65 to 0.92) on the training and hold-out test sets respectively, and outperformed other machine learning models. The TabNet-derived score, termed rad-CD73, was positively correlated with CD73 histological expression in matched CRLM (Spearman’s ρ = 0.6004; P < 0.0001). The median time to recurrence (TTR) and disease-specific survival (DSS) after CRLM resection in rad-CD73High vs rad-CD73 Low patients was 13.0 vs 23.6 months (P = 0.0098) and 53.4 vs 126.0 months (P = 0.0222), respectively. The prognostic value of rad-CD73 was independent of the standard clinical risk score, for both TTR (HR = 2.11, 95% CI 1.30 to 3.45, P < 0.005) and DSS (HR = 1.88, 95% CI 1.11 to 3.18, P = 0.020)

Changes in Body Composition During Adjuvant FOLFOX Chemotherapy and Overall Survival in Non-Metastatic Colon Cancer

The impact of longitudinal anthropometric changes during adjuvant chemotherapy on long-term survival in non-metastatic colon cancer is unclear. Herein, we analyzed the prognostic significance of computed tomography (CT)-measured body composition changes in colon cancer patients who underwent surgery followed by adjuvant FOLFOX (folinic acid, 5-fluorouracil, oxaliplatin) chemotherapy. Data of 167 patients with stage III or high-risk stage II colon cancer were analyzed. Skeletal muscle index (SMI), skeletal muscle radiodensity (SMR), visceral fat index (VFI), subcutaneous fat index (SFI), and total fat index (TFI) changes during chemotherapy were calculated using preoperative and postchemotherapy CT image data. The Cox proportional hazard model was used to determine the correlation between changes in anthropometric values and overall survival (OS). The median changes (%) in SMI, SMR, VFI, SFI, and TFI over 210 days during chemotherapy were 8.7% (p < 0.001), 3.4% (p = 0.001), -19% (p < 0.001), -3.4% (p = 0.936), and -11.9% (p < 0.001), respectively. Cut-off values of changes in SMI (skeletal muscle index change, SMIC) and SMR (skeletal muscle radiodensity change, SMRC) were defined at -2% and -2 Hounsfield units (HU) respectively, whereas those of changes in VFI (visceral fat index change, VFIC), SFI (subcutaneous fat index change, SFIC), and TFI (total fat index change, TFIC) were based on values that provided the largest χ2 on the Mantel-Cox test. Multivariable analysis revealed that low SMR measured on a postchemotherapy CT scan (hazard ratio, HR: 0.32, 95% confidence interval, CI: 0.15-0.70, p = 0.004) and visceral fat loss of at least 46.57% (HR: 0.31, 95% CI: 0.14-0.69, p = 0.004) were independent poor prognostic factors for OS. Severe visceral fat loss during FOLFOX chemotherapy and low skeletal muscle radiodensity measured on postchemotherapy CT scans are associated with poor OS in stage III and high-risk stage II colon cancer patients.ope

Artificial intelligence for breast cancer precision pathology

Breast cancer is the most common cancer type in women globally but is associated with a continuous decline in mortality rates. The improved prognosis can be partially attributed to effective treatments developed for subgroups of patients. However, nowadays, it remains challenging to optimise treatment plans for each individual. To improve disease outcome and to decrease the burden associated with unnecessary treatment and adverse drug effects, the current thesis aimed to develop artificial intelligence based tools to improve individualised medicine for breast cancer patients. In study I, we developed a deep learning based model (DeepGrade) to stratify patients that were associated with intermediate risks. The model was optimised with haematoxylin and eosin (HE) stained whole slide images (WSIs) with grade 1 and 3 tumours and applied to stratify grade 2 tumours into grade 1-like (DG2-low) and grade 3-like (DG2-high) subgroups. The efficacy of the DeepGrade model was validated using recurrence free survival where the dichotomised groups exhibited an adjusted hazard ratio (HR) of 2.94 (95% confidence interval [CI] 1.24-6.97, P = 0.015). The observation was further confirmed in the external test cohort with an adjusted HR of 1.91 (95% CI: 1.11-3.29, P = 0.019). In study II, we investigated whether deep learning models were capable of predicting gene expression levels using the morphological patterns from tumours. We optimised convolutional neural networks (CNNs) to predict mRNA expression for 17,695 genes using HE stained WSIs from the training set. An initial evaluation on the validation set showed that a significant correlation between the RNA-seq measurements and model predictions was observed for 52.75% of the genes. The models were further tested in the internal and external test sets. Besides, we compared the model's efficacy in predicting RNA-seq based proliferation scores. Lastly, the ability of capturing spatial gene expression variations for the optimised CNNs was evaluated and confirmed using spatial transcriptomics profiling. In study III, we investigated the relationship between intra-tumour gene expression heterogeneity and patient survival outcomes. Deep learning models optimised from study II were applied to generate spatial gene expression predictions for the PAM50 gene panel. A set of 11 texture based features and one slide average gene expression feature per gene were extracted as input to train a Cox proportional hazards regression model with elastic net regularisation to predict patient risk of recurrence. Through nested cross-validation, the model dichotomised the training cohort into low and high risk groups with an adjusted HR of 2.1 (95% CI: 1.30-3.30, P = 0.002). The model was further validated on two external cohorts. In study IV, we investigated the agreement between the Stratipath Breast, which is the modified, commercialised DeepGrade model developed in study I, and the Prosigna® test. Both tests sought to stratify patients with distinct prognosis. The outputs from Stratipath Breast comprise a risk score and a two-level risk stratification whereas the outputs from Prosigna® include the risk of recurrence score and a three-tier risk stratification. By comparing the number of patients assigned to ‘low’ or ‘high’ risk groups, we found an overall moderate agreement (76.09%) between the two tests. Besides, the risk scores by two tests also revealed a good correlation (Spearman's rho = 0.59, P = 1.16E-08). In addition, a good correlation was observed between the risk score from each test and the Ki67 index. The comparison was also carried out in the subgroup of patients with grade 2 tumours where similar but slightly dropped correlations were found

Adipose Tissue Distribution and Survival Among Women with Nonmetastatic Breast Cancer.

ObjectivePrevious studies of breast cancer survival have not considered specific depots of adipose tissue such as subcutaneous adipose tissue (SAT) and visceral adipose tissue (VAT).MethodsThis study assessed these relationships among 3,235 women with stage II and III breast cancer diagnosed between 2005 and 2013 at Kaiser Permanente Northern California and between 2000 and 2012 at Dana Farber Cancer Institute. SAT and VAT areas (in centimeters squared) were calculated from routine computed tomography scans within 6 (median: 1.2) months of diagnosis, covariates were collected from electronic health records, and vital status was assessed by death records. Hazard ratios (HRs) and 95% CIs were estimated using Cox regression.ResultsSAT and VAT ranged from 19.0 to 891&nbsp;cm2 and from 0.484 to 454&nbsp;cm2 , respectively. SAT was related to increased risk of death (127-cm2 increase; HR [95% CI]: 1.13 [1.02-1.26]), but no relationship was found with VAT (78.18-cm2 increase; HR [95% CI]: 1.02 [0.91-1.14]). An association with VAT was noted among women with stage II cancer (stage II: HR: 1.17 [95% CI: 0.99-1.39]; stage III: HR: 0.90 [95% CI: 0.76-1.07]; P interaction &lt; 0.01). Joint increases in SAT and VAT were associated with mortality above either alone (simultaneous 1-SD increase: HR 1.19 [95% CI: 1.05-1.34]).ConclusionsSAT may be an underappreciated risk factor for breast cancer-related death

Comparison of radiomic feature aggregation methods for patients with multiple tumors.

Radiomic feature analysis has been shown to be effective at analyzing diagnostic images to model cancer outcomes. It has not yet been established how to best combine radiomic features in cancer patients with multifocal tumors. As the number of patients with multifocal metastatic cancer continues to rise, there is a need for improving personalized patient-level prognosis to better inform treatment. We compared six mathematical methods of combining radiomic features of 3,596 tumors in 831 patients with multiple brain metastases and evaluated the performance of these aggregation methods using three survival models: a standard Cox proportional hazards model, a Cox proportional hazards model with LASSO regression, and a random survival forest. Across all three survival models, the weighted average of the largest three metastases had the highest concordance index (95% confidence interval) of 0.627 (0.595-0.661) for the Cox proportional hazards model, 0.628 (0.591-0.666) for the Cox proportional hazards model with LASSO regression, and 0.652 (0.565-0.727) for the random survival forest model. This finding was consistent when evaluating patients with different numbers of brain metastases and different tumor volumes. Radiomic features can be effectively combined to estimate patient-level outcomes in patients with multifocal brain metastases. Future studies are needed to confirm that the volume-weighted average of the largest three tumors is an effective method for combining radiomic features across other imaging modalities and tumor types