Data-efficient Cross-domain Medical Image Segmentation

Deep learning-based segmentation methods have been widely employed for medical image analysis, especially for automatic disease diagnosis and prognosis. Nevertheless, existing deep-learning models benefit from large amounts of annotated data, bringing auxiliary data acquisition and annotation costs. In practice, privacy, security, and storage concerns often impede the availability of medical images for model training. On the other side, most of the deep learning models suffer from performance drops when validated on unseen datasets with distribution shifts. Unsupervised domain adaptation (UDA) has been developed to address this issue by transferring the knowledge from the labeled source data to the unlabeled target data. To further facilitate the data efficiency of the cross-domain segmentation methods, we explore UDA medical image segmentation problems using a few labeled source data and under a multi-source data-free situation in this work. For UDA image segmentation with few labeled source data, we first create a searching-based multi-style invariant mechanism to expand the data distribution with style diversity. A prototype consistency mechanism for the foreground objects is then developed to enable the alignment of the features for each kind of tissue in various image styles. The segmentation performance on the target photos is further improved by a cross-style self-supervised learning strategy. For a multi-source data-free UDA problem, a single student multiple teach network is initially established to distill knowledge from several pre-trained source models. The pre-trained model is then sorted to remove domain biases from the various source domains using a weighted transfer learning module. A Cross-domain averaging module also preserves overall consistency by accounting for model parameters. Our methods have outperformed several state-of-the-art UDA segmentation methods on both retinal fundus segmentation and MRI prostate segmentation tasks

An evaluation of a checklist in Musculoskeletal (MSK) radiographic image interpretation when using Artificial Intelligence (AI)

Background: AI is being used increasingly in image interpretation tasks. There are challenges for its optimal use in reporting environments. Human reliance on technology and bias can cause decision errors. Trust issues exist amongst radiologists and radiographers in both over-reliance (automation bias) and reluctance in AI use for decision support. A checklist, used with the AI to mitigate against such biases, may optimise the use of AI technologies and promote good decision hygiene. Method: A checklist, to be used in image interpretation with AI assistance, was developed. Participants interpreted 20 examinations with AI assistance and then re- interpreted the 20 examinations with AI and a checklist. The MSK images were presented to radiographers as patient examinations to replicate the image interpretation task in clinical practice. Image diagnosis and confidence levels on the diagnosis provided were collected following each interpretation. The participant perception of the use of the checklist was investigated via a questionnaire.Results: Data collection and analysis are underway and will be completed at the European Congress of Radiology in Vienna, March 2023. The impact of the use of a checklist in image interpretation with AI will be evaluated. Changes in accuracy and confidence will be investigated and results will be presented. Participant feedback will be analysed to determine perceptions and impact of the checklist also. Conclusion: A novel checklist has been developed to aid the interpretation of images when using AI. The checklist has been tested for its use in assisting radiographers in MSK image interpretation when using AI.<br/

Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

This two-volume set LNCS 12962 and 12963 constitutes the thoroughly refereed proceedings of the 7th International MICCAI Brainlesion Workshop, BrainLes 2021, as well as the RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge, the Federated Tumor Segmentation (FeTS) Challenge, the Cross-Modality Domain Adaptation (CrossMoDA) Challenge, and the challenge on Quantification of Uncertainties in Biomedical Image Quantification (QUBIQ). These were held jointly at the 23rd Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2020, in September 2021. The 91 revised papers presented in these volumes were selected form 151 submissions. Due to COVID-19 pandemic the conference was held virtually. This is an open access book

The impact of arterial input function determination variations on prostate dynamic contrast-enhanced magnetic resonance imaging pharmacokinetic modeling: a multicenter data analysis challenge, part II

This multicenter study evaluated the effect of variations in arterial input function (AIF) determination on pharmacokinetic (PK) analysis of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data using the shutter-speed model (SSM). Data acquired from eleven prostate cancer patients were shared among nine centers. Each center used a site-specific method to measure the individual AIF from each data set and submitted the results to the managing center. These AIFs, their reference tissue-adjusted variants, and a literature population-averaged AIF, were used by the managing center to perform SSM PK analysis to estimate Ktrans (volume transfer rate constant), ve (extravascular, extracellular volume fraction), kep (efflux rate constant), and τi (mean intracellular water lifetime). All other variables, including the definition of the tumor region of interest and precontrast T1 values, were kept the same to evaluate parameter variations caused by variations in only the AIF. Considerable PK parameter variations were observed with within-subject coefficient of variation (wCV) values of 0.58, 0.27, 0.42, and 0.24 for Ktrans, ve, kep, and τi, respectively, using the unadjusted AIFs. Use of the reference tissue-adjusted AIFs reduced variations in Ktrans and ve (wCV = 0.50 and 0.10, respectively), but had smaller effects on kep and τi (wCV = 0.39 and 0.22, respectively). kep is less sensitive to AIF variation than Ktrans, suggesting it may be a more robust imaging biomarker of prostate microvasculature. With low sensitivity to AIF uncertainty, the SSM-unique τi parameter may have advantages over the conventional PK parameters in a longitudinal study

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in the deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. In this paper, we provide an extensive survey of deep learning-based breast cancer imaging research, covering studies on mammogram, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods, publicly available datasets, and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are described in detail. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.Comment: Survey, 41 page

Complexity Reduction in Image-Based Breast Cancer Care

The diversity of malignancies of the breast requires personalized diagnostic and therapeutic decision making in a complex situation. This thesis contributes in three clinical areas: (1) For clinical diagnostic image evaluation, computer-aided detection and diagnosis of mass and non-mass lesions in breast MRI is developed. 4D texture features characterize mass lesions. For non-mass lesions, a combined detection/characterisation method utilizes the bilateral symmetry of the breast s contrast agent uptake. (2) To improve clinical workflows, a breast MRI reading paradigm is proposed, exemplified by a breast MRI reading workstation prototype. Instead of mouse and keyboard, it is operated using multi-touch gestures. The concept is extended to mammography screening, introducing efficient navigation aids. (3) Contributions to finite element modeling of breast tissue deformations tackle two clinical problems: surgery planning and the prediction of the breast deformation in a MRI biopsy device

Quantifying, Understanding and Predicting Differences Between Planned and Delivered Dose to Organs at Risk in Head & Neck Cancer Patients Undergoing Radical Radiotherapy to Promote Intelligently Targeted Adaptive Radiotherapy

Introduction: Radical radiotherapy (RT) is an effective but toxic treatment for head and neck cancer (HNC). Contemporary radiotherapy techniques sculpt dose to target disease and avoid organs at risk (OARs), but anatomical change during treatment mean that the radiation dose delivered to the patient – delivered dose (DA), is different to that anticipated at planning – planned dose (DP). Modifying the RT plan during treatment – Adaptive Radiotherapy (ART) – could mitigate these risks by reducing dose to OARs. However, clinical data to guide patient selection for, and timing of ART, are for lacking. Methods: 337 patients with HNC were recruited to the Cancer Research UK VoxTox study. Demographic, disease and treatment data were collated, and both DP and DA to organs at risk (OARs) were computed from daily megavoltage CT image guidance scans, using an open-source deformable image registration package (Elastix). Toxicity data were prospectively collected. Relationships between DP, DA and late toxicities were investigated with univariate, and logistic regression normal tissue complication probability (NTCP) modelling approaches. A sub-study of VoxTox recruited 18 patients who had MRI scans before RT fractions 1, 6, 16, and 26. Changes in salivary gland volumes and relative apparent diffusion coefficient (ADC) values were measured and related to toxicity events. Results: Spinal cord dose differences were small, and not predicted by weight loss or shape change. Mean DA to all other OARs was higher than DP; factors predicting higher DA included primary disease site, concomitant therapy, shape change and advanced neck disease. Nine patients (3.7%) saw DA>DP by 2Gy to more than half of the OARs assessed. These patients all had received bilateral neck RT for N-stage 2b oropharyngeal cancer. Strong uni- and multivariate relationships between OAR dose and toxicity were seen. Differences between DA and DP-based dose-toxicity models were minimal, and not statistically significant. On MRI, both parotid and submandibular glands shrank during treatment, whilst relative ADC rose. Relationships with toxicity were inconclusive. Conclusions: Small differences between OAR DP and DA mean that DA-based toxicity prediction models confer negligible additional benefit at the population level. Factors such as primary disease sub-site, concomitant systemic therapy, staging and shape change may help to select the patients that do develop clinically significant dose differences, and would benefit most from ART for toxicity reduction

Multimodal Data Fusion and Quantitative Analysis for Medical Applications

Medical big data is not only enormous in its size, but also heterogeneous and complex in its data structure, which makes conventional systems or algorithms difficult to process. These heterogeneous medical data include imaging data (e.g., Positron Emission Tomography (PET), Computerized Tomography (CT), Magnetic Resonance Imaging (MRI)), and non-imaging data (e.g., laboratory biomarkers, electronic medical records, and hand-written doctor notes). Multimodal data fusion is an emerging vital field to address this urgent challenge, aiming to process and analyze the complex, diverse and heterogeneous multimodal data. The fusion algorithms bring great potential in medical data analysis, by 1) taking advantage of complementary information from different sources (such as functional-structural complementarity of PET/CT images) and 2) exploiting consensus information that reflects the intrinsic essence (such as the genetic essence underlying medical imaging and clinical symptoms). Thus, multimodal data fusion benefits a wide range of quantitative medical applications, including personalized patient care, more optimal medical operation plan, and preventive public health. Though there has been extensive research on computational approaches for multimodal fusion, there are three major challenges of multimodal data fusion in quantitative medical applications, which are summarized as feature-level fusion, information-level fusion and knowledge-level fusion: • Feature-level fusion. The first challenge is to mine multimodal biomarkers from high-dimensional small-sample multimodal medical datasets, which hinders the effective discovery of informative multimodal biomarkers. Specifically, efficient dimension reduction algorithms are required to alleviate "curse of dimensionality" problem and address the criteria for discovering interpretable, relevant, non-redundant and generalizable multimodal biomarkers. • Information-level fusion. The second challenge is to exploit and interpret inter-modal and intra-modal information for precise clinical decisions. Although radiomics and multi-branch deep learning have been used for implicit information fusion guided with supervision of the labels, there is a lack of methods to explicitly explore inter-modal relationships in medical applications. Unsupervised multimodal learning is able to mine inter-modal relationship as well as reduce the usage of labor-intensive data and explore potential undiscovered biomarkers; however, mining discriminative information without label supervision is an upcoming challenge. Furthermore, the interpretation of complex non-linear cross-modal associations, especially in deep multimodal learning, is another critical challenge in information-level fusion, which hinders the exploration of multimodal interaction in disease mechanism. • Knowledge-level fusion. The third challenge is quantitative knowledge distillation from multi-focus regions on medical imaging. Although characterizing imaging features from single lesions using either feature engineering or deep learning methods have been investigated in recent years, both methods neglect the importance of inter-region spatial relationships. Thus, a topological profiling tool for multi-focus regions is in high demand, which is yet missing in current feature engineering and deep learning methods. Furthermore, incorporating domain knowledge with distilled knowledge from multi-focus regions is another challenge in knowledge-level fusion. To address the three challenges in multimodal data fusion, this thesis provides a multi-level fusion framework for multimodal biomarker mining, multimodal deep learning, and knowledge distillation from multi-focus regions. Specifically, our major contributions in this thesis include: • To address the challenges in feature-level fusion, we propose an Integrative Multimodal Biomarker Mining framework to select interpretable, relevant, non-redundant and generalizable multimodal biomarkers from high-dimensional small-sample imaging and non-imaging data for diagnostic and prognostic applications. The feature selection criteria including representativeness, robustness, discriminability, and non-redundancy are exploited by consensus clustering, Wilcoxon filter, sequential forward selection, and correlation analysis, respectively. SHapley Additive exPlanations (SHAP) method and nomogram are employed to further enhance feature interpretability in machine learning models. • To address the challenges in information-level fusion, we propose an Interpretable Deep Correlational Fusion framework, based on canonical correlation analysis (CCA) for 1) cohesive multimodal fusion of medical imaging and non-imaging data, and 2) interpretation of complex non-linear cross-modal associations. Specifically, two novel loss functions are proposed to optimize the discovery of informative multimodal representations in both supervised and unsupervised deep learning, by jointly learning inter-modal consensus and intra-modal discriminative information. An interpretation module is proposed to decipher the complex non-linear cross-modal association by leveraging interpretation methods in both deep learning and multimodal consensus learning. • To address the challenges in knowledge-level fusion, we proposed a Dynamic Topological Analysis framework, based on persistent homology, for knowledge distillation from inter-connected multi-focus regions in medical imaging and incorporation of domain knowledge. Different from conventional feature engineering and deep learning, our DTA framework is able to explicitly quantify inter-region topological relationships, including global-level geometric structure and community-level clusters. K-simplex Community Graph is proposed to construct the dynamic community graph for representing community-level multi-scale graph structure. The constructed dynamic graph is subsequently tracked with a novel Decomposed Persistence algorithm. Domain knowledge is incorporated into the Adaptive Community Profile, summarizing the tracked multi-scale community topology with additional customizable clinically important factors

Frameworks in medical image analysis with deep neural networks

In recent years, deep neural network based medical image analysis has become quite powerful and achieved similar results performance-wise as experts. Consequently, the integration of these tools into the clinical routine as clinical decision support systems is highly desired. The benefits of automatic image analysis for clinicians are massive, ranging from improved diagnostic as well as treatment quality to increased time-efficiency through automated structured reporting. However, implementations in the literature revealed a significant lack of standardization in pipeline building resulting in low reproducibility, high complexity through extensive knowledge requirements for building state-of-the-art pipelines, and difficulties for application in clinical research. The main objective of this work is the standardization of pipeline building in deep neural network based medical image segmentation and classification. This is why the Python frameworks MIScnn for medical image segmentation and AUCMEDI for medical image classification are proposed which simplify the implementation process through intuitive building blocks eliminating the need for time-consuming and error-prone implementation of common components from scratch. The proposed frameworks include state-of-the-art methodology, follow outstanding open-source principles like extensive documentation as well as stability, offer rapid as well as simple application capabilities for deep learning experts as well as clinical researchers, and provide cutting-edge high-performance competitive with the strongest implementations in the literature. As secondary objectives, this work presents more than a dozen in-house studies as well as discusses various external studies utilizing the proposed frameworks in order to prove the capabilities of standardized medical image analysis. The presented studies demonstrate excellent predictive capabilities in applications ranging from COVID-19 detection in computed tomography scans to the integration into a clinical study workflow for Gleason grading of prostate cancer microscopy sections and advance the state-of-the-art in medical image analysis by simplifying experimentation setups for research. Furthermore, studies for increasing reproducibility in performance assessment of medical image segmentation are presented including an open-source metric library for standardized evaluation and a community guideline on proper metric usage. The proposed contributions in this work improve the knowledge representation of the field, enable rapid as well as high-performing applications, facilitate further research, and strengthen the reproducibility of future studies