Deeper Image Quality Transfer: Training Low-Memory Neural Networks for 3D Images

In this paper we address the memory demands that come with the processing of 3-dimensional, high-resolution, multi-channeled medical images in deep learning. We exploit memory-efficient backpropagation techniques, to reduce the memory complexity of network training from being linear in the network's depth, to being roughly constant $-$ permitting us to elongate deep architectures with negligible memory increase. We evaluate our methodology in the paradigm of Image Quality Transfer, whilst noting its potential application to various tasks that use deep learning. We study the impact of depth on accuracy and show that deeper models have more predictive power, which may exploit larger training sets. We obtain substantially better results than the previous state-of-the-art model with a slight memory increase, reducing the root-mean-squared-error by $13\%$. Our code is publicly available.Comment: Accepted in: MICCAI 201

Feasibility of Data-Driven, Model-Free Quantitative MRI Protocol Design: Application to Brain and Prostate Diffusion-Relaxation Imaging

Brain; Protocol design; Quantitative MRI (qMRI)Cerebro; Diseño de protocolo; Resonancia magnética cuantitativa (qMRI)Cervell; Disseny del protocol; Ressonància magnètica quantitativa (qMRI)Purpose: We investigate the feasibility of data-driven, model-free quantitative MRI (qMRI) protocol design on in vivo brain and prostate diffusion-relaxation imaging (DRI). Methods: We select subsets of measurements within lengthy pilot scans, without identifying tissue parameters for which to optimise for. We use the “select and retrieve via direct upsampling” (SARDU-Net) algorithm, made of a selector, identifying measurement subsets, and a predictor, estimating fully-sampled signals from the subsets. We implement both using artificial neural networks, which are trained jointly end-to-end. We deploy the algorithm on brain (32 diffusion-/T1-weightings) and prostate (16 diffusion-/T2-weightings) DRI scans acquired on three healthy volunteers on two separate 3T Philips systems each. We used SARDU-Net to identify sub-protocols of fixed size, assessing reproducibility and testing sub-protocols for their potential to inform multi-contrast analyses via the T1-weighted spherical mean diffusion tensor (T1-SMDT, brain) and hybrid multi-dimensional MRI (HM-MRI, prostate) models, for which sub-protocol selection was not optimised explicitly. Results: In both brain and prostate, SARDU-Net identifies sub-protocols that maximise information content in a reproducible manner across training instantiations using a small number of pilot scans. The sub-protocols support T1-SMDT and HM-MRI multi-contrast modelling for which they were not optimised explicitly, providing signal quality-of-fit in the top 5% against extensive sub-protocol comparisons. Conclusions: Identifying economical but informative qMRI protocols from subsets of rich pilot scans is feasible and potentially useful in acquisition-time-sensitive applications in which there is not a qMRI model of choice. SARDU-Net is demonstrated to be a robust algorithm for data-driven, model-free protocol design.This project was funded by the Engineering and Physical Sciences Research Council (EPSRC EP/R006032/1, M020533/1, G007748, I027084, N018702). This project has received funding under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 634541 and 666992, and from: Rosetrees Trust (United Kingdom, funding FG); Prostate Cancer United Kingdom Targeted Call 2014 (Translational Research St.2, project reference PG14-018-TR2); Cancer Research United Kingdom grant ref. A21099; Spinal Research (United Kingdom), Wings for Life (Austria), Craig H. Neilsen Foundation (United States) for jointly funding the INSPIRED study; Wings for Life (#169111); United Kingdom Multiple Sclerosis Society (grants 892/08 and 77/2017); the Department of Health’s National Institute for Health Research (NIHR) Biomedical Research Centres and UCLH NIHR Biomedical Research Centre; Champalimaud Centre for the Unknown, Lisbon (Portugal); European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 101003390. FG is currently supported by the investigator-initiated PREdICT study at the Vall d’Hebron Institute of Oncology (Barcelona), funded by AstraZeneca and CRIS Cancer Foundation

Progressive Subsampling for Oversampled Data -- Application to Quantitative MRI

We present PROSUB: PROgressive SUBsampling, a deep learning based, automated methodology that subsamples an oversampled data set (e.g. multi-channeled 3D images) with minimal loss of information. We build upon a recent dual-network approach that won the MICCAI MUlti-DIffusion (MUDI) quantitative MRI measurement sampling-reconstruction challenge, but suffers from deep learning training instability, by subsampling with a hard decision boundary. PROSUB uses the paradigm of recursive feature elimination (RFE) and progressively subsamples measurements during deep learning training, improving optimization stability. PROSUB also integrates a neural architecture search (NAS) paradigm, allowing the network architecture hyperparameters to respond to the subsampling process. We show PROSUB outperforms the winner of the MUDI MICCAI challenge, producing large improvements >18% MSE on the MUDI challenge sub-tasks and qualitative improvements on downstream processes useful for clinical applications. We also show the benefits of incorporating NAS and analyze the effect of PROSUB's components. As our method generalizes to other problems beyond MRI measurement selection-reconstruction, our code is https://github.com/sbb-gh/PROSU

Disease Knowledge Transfer across Neurodegenerative Diseases

We introduce Disease Knowledge Transfer (DKT), a novel technique for transferring biomarker information between related neurodegenerative diseases. DKT infers robust multimodal biomarker trajectories in rare neurodegenerative diseases even when only limited, unimodal data is available, by transferring information from larger multimodal datasets from common neurodegenerative diseases. DKT is a joint-disease generative model of biomarker progressions, which exploits biomarker relationships that are shared across diseases. Our proposed method allows, for the first time, the estimation of plausible, multimodal biomarker trajectories in Posterior Cortical Atrophy (PCA), a rare neurodegenerative disease where only unimodal MRI data is available. For this we train DKT on a combined dataset containing subjects with two distinct diseases and sizes of data available: 1) a larger, multimodal typical AD (tAD) dataset from the TADPOLE Challenge, and 2) a smaller unimodal Posterior Cortical Atrophy (PCA) dataset from the Dementia Research Centre (DRC), for which only a limited number of Magnetic Resonance Imaging (MRI) scans are available. Although validation is challenging due to lack of data in PCA, we validate DKT on synthetic data and two patient datasets (TADPOLE and PCA cohorts), showing it can estimate the ground truth parameters in the simulation and predict unseen biomarkers on the two patient datasets. While we demonstrated DKT on Alzheimer's variants, we note DKT is generalisable to other forms of related neurodegenerative diseases. Source code for DKT is available online: https://github.com/mrazvan22/dkt.Comment: accepted at MICCAI 2019, 13 pages, 5 figures, 2 table

Degenerative adversarial neuroimage nets for brain scan simulations: Application in ageing and dementia

© 2021 The Author(s). Published by Elsevier B.V. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY), https://creativecommons.org/licenses/by/4.0/Accurate and realistic simulation of high-dimensional medical images has become an important research area relevant to many AI-enabled healthcare applications. However, current state-of-the-art approaches lack the ability to produce satisfactory high-resolution and accurate subject-specific images. In this work, we present a deep learning framework, namely 4D-Degenerative Adversarial NeuroImage Net (4D-DANI-Net), to generate high-resolution, longitudinal MRI scans that mimic subject-specific neurodegeneration in ageing and dementia. 4D-DANI-Net is a modular framework based on adversarial training and a set of novel spatiotemporal, biologically-informed constraints. To ensure efficient training and overcome memory limitations affecting such high-dimensional problems, we rely on three key technological advances: i) a new 3D training consistency mechanism called Profile Weight Functions (PWFs), ii) a 3D super-resolution module and iii) a transfer learning strategy to fine-tune the system for a given individual. To evaluate our approach, we trained the framework on 9852 T1-weighted MRI scans from 876 participants in the Alzheimer's Disease Neuroimaging Initiative dataset and held out a separate test set of 1283 MRI scans from 170 participants for quantitative and qualitative assessment of the personalised time series of synthetic images. We performed three evaluations: i) image quality assessment; ii) quantifying the accuracy of regional brain volumes over and above benchmark models; and iii) quantifying visual perception of the synthetic images by medical experts. Overall, both quantitative and qualitative results show that 4D-DANI-Net produces realistic, low-artefact, personalised time series of synthetic T1 MRI that outperforms benchmark models.Peer reviewe

Learning Morphological Feature Perturbations for Calibrated Semi-Supervised Segmentation

We propose MisMatch, a novel consistency-driven semi-supervised segmentation framework which produces predictions that are invariant to learnt feature perturbations. MisMatch consists of an encoder and a two-head decoders. One decoder learns positive attention to the foreground regions of interest (RoI) on unlabelled images thereby generating dilated features. The other decoder learns negative attention to the foreground on the same unlabelled images thereby generating eroded features. We then apply a consistency regularisation on the paired predictions. MisMatch outperforms state-of-the-art semi-supervised methods on a CT-based pulmonary vessel segmentation task and a MRI-based brain tumour segmentation task. In addition, we show that the effectiveness of MisMatch comes from better model calibration than its supervised learning counterpart