BrainPainter: A software for the visualisation of brain structures, biomarkers and associated pathological processes

We present BrainPainter, a software that automatically generates images of highlighted brain structures given a list of numbers corresponding to the output colours of each region. Compared to existing visualisation software (i.e. Freesurfer, SPM, 3D Slicer), BrainPainter has three key advantages: (1) it does not require the input data to be in a specialised format, allowing BrainPainter to be used in combination with any neuroimaging analysis tools, (2) it can visualise both cortical and subcortical structures and (3) it can be used to generate movies showing dynamic processes, e.g. propagation of pathology on the brain. We highlight three use cases where BrainPainter was used in existing neuroimaging studies: (1) visualisation of the degree of atrophy through interpolation along a user-defined gradient of colours, (2) visualisation of the progression of pathology in Alzheimer's disease as well as (3) visualisation of pathology in subcortical regions in Huntington's disease. Moreover, through the design of BrainPainter we demonstrate the possibility of using a powerful 3D computer graphics engine such as Blender to generate brain visualisations for the neuroscience community. Blender's capabilities, e.g. particle simulations, motion graphics, UV unwrapping, raster graphics editing, raytracing and illumination effects, open a wealth of possibilities for brain visualisation not available in current neuroimaging software. BrainPainter is customisable, easy to use, and can run straight from the web browser: https://brainpainter.csail.mit.edu , as well as from source-code packaged in a docker container: https://github.com/mrazvan22/brain-coloring . It can be used to visualise biomarker data from any brain imaging modality, or simply to highlight a particular brain structure for e.g. anatomy courses.Comment: Accepted at the MICCAI Multimodal Brain Imaging Analysis (MBIA) workshop, 201

Modelling the Neuroanatomical Progression of Alzheimer's Disease and Posterior Cortical Atrophy

In order to find effective treatments for Alzheimer's disease (AD), we need to identify subjects at risk of AD as early as possible. To this end, recently developed disease progression models can be used to perform early diagnosis, as well as predict the subjects' disease stages and future evolution. However, these models have not yet been applied to rare neurodegenerative diseases, are not suitable to understand the complex dynamics of biomarkers, work only on large multimodal datasets, and their predictive performance has not been objectively validated. In this work I developed novel models of disease progression and applied them to estimate the progression of Alzheimer's disease and Posterior Cortical atrophy, a rare neurodegenerative syndrome causing visual deficits. My first contribution is a study on the progression of Posterior Cortical Atrophy, using models already developed: the Event-based Model (EBM) and the Differential Equation Model (DEM). My second contribution is the development of DIVE, a novel spatio-temporal model of disease progression that estimates fine-grained spatial patterns of pathology, potentially enabling us to understand complex disease mechanisms relating to pathology propagation along brain networks. My third contribution is the development of Disease Knowledge Transfer (DKT), a novel disease progression model that estimates the multimodal progression of rare neurodegenerative diseases from limited, unimodal datasets, by transferring information from larger, multimodal datasets of typical neurodegenerative diseases. My fourth contribution is the development of novel extensions for the EBM and the DEM, and the development of novel measures for performance evaluation of such models. My last contribution is the organization of the TADPOLE challenge, a competition which aims to identify algorithms and features that best predict the evolution of AD.Comment: PhD thesis; Defended in Jan 2019 at University College Londo

Bayesian Image Reconstruction using Deep Generative Models

Machine learning models are commonly trained end-to-end and in a supervised setting, using paired (input, output) data. Examples include recent super-resolution methods that train on pairs of (low-resolution, high-resolution) images. However, these end-to-end approaches require re-training every time there is a distribution shift in the inputs (e.g., night images vs daylight) or relevant latent variables (e.g., camera blur or hand motion). In this work, we leverage state-of-the-art (SOTA) generative models (here StyleGAN2) for building powerful image priors, which enable application of Bayes' theorem for many downstream reconstruction tasks. Our method, Bayesian Reconstruction through Generative Models (BRGM), uses a single pre-trained generator model to solve different image restoration tasks, i.e., super-resolution and in-painting, by combining it with different forward corruption models. We keep the weights of the generator model fixed, and reconstruct the image by estimating the Bayesian maximum a-posteriori (MAP) estimate over the input latent vector that generated the reconstructed image. We further use variational inference to approximate the posterior distribution over the latent vectors, from which we sample multiple solutions. We demonstrate BRGM on three large and diverse datasets: (i) 60,000 images from the Flick Faces High Quality dataset (ii) 240,000 chest X-rays from MIMIC III and (iii) a combined collection of 5 brain MRI datasets with 7,329 scans. Across all three datasets and without any dataset-specific hyperparameter tuning, our simple approach yields performance competitive with current task-specific state-of-the-art methods on super-resolution and in-painting, while being more generalisable and without requiring any training. Our source code and pre-trained models are available online: https://razvanmarinescu.github.io/brgm/.Comment: 27 pages, 17 figures, 5 table

Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference

The heterogeneity of neurodegenerative diseases is a key confound to disease understanding and treatment development, as study cohorts typically include multiple phenotypes on distinct disease trajectories. Here we introduce a machine-learning technique-Subtype and Stage Inference (SuStaIn)-able to uncover data-driven disease phenotypes with distinct temporal progression patterns, from widely available cross-sectional patient studies. Results from imaging studies in two neurodegenerative diseases reveal subgroups and their distinct trajectories of regional neurodegeneration. In genetic frontotemporal dementia, SuStaIn identifies genotypes from imaging alone, validating its ability to identify subtypes; further the technique reveals within-genotype heterogeneity. In Alzheimer's disease, SuStaIn uncovers three subtypes, uniquely characterising their temporal complexity. SuStaIn provides fine-grained patient stratification, which substantially enhances the ability to predict conversion between diagnostic categories over standard models that ignore subtype (p = 7.18 × 10-4) or temporal stage (p = 3.96 × 10-5). SuStaIn offers new promise for enabling disease subtype discovery and precision medicine

BrainPainter: A Software for the Visualisation of Brain Structures, Biomarkers and Associated Pathological Processes

Part of the Lecture Notes in Computer Science book series (LNCS, volume 11846)We present BrainPainter, a software that automatically generates images of highlighted brain structures given a list of numbers corresponding to the output colours of each region. Compared to existing visualisation software (i.e. Freesurfer, SPM, 3D Slicer), BrainPainter has three key advantages: (1) it does not require the input data to be in a specialised format, allowing BrainPainter to be used in combination with any neuroimaging analysis tools, (2) it can visualise both cortical and subcortical structures and (3) it can be used to generate movies showing dynamic processes, e.g. propagation of pathology on the brain. We highlight three use cases where BrainPainter was used in existing neuroimaging studies: (1) visualisation of the degree of atrophy through interpolation along a user-defined gradient of colours, (2) visualisation of the progression of pathology in Alzheimer’s disease as well as (3) visualisation of pathology in subcortical regions in Huntington’s disease. Moreover, through the design of BrainPainter we demonstrate the possibility of using a powerful 3D computer graphics engine such as Blender to generate brain visualisations for the neuroscience community. Blender’s capabilities, e.g. particle simulations, motion graphics, UV unwrapping, raster graphics editing, raytracing and illumination effects, open a wealth of possibilities for brain visualisation not available in current neuroimaging software. BrainPainter (Source code: https://github.com/mrazvan22/brain-coloring ) is customisable, easy to use, and can run straight from the web browser: http://brainpainter.csail.mit.edu. It can be used to visualise biomarker data from any brain imaging modality, or simply to highlight a particular brain structure for e.g. anatomy courses.NIH (Grants NIBIB NAC P41EB015902 and NINDS R01NS086905

Data-Driven multi-Contrast spectral microstructure imaging with InSpect:INtegrated SPECTral component estimation and mapping

We introduce and demonstrate an unsupervised machine learning technique for spectroscopic analysis of quantitative MRI experiments. Our algorithm supports estimation of one-dimensional spectra from single-contrast data, and multidimensional correlation spectra from simultaneous multi-contrast data. These spectrum-based approaches allow model-free investigation of tissue properties, but require regularised inversion of a Laplace transform or Fredholm integral, which is an ill-posed calculation. Here we present a method that addresses this limitation in a data-driven way. The algorithm simultaneously estimates a canonical basis of spectral components and voxelwise maps of their weightings, thereby pooling information across whole images to regularise the ill-posed problem. We show in simulations that our algorithm substantially outperforms current voxelwise spectral approaches. We demonstrate the method on multi-contrast diffusion-relaxometry placental MRI scans, revealing anatomically-relevant sub-structures, and identifying dysfunctional placentas. Our algorithm vastly reduces the data required to reliably estimate spectra, opening up the possibility of quantitative MRI spectroscopy in a wide range of new applications. Our InSpect code is available at github.com/paddyslator/inspect

Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference.

The heterogeneity of neurodegenerative diseases is a key confound to disease understanding and treatment development, as study cohorts typically include multiple phenotypes on distinct disease trajectories. Here we introduce a machine-learning technique-Subtype and Stage Inference (SuStaIn)-able to uncover data-driven disease phenotypes with distinct temporal progression patterns, from widely available cross-sectional patient studies. Results from imaging studies in two neurodegenerative diseases reveal subgroups and their distinct trajectories of regional neurodegeneration. In genetic frontotemporal dementia, SuStaIn identifies genotypes from imaging alone, validating its ability to identify subtypes; further the technique reveals within-genotype heterogeneity. In Alzheimer's disease, SuStaIn uncovers three subtypes, uniquely characterising their temporal complexity. SuStaIn provides fine-grained patient stratification, which substantially enhances the ability to predict conversion between diagnostic categories over standard models that ignore subtype (p = 7.18 × 10-4) or temporal stage (p = 3.96 × 10-5). SuStaIn offers new promise for enabling disease subtype discovery and precision medicine

Progression of regional grey matter atrophy in multiple sclerosis

See Stankoff and Louapre (doi:) for a scientific commentary on this article. Grey matter atrophy in multiple sclerosis affects certain areas preferentially. Eshaghi et al. use a data-driven computational model to predict the order in which regions atrophy, and use this sequence to stage patients. Atrophy begins in deep grey matter nuclei and posterior cortical regions, before spreading to other cortical areas. See Stankoff and Louapre (doi:) for a scientific commentary on this article. Grey matter atrophy is present from the earliest stages of multiple sclerosis, but its temporal ordering is poorly understood. We aimed to determine the sequence in which grey matter regions become atrophic in multiple sclerosis and its association with disability accumulation. In this longitudinal study, we included 1417 subjects: 253 with clinically isolated syndrome, 708 with relapsing-remitting multiple sclerosis, 128 with secondary-progressive multiple sclerosis, 125 with primary-progressive multiple sclerosis, and 203 healthy control subjects from seven European centres. Subjects underwent repeated MRI (total number of scans 3604); the mean follow-up for patients was 2.41 years (standard deviation = 1.97). Disability was scored using the Expanded Disability Status Scale. We calculated the volume of brain grey matter regions and brainstem using an unbiased within-subject template and used an established data-driven event-based model to determine the sequence of occurrence of atrophy and its uncertainty. We assigned each subject to a specific event-based model stage, based on the number of their atrophic regions. Linear mixed-effects models were used to explore associations between the rate of increase in event-based model stages, and T lesion load, disease-modifying treatments, comorbidity, disease duration and disability accumulation. The first regions to become atrophic in patients with clinically isolated syndrome and relapse-onset multiple sclerosis were the posterior cingulate cortex and precuneus, followed by the middle cingulate cortex, brainstem and thalamus. A similar sequence of atrophy was detected in primary-progressive multiple sclerosis with the involvement of the thalamus, cuneus, precuneus, and pallidum, followed by the brainstem and posterior cingulate cortex. The cerebellum, caudate and putamen showed early atrophy in relapse-onset multiple sclerosis and late atrophy in primary-progressive multiple sclerosis. Patients with secondary-progressive multiple sclerosis showed the highest event-based model stage (the highest number of atrophic regions, P < 0.001) at the study entry. All multiple sclerosis phenotypes, but clinically isolated syndrome, showed a faster rate of increase in the event-based model stage than healthy controls. T lesion load and disease duration in all patients were associated with increased event-based model stage, but no effects of disease-modifying treatments and comorbidity on event-based model stage were observed. The annualized rate of event-based model stage was associated with the disability accumulation in relapsing-remitting multiple sclerosis, independent of disease duration (P < 0.0001). The data-driven staging of atrophy progression in a large multiple sclerosis sample demonstrates that grey matter atrophy spreads to involve more regions over time. The sequence in which regions become atrophic is reasonably consistent across multiple sclerosis phenotypes. The spread of atrophy was associated with disease duration and with disability accumulation over time in relapsing-remitting multiple sclerosis

Progression of regional grey matter atrophy in multiple sclerosis

See Stankoff and Louapre (doi:10.1093/brain/awy114) for a scientific commentary on this article.Grey matter atrophy is present from the earliest stages of multiple sclerosis, but its temporal ordering is poorly understood. We aimed to determine the sequence in which grey matter regions become atrophic in multiple sclerosis and its association with disability accumulation. In this longitudinal study, we included 1417 subjects: 253 with clinically isolated syndrome, 708 with relapsing-remitting multiple sclerosis, 128 with secondary-progressive multiple sclerosis, 125 with primary-progressive multiple sclerosis, and 203 healthy control subjects from seven European centres. Subjects underwent repeated MRI (total number of scans 3604); the mean follow-up for patients was 2.41 years (standard deviation = 1.97). Disability was scored using the Expanded Disability Status Scale. We calculated the volume of brain grey matter regions and brainstem using an unbiased within-subject template and used an established data-driven event-based model to determine the sequence of occurrence of atrophy and its uncertainty. We assigned each subject to a specific event-based model stage, based on the number of their atrophic regions. Linear mixed-effects models were used to explore associations between the rate of increase in event-based model stages, and T2 lesion load, disease-modifying treatments, comorbidity, disease duration and disability accumulation. The first regions to become atrophic in patients with clinically isolated syndrome and relapse-onset multiple sclerosis were the posterior cingulate cortex and precuneus, followed by the middle cingulate cortex, brainstem and thalamus. A similar sequence of atrophy was detected in primary-progressive multiple sclerosis with the involvement of the thalamus, cuneus, precuneus, and pallidum, followed by the brainstem and posterior cingulate cortex. The cerebellum, caudate and putamen showed early atrophy in relapse-onset multiple sclerosis and late atrophy in primary-progressive multiple sclerosis. Patients with secondary-progressive multiple sclerosis showed the highest event-based model stage (the highest number of atrophic regions, P < 0.001) at the study entry. All multiple sclerosis phenotypes, but clinically isolated syndrome, showed a faster rate of increase in the event-based model stage than healthy controls. T2 lesion load and disease duration in all patients were associated with increased event-based model stage, but no effects of disease-modifying treatments and comorbidity on event-based model stage were observed. The annualized rate of event-based model stage was associated with the disability accumulation in relapsing-remitting multiple sclerosis, independent of disease duration (P < 0.0001). The data-driven staging of atrophy progression in a large multiple sclerosis sample demonstrates that grey matter atrophy spreads to involve more regions over time. The sequence in which regions become atrophic is reasonably consistent across multiple sclerosis phenotypes. The spread of atrophy was associated with disease duration and with disability accumulation over time in relapsing-remitting multiple sclerosis