A Patch-Wise Generative Adversarial Network for PET-MR Image Generation with Feature Attribution for Detection of Focal Cortical Dysplasia

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica) Universidade de Lisboa, Faculdade de Ciências, 2022More than 50 million people worldwide suffer from epilepsy with a third of those being diagnosed with drug-resistant epilepsy where the seizures cannot be treated through pharmacotherapy. In these cases, surgical removal of the epileptic brain tissue in patients is presented as an effective solution for treatment. However, for surgery success, it is vital that the accurate location of epileptic regions in the brain are known. Neuroimaging, specifically magnetic resonance imaging (MRI) and positron emission tomography (PET), commonly are the doctor’s allies in identifying these lesions’ locations responsible for the seizures. Focal cortical dysplasias (FCDs) are the most common type of cortical lesions respon sible for drug-resistant epilepsy in children. These lesions have highly heterogeneous masses, occur in different brain regions and result in different levels of visibility, corresponding to the second most in tractable type of lesion in adults with epilepsy. Moreover, among drug-resistant epilepsy cases, a third of these lesions cannot be correctly identified by neuroimaging experts, resulting in unsuccessful surgical planning and consequently ineffective treatment for patients. Recently, Generative Adversarial Networks (GANs) have demonstrated their value in neuroimaging anomaly detection. Therefore, this work pro poses the application of two different GAN methods – WGAN and CycleGAN - for anomaly detection of FCDs, in PET-MRI data of epileptic patients. A 3D patch-basis anomaly detection approach was therefore developed, inspired by previous works, to detect FCDs location by deconfounding acquisition noise and normal cortical variabilities in PET-MR brain scans of epilepsy patients. Therefore, the GAN models applied two different approaches for lesion detection: detection through reconstruction (WGAN) and detection through translation (CycleGAN). Moreover, the combination of PET and MR modalities was studied and compared to training the networks with individual imaging modalities instead. Through the results, it was possible to understand and correct some issues GAN models have when training with multimodal 3D data. However, both methods for anomaly detection were able to detect diseased brain areas in patients with very visible FCDs, although failing to identify them in patients with very subtle lesions. Recent studies will be briefly discussed in the conclusion, which propose new approaches and architectures for multimodality training, with great potential to improve the performance of the networks for anomaly detection in future works

Individualised, interpretable and reproducible computer-aided diagnosis of dementia: towards application in clinical practice

Neuroimaging offers an unmatched description of the brain’s structure and physiology, but the information it provides is not easy to extract and interpret. A popular way to extract meaningful information from brain images is to use computational methods based on machine learning and deep learning to predict the current or future diagnosis of a patient. A large number of these approaches have been dedicated to the computer-aided diagnosis of dementia, and more specifically of Alzheimer's disease. However, only a few are translated to the clinic. This can be explained by different factors such as the lack of rigorous validation of these approaches leading to over-optimistic performance and their lack of reproducibility, but also the limited interpretability of these methods and their limited generalisability when moving from highly controlled research data to routine clinical data. This manuscript describes how we tried to address these limitations.We have proposed reproducible frameworks for the evaluation of Alzheimer's disease classification methods and developed two open-source software platforms for clinical neuroimaging studies (Clinica) and neuroimaging processing with deep learning (ClinicaDL). We have implemented and assessed the robustness of a visualisation method aiming to interpret convolutional neural networks and used it to study the stability of the network training. We concluded that, currently, combining a convolutional neural networks classifier with an interpretability method may not constitute a robust tool for individual computer-aided diagnosis. As an alternative, we have proposed an approach that detects anomalies in the brain by generating what would be the healthy version of a patient's image and comparing this healthy version with the real image. Finally, we have studied the performance of machine and deep learning algorithms for the computer-aided diagnosis of dementia from images acquired in clinical routine.La neuro-imagerie offre une description inégalée de la structure et de la physiologie du cerveau, mais les informations qu'elle fournit ne sont pas faciles à extraire et à interpréter. Une façon populaire d'extraire des informations pertinentes d'images cérébrales consiste à utiliser des méthodes basées sur l'apprentissage statistique et l'apprentissage profond pour prédire le diagnostic actuel ou futur d'un patient. Un grand nombre de ces approches ont été dédiées au diagnostic assisté par ordinateur de la démence, et plus spécifiquement de la maladie d'Alzheimer. Cependant, seules quelques-unes sont transposées en clinique. Cela peut s'expliquer par différents facteurs tels que l'absence de validation rigoureuse de ces approches conduisant à des performances trop optimistes et à leur manque de reproductibilité, mais aussi l'interprétabilité limitée de ces méthodes et leur généralisation limitée lors du passage de données de recherche hautement contrôlées à des données cliniques de routine. Ce manuscrit décrit comment nous avons tenté de remédier à ces limites.Nous avons proposé des cadres reproductibles pour l'évaluation des méthodes de classification de la maladie d'Alzheimer et développé deux plateformes logicielles open-source pour les études de neuroimagerie clinique (Clinica) et le traitement de la neuroimagerie par apprentissage profond (ClinicaDL). Nous avons implémenté et évalué la robustesse d'une méthode de visualisation visant à interpréter les réseaux neuronaux convolutifs et l'avons utilisée pour étudier la stabilité de l'entraînement du réseau. Nous avons conclu qu'actuellement, la combinaison de réseaux neuronaux convolutifs avec une méthode d'interprétabilité peut ne pas constituer un outil robuste pour le diagnostic individuel assisté par ordinateur. De façon alternative, nous avons proposé une approche qui détecte les anomalies dans le cerveau en générant ce qui serait la version saine de l'image d'un patient et en comparant cette version saine avec l'image réelle. Enfin, nous avons étudié les performances des algorithmes d'apprentissage statistique et profond pour le diagnostic assisté par ordinateur de la démence à partir d'images acquises en routine clinique

Pseudo-normal PET synthesis with generative adversarial networks for localising hypometabolism in epilepsies

International audienc