Visual and Contextual Modeling for the Detection of Repeated Mild Traumatic Brain Injury.

Currently, there is a lack of computational methods for the evaluation of mild traumatic brain injury (mTBI) from magnetic resonance imaging (MRI). Further, the development of automated analyses has been hindered by the subtle nature of mTBI abnormalities, which appear as low contrast MR regions. This paper proposes an approach that is able to detect mTBI lesions by combining both the high-level context and low-level visual information. The contextual model estimates the progression of the disease using subject information, such as the time since injury and the knowledge about the location of mTBI. The visual model utilizes texture features in MRI along with a probabilistic support vector machine to maximize the discrimination in unimodal MR images. These two models are fused to obtain a final estimate of the locations of the mTBI lesion. The models are tested using a novel rodent model of repeated mTBI dataset. The experimental results demonstrate that the fusion of both contextual and visual textural features outperforms other state-of-the-art approaches. Clinically, our approach has the potential to benefit both clinicians by speeding diagnosis and patients by improving clinical care

The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low-and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource

Automatic Segmentation of Intramedullary Multiple Sclerosis Lesions

Contexte: La moelle épinière est un composant essentiel du système nerveux central. Elle contient des neurones responsables d’importantes fonctionnalités et assure la transmission d’informations motrices et sensorielles entre le cerveau et le système nerveux périphérique. Un endommagement de la moelle épinière, causé par un choc ou une maladie neurodégénérative, peut mener à un sérieux handicap, pouvant entraîner des incapacités fonctionnelles, de la paralysie et/ou de la douleur. Chez les patients atteints de sclérose en plaques (SEP), la moelle épinière est fréquemment affectée par de l’atrophie et/ou des lésions. L’imagerie par résonance magnétique (IRM) conventionnelle est largement utilisée par des chercheurs et des cliniciens pour évaluer et caractériser, de façon non-invasive, des altérations micro-structurelles. Une évaluation quantitative des atteintes structurelles portées à la moelle épinière (e.g. sévérité de l’atrophie, extension des lésions) est essentielle pour le diagnostic, le pronostic et la supervision sur le long terme de maladies, telles que la SEP. De plus, le développement de biomarqueurs impartiaux est indispensable pour évaluer l’effet de nouveaux traitements thérapeutiques. La segmentation de la moelle épinière et des lésions intramédullaires de SEP sont, par conséquent, pertinentes d’un point de vue clinique, aussi bien qu’une étape nécessaire vers l’interprétation d’images RM multiparamétriques. Cependant, la segmentation manuelle est une tâche extrêmement chronophage, fastidieuse et sujette à des variations inter- et intra-expert. Il y a par conséquent un besoin d’automatiser les méthodes de segmentations, ce qui pourrait faciliter l’efficacité procédures d’analyses. La segmentation automatique de lésions est compliqué pour plusieurs raisons: (i) la variabilité des lésions en termes de forme, taille et position, (ii) les contours des lésions sont la plupart du temps difficilement discernables, (iii) l’intensité des lésions sur des images MR sont similaires à celles de structures visiblement saines. En plus de cela, réaliser une segmentation rigoureuse sur l’ensemble d’une base de données multi-centrique d’IRM est rendue difficile par l’importante variabilité des protocoles d’acquisition (e.g. résolution, orientation, champ de vue de l’image). Malgré de considérables récents développements dans le traitement d’images MR de moelle épinière, il n’y a toujours pas de méthode disponible pouvant fournir une segmentation rigoureuse et fiable de la moelle épinière pour un large spectre de pathologies et de protocoles d’acquisition. Concernant les lésions intramédullaires, une recherche approfondie dans la littérature n’a pas pu fournir une méthode disponible de segmentation automatique. Objectif: Développer un système complètement automatique pour segmenter la moelle épinière et les lésions intramédullaires sur des IRM conventionnelles humaines. Méthode: L’approche présentée est basée de deux réseaux de neurones à convolution mis en cascade. La méthode a été pensée pour faire face aux principaux obstacles que présentent les données IRM de moelle épinière. Le procédé de segmentation a été entrainé et validé sur une base de données privée composée de 1943 images, acquises dans 30 différents centres avec des protocoles hétérogènes. Les sujets scannés comportent 459 sujets sains, 471 patients SEP et 112 avec d’autres pathologies affectant la moelle épinière. Le module de segmentation de la moelle épinière a été comparé à une méthode existante reconnue par la communauté, PropSeg. Résultats: L’approche basée sur les réseaux de neurones à convolution a fourni de meilleurs résultats que PropSeg, atteignant un Dice médian (intervalle inter-quartiles) de 94.6 (4.6) vs. 87.9 (18.3) %. Pour les lésions, notre segmentation automatique a permis d'obtenir un Dice de 60.0 (21.4) % en le comparant à la segmentation manuelle, un ratio de vrai positifs de 83 (34) %, et une précision de 77 (44) %. Conclusion: Une méthode complètement automatique et innovante pour segmenter la moelle épinière et les lésions SEP intramédullaires sur des données IRM a été conçue durant ce projet de maîtrise. La méthode a été abondamment validée sur une base de données clinique. La robustesse de la méthode de segmentation de moelle épinière a été démontrée, même sur des cas pathologiques. Concernant la segmentation des lésions, les résultats sont encourageants, malgré un taux de faux positifs relativement élevé. Je crois en l’impact que peut potentiellement avoir ces outils pour la communauté de chercheurs. Dans cette optique, les méthodes ont été intégrées et documentées dans un logiciel en accès-ouvert, la “Spinal Cord Toolbox”. Certains des outils développés pendant ce projet de Maîtrise sont déjà utilisés par des analyses d’études cliniques, portant sur des patients SEP et sclérose latérale amyotrophique.----------ABSTRACT Context: The spinal cord is a key component of the central nervous system, which contains neurons responsible for complex functions, and ensures the conduction of motor and sensory information between the brain and the peripheral nervous system. Damage to the spinal cord, through trauma or neurodegenerative diseases, can lead to severe impairment, including functional disabilities, paralysis and/or pain. In multiple sclerosis (MS) patients, the spinal cord is frequently affected by atrophy and/or lesions. Conventional magnetic resonance imaging (MRI) is widely used by researchers and clinicians to non-invasively assess and characterize spinal cord microstructural changes. Quantitative assessment of the structural damage to the spinal cord (e.g. atrophy severity, lesion extent) is essential for the diagnosis, prognosis and longitudinal monitoring of diseases, such as MS. Furthermore, the development of objective biomarkers is essential to evaluate the effect of new therapeutic treatments. Spinal cord and intramedullary MS lesions segmentation is consequently clinically relevant, as well as a necessary step towards the interpretation of multi-parametric MR images. However, manual segmentation is highly time-consuming, tedious and prone to intra- and inter-rater variability. There is therefore a need for automated segmentation methods to facilitate the efficiency of analysis pipelines. Automatic lesion segmentation is challenging for various reasons: (i) lesion variability in terms of shape, size and location, (ii) lesion boundaries are most of the time not well defined, (iii) lesion intensities on MR data are confounding with those of normal-appearing structures. Moreover, achieving robust segmentation across multi-center MRI data is challenging because of the broad variability of data features (e.g. resolution, orientation, field of view). Despite recent substantial developments in spinal cord MRI processing, there is still no method available that can yield robust and reliable spinal cord segmentation across the very diverse spinal pathologies and data features. Regarding the intramedullary lesions, a thorough search of the relevant literature did not yield available method of automatic segmentation. Goal: To develop a fully-automatic framework for segmenting the spinal cord and intramedullary MS lesions from conventional human MRI data. Method: The presented approach is based on a cascade of two Convolutional Neural Networks (CNN). The method has been designed to face the main challenges of ‘real world’ spinal cord MRI data. It was trained and validated on a private dataset made up of 1943 MR volumes, acquired in different 30 sites with heterogeneous acquisition protocols. Scanned subjects involve 459 healthy controls, 471 MS patients and 112 with other spinal pathologies. The proposed spinal cord segmentation method was compared to a state-of-the-art spinal cord segmentation method, PropSeg. Results: The CNN-based approach achieved better results than PropSeg, yielding a median (interquartile range) Dice of 94.6 (4.6) vs. 87.9 (18.3) % when compared to the manual segmentation. For the lesion segmentation task, our method provided a median Dice-overlap with the manual segmentation of 60.0 (21.4) %, a lesion-based true positive rate of 83 (34) % and a lesion-based precision de 77 (44) %. Conclusion: An original fully-automatic method to segment the spinal cord and intramedullary MS lesions on MRI data has been devised during this Master’s project. The method was validated extensively against a clinical dataset. The robustness of the spinal cord segmentation has been demonstrated, even on challenging pathological cases. Regarding the lesion segmentation, the results are encouraging despite the fairly high false positive rate. I believe in the potential value of these developed tools for the research community. In this vein, the methods are integrated and documented into an open-source software, the Spinal Cord Toolbox. Some of the tools developed during this Master’s project are already integrated into automated analysis pipelines of clinical studies, including MS and Amyotrophic Lateral Sclerosis patients

Development of an image processing pipeline for the study of corticol lesions in multiple sclerosis patients using ultra-high field MRI

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Biofísica Médica e Fisiologia de Sistemas), Universidade de Lisboa, Faculdade de Ciências, 2019A esclerose múltipla é uma doença crónica e inflamatória do sistema nervoso central de alta prevalência nos dias de hoje. Durante anos, o foco da doença foi a patologia visível na matéria branca. Apesar dos primeiros estudos de patologia cortical em esclerose múltipla apontarem para a década de 60, foi apenas no início do novo século que o córtex passou a ser estudado como parte integral da doença. Desde então, estudos têm vindo a demonstrar que o comprometimento do córtex parece estar relacionado com danos cognitivos e físicos, frequentemente associados à doença. A necessidade de melhor compreender o impacto das lesões corticais no desenvolvimento da doença e na vida diária destes pacientes tem motivado o seu estudo, sendo a Ressonância Magnética (RM), em particular scanners de campo ultra-alto, a melhor ferramenta para as detetar e estudar. A melhoria da razão sinal-ruído e da resolução espacial dos scanners de RM de campo ultra-alto tem permitido o aumento da deteção de lesões corticais. Ainda assim, a sua sensibilidade continua a não ser ideal e a estar fortemente dependente do tipo de lesão cortical, do contraste de RM usado na sua deteção e da existência de ferramentas robustas que permitam a sua deteção de modo automático, mais eficiente e com menor espaço para erro. A falta de marcadores de imagem para a remielinização ou desmielinização parcial, tal como a ausência de diretrizes para a deteção destas lesões com campos de 7 (T)esla parece explicar a dificuldade em distinguir e identificar falsos positivos e as diferenças encontradas nas deteções realizadas por diferentes avaliadores. Uma desvantagem dos scanners de campo ultra-alto é o maior efeito de bias que, caso não seja removido aquando da aquisição de imagens, terá de ser removido na fase de processamento por softwares e algoritmos que não estão originalmente construídos para trabalhar com imagens de maior resolução e cuja prestação não está ainda bem explorada nestas condições. Estes desafios comprometem o potencial dos scanners de RM de campo ultra-alto para o estudo das lesões corticais na esclerose múltipla. Este projeto procura desenvolver uma pipeline semiautomática para o pré-processamento e processamento de imagens de RM de cariz estrutural de doentes com esclerose múltipla obtidas num scanner de campo ultra-alto. A pipeline é criada de modo gradual, recorrendo a análises visuais, ou de outro tipo, para confirmar a qualidade de cada passo antes de avançar para o seguinte, no pressuposto de que a qualidade dos softwares de imagem comercialmente disponíveis será menor ao utilizar imagens de maior resolução. A ocorrência de lesões corticais no córtex sensório-motor (SM1) é igualmente determinada e usada para validar a qualidade da pipeline. Doze doentes com esclerose múltipla na sua forma recidivante-remitente ou secundariamente progressiva e seis controlos foram incluídos neste projeto. Todas as permissões necessárias do comité local de ética, proteção de dados e da Danish Medicines Agency foram previamente obtidas. Os doentes foram estudados num scanner de RM de corpo inteiro da Philips, Achieva 7,0 T, dedicado a investigação. Os participantes foram observados usando quatro tipos distintos de contraste: magnetization prepared rapid acquisition by gradient echo (MPRAGE) a três dimensões (3D) com 0,65-mm de resolução isotrópica, 3D fluid attenuated inversion recovery (FLAIR) com 0,7-mm de resolução isotrópica, 3D T1-weighted (T1w) de resolução 0,85x0,85x1,0 mm3 e 3D T2-weighted Turbo Spin Echo (T2w-TSE) de 0,4-mm de resolução isotrópica. A vertente de pré-processamento da pipeline incluiu uma correção de bias e o co-registo de imagens. Para a correção de bias, o software SPM foi testado utilizando os parâmetros habituais e uma alteração dos parâmetros relativos à smoothness e regularização, como sugerido na literatura. O processo de co-registo seguiu o procedimento utilizado no processamento de imagens de doentes com esclerose múltipla de 3 T no Danish Research Centre for Magnetic Resonance (DRCMR), com alterações posteriormente adicionadas para melhorar a qualidade do alinhamento das imagens de cada indivíduo a 7 T. Após o pré-processamento, uma deteção de lesões corticais, seguida da sua segmentação, foi realizada manualmente utilizando as ferramentas do software FSL. A vertente de processamento da pipeline incluiu uma segmentação do cérebro, um registo das imagens dos doentes e a criação de superfícies corticais. A segmentação foi testada utilizando três diferentes ferramentas: o software SPM, uma toolbox do SPM, CAT, e a ferramenta de segmentação do FSL, FAST. A toolbox do SPM, DARTEL, foi usada no registo de imagens e o software FreeSurfer permitiu a criação de superfícies individuais e de grupo no último passo da pipeline. As máscaras com as lesões criadas após a segmentação manual de lesões seguiram um caminho semelhante de processamento de modo a permitir a sua correta sobreposição no respetivo volume, e, posteriormente, superfície, e a possibilidade de fazer análises individuais ou de grupo. Os resultados obtidos mostraram que os softwares para processamento de imagens de RM disponíveis apresentam, em geral, uma boa prestação e fornecem resultados de confiança. Ainda assim, a sua prestação pode ser otimizada incluindo procedimentos adicionais em cada passo ou por alteração das configurações originais dos softwares. A diminuição do parâmetro de largura à meia altura com um aumento do parâmetro de regularização na correção de bias com o SPM permitiu a criação de campos de bias mais fieis às imagens originais, consequentemente melhorando a sua correção e a diferenciação da matéria branca e matéria cinzenta nas imagens resultantes. A criação adicional de máscaras contendo apenas o cérebro e a utilização exclusiva de transformações de corpo rígido no co-registo de imagens permitiu a utilização de vários contrastes na tarefa de deteção de lesões, sem interferir com a sua localização ou morfologia. Na segmentação, a toolbox do SPM, CAT, mostrou melhorias na capacidade de separar as diferentes classes de tecidos com maior confiança e qualidade, particularmente nas regiões de contacto entre a matéria branca e cinzenta. Consequentemente, a qualidade do alinhamento das imagens dos diferentes doentes e a posterior criação de uma imagem média a partir de imagens individuais foi melhorada. O sucesso da pipeline permitiu a sobreposição das lesões corticais manualmente segmentadas nas superfícies individuais e/ou comuns criadas, onde foi descoberto que a maioria das lesões ocorreu no hemisfério direito, com sobreposições de lesões respetivas a diferentes doentes a ocorrer maioritariamente nos sulcos corticais, comparativamente aos giros. Porém, a segmentação de lesões demonstrou ser dispendiosa, dependente do avaliador e altamente influenciada por fatores inerentes ao avaliador, tal como o cansaço, nível de concentração ou de aborrecimento, e fatores externos, no qual se destacam a luminosidade do computador ou a luminosidade da sala onde a deteção foi feita. A feature do FreeSurfer para imagens de maior resolução não se mostrou fiável no tratamento dos dados de resolução isotrópica de 0,5-mm deste projeto, uma possível razão pela qual ainda se encontra em desenvolvimento. Apesar dos bons resultados obtidos, investigação adicional será necessária para melhor compreender a prestação destes e de outros softwares para imagem médica no processamento de imagens de RM de maior resolução, tal como a melhor maneira de tirar partido dos mesmos em estudos clínicos a 7 T. A extensão da pipeline a outros doentes com esclerose múltipla irá aumentar a amostra em estudo e permitir um estudo mais extensivo da patologia cortical e a compreensão do impacto de uma ou mais lesões localizadas na região SM1 na conectividade e integridade funcional da região cortical afetada.The importance of grey matter pathology to the understanding of multiple sclerosis has been acknowledged. However, the sensitivity to cortical lesions is limited when using conventional magnetic resonance imaging (MRI) systems. Ultra-high field (UHF) MRI systems have improved detection sensitivity but impose the additional challenge of a higher effect of bias to account for. Currently, image processing tools are not designed for higher resolution data and the performance of common software packages under these conditions has not been properly explored. These challenges have impaired the potential of UHF-MRI to study cortical lesions in multiple sclerosis. This project aims at developing a semi-automated pipeline for the pre-processing and processing of structural UHF-MRI data of multiple sclerosis patients. The pipeline is built in a step-by-step fashion, making use of visual assessments and other analyses to confirm the quality of each step before advancing to the next, under the assumption that the performance of common imaging software packages will be poorer when using higher resolution data. The occurrence of cortical lesions within the primary sensory-motor cortex (SM1) is also determined and used to validate the quality of the pipeline. Twelve patients with relapsing-remitting multiple sclerosis or secondary progressive multiple sclerosis and six healthy age-matched controls were included in this project. All relevant permissions from the local ethics committee and data protection had been obtained beforehand. All participants were studied with whole-brain ultra-high field MRI at 7 Tesla (T), using a research-only 7 T Achieva MR system. The participants were scanned using four different MRI modalities, namely 3-dimensional (3D) magnetization prepared rapid acquisition by gradient echo (MPRAGE) at 0.65-mm isotropic resolution, 3D fluid attenuated inversion recovery (FLAIR) at 0.7-mm isotropic resolution, 3D T1-weighted (T1w) of 0.85x0.85x1.0 mm3 reconstructed resolution and 3D T2-weighted Turbo Spin Echo (T2w-TSE) at 0.4-mm isotropic reconstructed resolution. The pre-processing pipeline included a bias correction and a coregistration step. For the bias correction, SPM was tested using its default parameters and an alternative configuration that altered the smoothness and regularization parameters. The coregistration followed an approach used in the processing of multiple sclerosis data at 3 T, with changes added to improve the quality of the within-subject alignment at 7 T. After the data pre-processing, manual detection and segmentation of cortical lesions was performed using FSLeyes. The processing pipeline included brain segmentation, subject registration and cortical surface creation. Brain segmentation was tested with SPM, with SPM’s toolbox, CAT, and with FSL’s segmentation tool, FAST. SPM’s DARTEL tool was used for subject registration and FreeSurfer allowed the creation of individual and an average cortical surface. The lesion masks created after the manual segmentation task followed a similar processing route to allow their overlay on the respective brain volumes and, posteriorly, surfaces, and the possibility of individual and group analyses. Results showed that the currently available MRI image processing tools present overall good performance and reliability in the processing of higher resolution data of multiple sclerosis patients. Still, the quality of the outcomes can be optimized by including additional steps or changes to the original software configurations. Modifying SPM’s smoothness and regularization parameters for the estimation of bias minimized its effect in the data, allowing a better differentiation between grey matter and white matter. Removing the skull whilst keeping the coregistration to rigid body transformations allowed the use of several contrasts in the lesion detection task without interfering with the lesions’ morphology and topography. Brain segmentation using CAT showed more stability across the dataset, improving the quality of the subsequent subject registration and consequently of the average brain created. The success of the pipeline led to the possibility of overlaying the manually segmented lesions on the individual and group surfaces where it was found that the majority of lesions occurred on the right hemisphere and that lesion overlaps were more common in cortical sulci. Despite the results obtained, further research is needed to understand the performance of other software packages in the processing of higher resolution MRI data and how to fully exploit these tools in the study of clinical data at 7 T

Automated brain segmentation methods for clinical quality MRI and CT images

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder associated with brain tissue loss. Accurate estimation of this loss is critical for the diagnosis, prognosis, and tracking the progression of AD. Structural magnetic resonance imaging (sMRI) and X-ray computed tomography (CT) are widely used imaging modalities that help to in vivo map brain tissue distributions. As manual image segmentations are tedious and time-consuming, automated segmentation methods are increasingly applied to head MRI and head CT images to estimate brain tissue volumes. However, existing automated methods can be applied only to images that have high spatial resolution and their accuracy on heterogeneous low-quality clinical images has not been tested. Further, automated brain tissue segmentation methods for CT are not available, although CT is more widely acquired than MRI in the clinical setting. For these reasons, large clinical imaging archives are unusable for research studies. In this work, we identify and develop automated tissue segmentation and brain volumetry methods that can be applied to clinical quality MRI and CT images. In the first project, we surveyed the current MRI methods and validated the accuracy of these methods when applied to clinical quality images. We then developed CTSeg, a tissue segmentation method for CT images, by adopting the MRI technique that exhibited the highest reliability. CTSeg is an atlas-based statistical modeling method that relies on hand-curated features and cannot be applied to images of subjects with different diseases and age groups. Advanced deep learning-based segmentation methods use hierarchical representations and learn complex features in a data-driven manner. In our final project, we develop a fully automated deep learning segmentation method that uses contextual information to segment clinical quality head CT images. The application of this method on an AD dataset revealed larger differences between brain volumes of AD and control subjects. This dissertation demonstrates the potential of applying automated methods to large clinical imaging archives to answer research questions in a variety of studies