Robust chest CT image segmentation of COVID-19 lung infection based on limited data

Background The coronavirus disease 2019 (COVID-19) affects billions of lives around the world and has a significant impact on public healthcare. For quantitative assessment and disease monitoring medical imaging like computed tomography offers great potential as alternative to RT-PCR methods. For this reason, automated image segmentation is highly desired as clinical decision support. However, publicly available COVID-19 imaging data is limited which leads to overfitting of traditional approaches. Methods To address this problem, we propose an innovative automated segmentation pipeline for COVID-19 infected regions, which is able to handle small datasets by utilization as variant databases. Our method focuses on on-the-fly generation of unique and random image patches for training by performing several preprocessing methods and exploiting extensive data augmentation. For further reduction of the overfitting risk, we implemented a standard 3D U-Net architecture instead of new or computational complex neural network architectures. Results Through a k-fold cross-validation on 20 CT scans as training and validation of COVID-19, we were able to develop a highly accurate as well as robust segmentation model for lungs and COVID-19 infected regions without overfitting on limited data. We performed an in-detail analysis and discussion on the robustness of our pipeline through a sensitivity analysis based on the cross-validation and impact on model generalizability of applied preprocessing techniques. Our method achieved Dice similarity coefficients for COVID-19 infection between predicted and annotated segmentation from radiologists of 0.804 on validation and 0.661 on a separate testing set consisting of 100 patients. Conclusions We demonstrated that the proposed method outperforms related approaches, advances the state-of-the-art for COVID-19 segmentation and improves robust medical image analysis based on limited data

Reference tissue normalization of prostate MRI with automatic multi-organ deep learning pelvis segmentation

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica) Universidade de Lisboa, Faculdade de Ciências, 2018Prostate cancer is the most common cancer among male patients and second leading cause of death from cancer in men (excluding non-melanoma skin cancer). Magnetic Resonance Imaging (MRI) is currently becoming the modality of choice for clinical staging of localized prostate cancer. However, MRI lacks intensity quantification which hinders its diagnostic ability. The overall aim of this dissertation is to automate a novel normalization method that can potentially quantify general MR intensities, thus improving the diagnostic ability of MRI. Two Prostate multi-parametric MRI cohorts, of 2012 and 2016, were used in this retrospective study. To improve the diagnostic ability of T2-Weighted MRI, a novel multi-reference tissue normalization method was tested and automated. This method consists of computing the average intensity of the reference tissues and the corresponding normalized reference values to define a look-up-table through interpolation. Since the method requires delineation of multiple reference tissues, an MRI-specific Deep Learning model, Aniso-3DUNET, was trained on manual segmentations and tested to automate this segmentation step. The output of the Deep Learning model, that consisted of automatic segmentations, was validated and used in an automatic normalization approach. The effect of the manual and automatic normalization approaches on diagnostic accuracy of T2-weighted intensities was determined with Receiver Operating Characteristic (ROC) analyses. The Areas Under the Curve (AUC) were compared. The automatic segmentation of multiple reference-tissues was validated with an average DICE score higher than 0.8 in the test phase. Thereafter, the method developed demonstrated that the normalized intensities lead to an improved diagnostic accuracy over raw intensities using the manual approach, with an AUC going from 0.54 (raw) to 0.68 (normalized), and automatic approach, with an AUC going from 0.68 to 0.73. This study demonstrates that multi-reference tissue normalization improves quantification of T2-weighted images and diagnostic accuracy, possibly leading to a decrease in radiologist’s interpretation variability. It is also possible to conclude that this novel T2-weighted MRI normalization method can be automatized, becoming clinically applicable

PREDICTION OF 1P/19Q CODELETION STATUS IN DIFFUSE GLIOMA PATIENTS USING PREOPERATIVE MULTIPARAMETRIC MAGNETIC RESONANCE IMAGING

A complete codeletion of chromosome 1p/19q is strongly correlated with better overall survival of diffuse glioma patients, hence determining the codeletion status early in the course of a patient’s disease would be valuable in that patient’s care. The current practice requires a surgical biopsy in order to assess the codeletion status, which exposes patients to risks and is limited in its accuracy by sampling variations. To overcome such limitations, we utilized four conventional magnetic resonance imaging sequences to predict the 1p/19q status. We extracted three sets of image-derived features, namely texture-based, topology-based, and convolutional neural network (CNN)-based, and analyzed each feature’s prediction performance. The topology-based model (AUC = 0.855 +/- 0.079) performed significantly better compared to the texture-based model (AUC = 0.707 +/- 0.118) while comparably against the CNN-based model (0.787 +/- 0.195). However, none of the models performed better than the baseline model that is built with only clinical variables, namely, age, gender, and Karnofsky Performance Score (AUC = 0.703 +/- 0.256). In summary, predicting 1p/19q chromosome codeletion status via MRI scan analysis can be a viable non-invasive assessment tool at an early stage of gliomas and in follow-ups although further investigation is needed to improve the model performance

Automatic Segmentation of Intramedullary Multiple Sclerosis Lesions

Contexte: La moelle épinière est un composant essentiel du système nerveux central. Elle contient des neurones responsables d’importantes fonctionnalités et assure la transmission d’informations motrices et sensorielles entre le cerveau et le système nerveux périphérique. Un endommagement de la moelle épinière, causé par un choc ou une maladie neurodégénérative, peut mener à un sérieux handicap, pouvant entraîner des incapacités fonctionnelles, de la paralysie et/ou de la douleur. Chez les patients atteints de sclérose en plaques (SEP), la moelle épinière est fréquemment affectée par de l’atrophie et/ou des lésions. L’imagerie par résonance magnétique (IRM) conventionnelle est largement utilisée par des chercheurs et des cliniciens pour évaluer et caractériser, de façon non-invasive, des altérations micro-structurelles. Une évaluation quantitative des atteintes structurelles portées à la moelle épinière (e.g. sévérité de l’atrophie, extension des lésions) est essentielle pour le diagnostic, le pronostic et la supervision sur le long terme de maladies, telles que la SEP. De plus, le développement de biomarqueurs impartiaux est indispensable pour évaluer l’effet de nouveaux traitements thérapeutiques. La segmentation de la moelle épinière et des lésions intramédullaires de SEP sont, par conséquent, pertinentes d’un point de vue clinique, aussi bien qu’une étape nécessaire vers l’interprétation d’images RM multiparamétriques. Cependant, la segmentation manuelle est une tâche extrêmement chronophage, fastidieuse et sujette à des variations inter- et intra-expert. Il y a par conséquent un besoin d’automatiser les méthodes de segmentations, ce qui pourrait faciliter l’efficacité procédures d’analyses. La segmentation automatique de lésions est compliqué pour plusieurs raisons: (i) la variabilité des lésions en termes de forme, taille et position, (ii) les contours des lésions sont la plupart du temps difficilement discernables, (iii) l’intensité des lésions sur des images MR sont similaires à celles de structures visiblement saines. En plus de cela, réaliser une segmentation rigoureuse sur l’ensemble d’une base de données multi-centrique d’IRM est rendue difficile par l’importante variabilité des protocoles d’acquisition (e.g. résolution, orientation, champ de vue de l’image). Malgré de considérables récents développements dans le traitement d’images MR de moelle épinière, il n’y a toujours pas de méthode disponible pouvant fournir une segmentation rigoureuse et fiable de la moelle épinière pour un large spectre de pathologies et de protocoles d’acquisition. Concernant les lésions intramédullaires, une recherche approfondie dans la littérature n’a pas pu fournir une méthode disponible de segmentation automatique. Objectif: Développer un système complètement automatique pour segmenter la moelle épinière et les lésions intramédullaires sur des IRM conventionnelles humaines. Méthode: L’approche présentée est basée de deux réseaux de neurones à convolution mis en cascade. La méthode a été pensée pour faire face aux principaux obstacles que présentent les données IRM de moelle épinière. Le procédé de segmentation a été entrainé et validé sur une base de données privée composée de 1943 images, acquises dans 30 différents centres avec des protocoles hétérogènes. Les sujets scannés comportent 459 sujets sains, 471 patients SEP et 112 avec d’autres pathologies affectant la moelle épinière. Le module de segmentation de la moelle épinière a été comparé à une méthode existante reconnue par la communauté, PropSeg. Résultats: L’approche basée sur les réseaux de neurones à convolution a fourni de meilleurs résultats que PropSeg, atteignant un Dice médian (intervalle inter-quartiles) de 94.6 (4.6) vs. 87.9 (18.3) %. Pour les lésions, notre segmentation automatique a permis d'obtenir un Dice de 60.0 (21.4) % en le comparant à la segmentation manuelle, un ratio de vrai positifs de 83 (34) %, et une précision de 77 (44) %. Conclusion: Une méthode complètement automatique et innovante pour segmenter la moelle épinière et les lésions SEP intramédullaires sur des données IRM a été conçue durant ce projet de maîtrise. La méthode a été abondamment validée sur une base de données clinique. La robustesse de la méthode de segmentation de moelle épinière a été démontrée, même sur des cas pathologiques. Concernant la segmentation des lésions, les résultats sont encourageants, malgré un taux de faux positifs relativement élevé. Je crois en l’impact que peut potentiellement avoir ces outils pour la communauté de chercheurs. Dans cette optique, les méthodes ont été intégrées et documentées dans un logiciel en accès-ouvert, la “Spinal Cord Toolbox”. Certains des outils développés pendant ce projet de Maîtrise sont déjà utilisés par des analyses d’études cliniques, portant sur des patients SEP et sclérose latérale amyotrophique.----------ABSTRACT Context: The spinal cord is a key component of the central nervous system, which contains neurons responsible for complex functions, and ensures the conduction of motor and sensory information between the brain and the peripheral nervous system. Damage to the spinal cord, through trauma or neurodegenerative diseases, can lead to severe impairment, including functional disabilities, paralysis and/or pain. In multiple sclerosis (MS) patients, the spinal cord is frequently affected by atrophy and/or lesions. Conventional magnetic resonance imaging (MRI) is widely used by researchers and clinicians to non-invasively assess and characterize spinal cord microstructural changes. Quantitative assessment of the structural damage to the spinal cord (e.g. atrophy severity, lesion extent) is essential for the diagnosis, prognosis and longitudinal monitoring of diseases, such as MS. Furthermore, the development of objective biomarkers is essential to evaluate the effect of new therapeutic treatments. Spinal cord and intramedullary MS lesions segmentation is consequently clinically relevant, as well as a necessary step towards the interpretation of multi-parametric MR images. However, manual segmentation is highly time-consuming, tedious and prone to intra- and inter-rater variability. There is therefore a need for automated segmentation methods to facilitate the efficiency of analysis pipelines. Automatic lesion segmentation is challenging for various reasons: (i) lesion variability in terms of shape, size and location, (ii) lesion boundaries are most of the time not well defined, (iii) lesion intensities on MR data are confounding with those of normal-appearing structures. Moreover, achieving robust segmentation across multi-center MRI data is challenging because of the broad variability of data features (e.g. resolution, orientation, field of view). Despite recent substantial developments in spinal cord MRI processing, there is still no method available that can yield robust and reliable spinal cord segmentation across the very diverse spinal pathologies and data features. Regarding the intramedullary lesions, a thorough search of the relevant literature did not yield available method of automatic segmentation. Goal: To develop a fully-automatic framework for segmenting the spinal cord and intramedullary MS lesions from conventional human MRI data. Method: The presented approach is based on a cascade of two Convolutional Neural Networks (CNN). The method has been designed to face the main challenges of ‘real world’ spinal cord MRI data. It was trained and validated on a private dataset made up of 1943 MR volumes, acquired in different 30 sites with heterogeneous acquisition protocols. Scanned subjects involve 459 healthy controls, 471 MS patients and 112 with other spinal pathologies. The proposed spinal cord segmentation method was compared to a state-of-the-art spinal cord segmentation method, PropSeg. Results: The CNN-based approach achieved better results than PropSeg, yielding a median (interquartile range) Dice of 94.6 (4.6) vs. 87.9 (18.3) % when compared to the manual segmentation. For the lesion segmentation task, our method provided a median Dice-overlap with the manual segmentation of 60.0 (21.4) %, a lesion-based true positive rate of 83 (34) % and a lesion-based precision de 77 (44) %. Conclusion: An original fully-automatic method to segment the spinal cord and intramedullary MS lesions on MRI data has been devised during this Master’s project. The method was validated extensively against a clinical dataset. The robustness of the spinal cord segmentation has been demonstrated, even on challenging pathological cases. Regarding the lesion segmentation, the results are encouraging despite the fairly high false positive rate. I believe in the potential value of these developed tools for the research community. In this vein, the methods are integrated and documented into an open-source software, the Spinal Cord Toolbox. Some of the tools developed during this Master’s project are already integrated into automated analysis pipelines of clinical studies, including MS and Amyotrophic Lateral Sclerosis patients

Frameworks in medical image analysis with deep neural networks

In recent years, deep neural network based medical image analysis has become quite powerful and achieved similar results performance-wise as experts. Consequently, the integration of these tools into the clinical routine as clinical decision support systems is highly desired. The benefits of automatic image analysis for clinicians are massive, ranging from improved diagnostic as well as treatment quality to increased time-efficiency through automated structured reporting. However, implementations in the literature revealed a significant lack of standardization in pipeline building resulting in low reproducibility, high complexity through extensive knowledge requirements for building state-of-the-art pipelines, and difficulties for application in clinical research. The main objective of this work is the standardization of pipeline building in deep neural network based medical image segmentation and classification. This is why the Python frameworks MIScnn for medical image segmentation and AUCMEDI for medical image classification are proposed which simplify the implementation process through intuitive building blocks eliminating the need for time-consuming and error-prone implementation of common components from scratch. The proposed frameworks include state-of-the-art methodology, follow outstanding open-source principles like extensive documentation as well as stability, offer rapid as well as simple application capabilities for deep learning experts as well as clinical researchers, and provide cutting-edge high-performance competitive with the strongest implementations in the literature. As secondary objectives, this work presents more than a dozen in-house studies as well as discusses various external studies utilizing the proposed frameworks in order to prove the capabilities of standardized medical image analysis. The presented studies demonstrate excellent predictive capabilities in applications ranging from COVID-19 detection in computed tomography scans to the integration into a clinical study workflow for Gleason grading of prostate cancer microscopy sections and advance the state-of-the-art in medical image analysis by simplifying experimentation setups for research. Furthermore, studies for increasing reproducibility in performance assessment of medical image segmentation are presented including an open-source metric library for standardized evaluation and a community guideline on proper metric usage. The proposed contributions in this work improve the knowledge representation of the field, enable rapid as well as high-performing applications, facilitate further research, and strengthen the reproducibility of future studies

Segmentação Automática de Lesões de Esclerose Múltipla em Imagens de Ressonância Magnética

A esclerose múltipla (EM) é o distúrbio neurológico mais comum diagnosticado em jovens adultos com causas inexplicáveis e grandes repercussões na vida dos pacientes, incitando os investigadores na procura ativa de respostas. Embora a doença não possa ser curada ou prevenida, neste momento, os tratamentos disponíveis permitem apenas reduzir a gravidade da mesma e retardar a sua progressão. Torna-se cada vez mais necessário recorrer a técnicas de imagiologia e de processamento e análise de imagem, para ajudar os médicos a realizar um diagnóstico precoce e iniciar o tratamento adequado a fim de proporcionar uma melhor qualidade de vida ao paciente. Várias abordagens baseadas em segmentação automática de lesões de esclerose múltipla tem sido amplamente investigadas nos últimos anos com esse objetivo.Para o desenvolvimento deste projeto, procurou-se por um lado, o reconhecimento das etapas necessárias para implementação e otimização de uma metodologia de processamento e análise de imagem para segmentação automática de lesões de EM, e por outro, a exploração de técnicas de pré-processamento, segmentação e classificação para caracterização objetiva e quantitativa das lesões. Neste trabalho serão ainda abordados conceitos fundamentais sobre a doença de esclerose múltipla e da técnica de ressonância magnética (RM), bem como o estudo bibliográfico de algumas das metodologias atualmente existentes.A metodologia desenvolvida nesta Dissertação teve como base a implementação de diversos algoritmos de pré-processamento para suavização e remoção de ruído, remoção de tecidos não-cerebrais, correção de contraste e normalização de intensidade das imagens. Para segmentação de lesões foi aplicado o estudo de redes neuronais, uma abordagem bastante promissora e atual para o problema proposto, e para classificação foram extraídas e analisadas algumas características das lesões através da sua forma e tamanho. Pretende-se que esta nova metodologia seja flexível e permita o ensaio e a análise dos resultados.Os resultados obtidos demonstram que as técnicas de pré-processamento se revelam essenciais para as etapas subsequentes permitindo uma melhor qualidade da imagem. A segmentação de lesões através do uso de redes neuronais revelou-se apropriada tal como comprovado pelas métricas analisadas, com índice de similaridade estrutural muito próximo de 1, taxa de erro absoluto médio de 3,8% e coeficiente de Dice de 0,58. Por fim, pelas várias aplicações práticas realizadas, foi possível demonstrar a utilidade e adequação das técnicas de processamento e análise de imagem no estudo e deteção de lesões de esclerose múltipla através de imagens de RM.Multiple sclerosis is the most commonly diagnosed neurological disorder in young adults with unexplained causes and major repercussions in the lives of patients, urging researchers to actively search for answers. Although the disease cannot be cured or prevented, the available treatments nowadays reduce its severity and delay its progression. It is becoming increasingly necessary to use imaging techniques and also image processing and analysis techniques, to help doctors perform an early diagnosis and start appropriate treatment in order to provide a better quality of life for the patient. Several approaches based on automatic segmentation of multiple sclerosis lesions have been extensively investigated in recent years for this purpose.This project was developed, firstly, with the recognition of the steps necessary to implement and optimize an image processing and analysis methodology for automatic segmentation of MS lesions, and secondly, by the exploration of pre-processing, segmentation and classification techniques for objective and quantitative characterization of the lesions. This work will also be discussed basic concepts of multiple sclerosis disease and magnetic resonance imaging (MRI), as well as the bibliographical study of some of the currently existing methodologies.The methodology developed in this dissertation was based on the implementation of several pre-processing algorithms for noise smoothing and removal, non-cerebral tissue removal, contrast correction and normalization of images intensity. For lesion segmentation was applied to the study of neural networks, a very promising and current approach to the proposed problem, and to classify were extracted and analyzed some characteristics of the lesions through shape and size. It is intended that this new methodology is flexible and allow the testing and analysis of the results.The results obtained demonstrate that pre-processing techniques are essential to the subsequent steps allowing better image quality. Segmentation of lesions through the use of neural networks proved to be appropriate for this study as shown by the metrics analyzed, with a structural similarity index very close to 1, mean absolute error rate of 3.8% and Dice coefficient of 0.58. Finally, the various practical applications performed was possible to demonstrate the usefulness and adequacy of image processing and analysis techniques in the study and detection of multiple sclerosis lesions through MR images