A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Early detection of lung cancer - A challenge

Lung cancer or lung carcinoma, is a common and serious type of cancer caused by rapid cell growth in tissues of the lung. Lung cancer detection at its earlier stage is very difficult because of the structure of the cell alignment which makes it very challenging. Computed tomography (CT) scan is used to detect the presence of cancer and its spread. Visual analysis of CT scan can lead to late treatment of cancer; therefore, different steps of image processing can be used to solve this issue. A comprehensive framework is used for the classification of pulmonary nodules by combining appearance and shape feature descriptors, which helps in the early diagnosis of lung cancer. 3D Histogram of Oriented Gradient (HOG), Resolved Ambiguity Local Binary Pattern (RALBP) and Higher Order Markov Gibbs Random Field (MGRF) are the feature descriptors used to explain the nodule’s appearance and compared their performance. Lung cancer screening methods, image processing techniques and nodule classification using radiomic-based framework are discussed in this paper which proves to be very effective in lung cancer prediction. Good performance is shown by using RALBP descriptor

An Intelligent Radiomic Approach for Lung Cancer Screening

Funding: This project is supported by the Ministerio de Ciencia e Innovación (MCI), Agencia Estatal de Investigación (AEI) and Fondo Europeo de Desarrollo Regional (FEDER), RTI2018-095209-B-C21 (MCI/AEI/FEDER, UE), Generalitat de Catalunya, 2017-SGR-1624 and CERCA-Programme. Debora Gil is supported by Serra Hunter Fellow.This project is supported by the Ministerio de Ciencia e Innovaci?n (MCI), Agencia Estatal de Investigaci?n (AEI) and Fondo Europeo de Desarrollo Regional (FEDER), RTI2018-095209-B-C21 (MCI/AEI/FEDER, UE), Generalitat de Catalunya, 2017-SGR-1624 and CERCA-Programme. Debora Gil is supported by Serra Hunter Fellow. Barcelona Respiratory Network (BRN), Acad?mia de Ci?ncies M?diques de Catalunya i Balears, i Fundaci? Ramon Pla i Armengol.The efficiency of lung cancer screening for reducing mortality is hindered by the high rate of false positives. Artificial intelligence applied to radiomics could help to early discard benign cases from the analysis of CT scans. The available amount of data and the fact that benign cases are a minority, constitutes a main challenge for the successful use of state of the art methods (like deep learning), which can be biased, over-fitted and lack of clinical reproducibility. We present an hybrid approach combining the potential of radiomic features to characterize nodules in CT scans and the generalization of the feed forward networks. In order to obtain maximal reproducibility with minimal training data, we propose an embedding of nodules based on the statistical significance of radiomic features for malignancy detection. This representation space of lesions is the input to a feed forward network, which architecture and hyperparameters are optimized using own-defined metrics of the diagnostic power of the whole system. Results of the best model on an independent set of patients achieve 100% of sensitivity and 83% of specificity (AUC = 0.94) for malignancy detection

Texture Analysis Platform for Imaging Biomarker Research

abstract: The rate of progress in improving survival of patients with solid tumors is slow due to late stage diagnosis and poor tumor characterization processes that fail to effectively reflect the nature of tumor before treatment or the subsequent change in its dynamics because of treatment. Further advancement of targeted therapies relies on advancements in biomarker research. In the context of solid tumors, bio-specimen samples such as biopsies serve as the main source of biomarkers used in the treatment and monitoring of cancer, even though biopsy samples are susceptible to sampling error and more importantly, are local and offer a narrow temporal scope. Because of its established role in cancer care and its non-invasive nature imaging offers the potential to complement the findings of cancer biology. Over the past decade, a compelling body of literature has emerged suggesting a more pivotal role for imaging in the diagnosis, prognosis, and monitoring of diseases. These advances have facilitated the rise of an emerging practice known as Radiomics: the extraction and analysis of large numbers of quantitative features from medical images to improve disease characterization and prediction of outcome. It has been suggested that radiomics can contribute to biomarker discovery by detecting imaging traits that are complementary or interchangeable with other markers. This thesis seeks further advancement of imaging biomarker discovery. This research unfolds over two aims: I) developing a comprehensive methodological pipeline for converting diagnostic imaging data into mineable sources of information, and II) investigating the utility of imaging data in clinical diagnostic applications. Four validation studies were conducted using the radiomics pipeline developed in aim I. These studies had the following goals: (1 distinguishing between benign and malignant head and neck lesions (2) differentiating benign and malignant breast cancers, (3) predicting the status of Human Papillomavirus in head and neck cancers, and (4) predicting neuropsychological performances as they relate to Alzheimer’s disease progression. The long-term objective of this thesis is to improve patient outcome and survival by facilitating incorporation of routine care imaging data into decision making processes.Dissertation/ThesisDoctoral Dissertation Biomedical Informatics 201

Radiomics and prostate MRI: Current role and future applications

Multiparametric prostate magnetic resonance imaging (mpMRI) is widely used as a triage test for men at a risk of prostate cancer. However, the traditional role of mpMRI was confined to prostate cancer staging. Radiomics is the quantitative extraction and analysis of minable data from medical images; it is emerging as a promising tool to detect and categorize prostate lesions. In this paper we review the role of radiomics applied to prostate mpMRI in detection and localization of prostate cancer, prediction of Gleason score and PI-RADS classification, prediction of extracapsular extension and of biochemical recurrence. We also provide a future perspective of artificial intelligence (machine learning and deep learning) applied to the field of prostate cancer

Quantitative image analysis of peripheral nerves in whiplash injury patients

The research in this thesis has examined the use of texture and shape analysis to characterise Magnetic Resonance (MR) images of peripheral nerves in order to provide a potential quantitative tool for better diagnosis and treatments. Texture and shape can be considered as inherent properties of all surfaces and have the potential to provide sensitive information which cannot be quantitatively perceived by human vision. Texture analysis has been successfully used in image classification of aerial and satellite imagery and the diagnosis and prognosis of several types of cancer. However, to date, it has never been used in investigating peripheral nerve damage. In this thesis, we study the application of texture and shape analysis to the peripheral nerves in the upper extremities of patients suffering from Whiplash Associated Disorders (WAD). Specifically, quantitative texture analysis was performed on MR images of the carpal tunnel which contains the median nerve. The median nerve was studied to identify differences in textural patterns. Texture methods such as: first order features; co-occurrence matrices; run-length matrices and autocorrelation function were applied and their performance was assessed. Texture analysis was also performed to investigate nerve damage in the MR images of the brachial plexus, both in controls and patients. Further, spatial domain shape metrics were used to quantify and study the morphological differences of the median nerve in controls and patients. This highlighted that some significant differences exist between groups and thus could potentially be reliably used in combination with clinical scale metrics to identify possible nerve damage. As MR images contain noise, locating the median nerve accurately to perform image analysis is very important. Therefore, we further investigated the application of an enhanced correlation filtering method that could be trained on images of the median nerve and then applied to detect the median nerve in test images. The Optimal Trade-off Maximum Average Correlation Height (OT-MACH) filter includes the expected distortions in the target in the construction of the filter reference function. The OT-MACH filter was tuned in a bandpass to maximize the correlation peak and thereby successfully locate the position of the median nerve in the carpal tunnel. This study has successfully demonstrated that texture and shape analysis can be used to investigate possible peripheral nerve damage. Further research is required using larger datasets to establish a quantitative image analysis tool to support clinical decision making and thereby improve patient care and treatment outcome

Identification and quantification of the alveolar compartment by confocal laser endomicroscopy in patients with interstitial lung diseases

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Biofísica Médica e Fisiologia de Sistemas), Universidade de Lisboa, Faculdade de Ciências, 2018Doenças Intersticiais Pulmonares (DIP) é um termo que inclui mais de 200 doenças que afectam o parênquima pulmonar, partilhando manifestações clínicas, radiográficas e patológicas semelhantes. Este conjunto de doenças é bastante heterogéneo, apresentando cada tipo de DIP em diferente grau os elementos de inflamação e fibrose: enquanto a inflamação é reflectida pelo aumento de células inflamatórias e presença de nódulos ou edema, a fibrose reflecte-se pelas fibras adicionais de colagénio e elastina. Identificar o tipo de DIP de um doente é um processo difícil, sendo a Discussão Multidisciplinar o actual método de diagnóstico "gold standard": vários médicos especialistas compõem uma equipa multidisciplinar que vai ter em conta os dados clínicos, radiológicos e patológicos disponíveis para chegar a uma conclusão. Estes dados incluem imagens de tomografia computorizada de alta resolução (TCAR), a descrição da lavagem broncoalveolar e, quando possível, dados de biópsias. Apesar do esforço e competência da equipa multidisciplinar, 10% dos pacientes são categorizados como inclassificáveis devido a dados inadequados ou discrepância entre os dados existentes. A maior causa para DIP inclassificáveis é a ausência de dados histopatológicos associada aos riscos das biópsias cirúrgicas. É muito importante determinar a DIP específica de um doente, dadas as suas implicações no tratamento e gestão do mesmo. É particularmente crítica a distinção entre doentes com Fibrose Pulmonar Idiopática (FPI) e doentes sem FPI, dado que há terapias anti-fibróticas – como o Pirfenidone – indicadas para FPI que são extremamente dispendiosas, exigindo certeza no diagnóstico antes de serem prescritas. Além disso, o tratamento com agentes imunossupressores pode funcionar com o grupo dos não-FPI mas aumenta a morte e hospitalizações nos doentes com FPI. A discussão multidisciplinar pode beneficiar da informação adicional oferecida pelo Confocal Laser Endomicroscopy (CLE), uma técnica de imagiologia que torna possível visualizar os alvéolos pulmonares com resolução microscópica de forma minimamente invasiva, através de uma broncoscopia. O laser do CLE tem um comprimento de onda de 488 nm que permite observar a autofluorescência das fibras de elastina. Há evidências de que a quantidade de fibras de elastina é aumentada e a arquitectura destas fibras é alterada na presença de fibrose pulmonar, a qual está associada a algumas doenças intersticiais pulmonares incluindo a fibrose pulmonar idiopática. Até à data, os vídeos de Confocal Laser Endomicroscopy são, na maioria dos casos, analisados apenas visualmente, e pouca informação objectiva e consistente foi conseguida destes vídeos em doentes de DIP. No entanto, é possível obter informação mais relevante dos mesmos, convertendo-os em frames, pré-processando as imagens e extraindo atributos numéricos. Neste projecto, foram obtidas imagens dos alvéolos pulmonares de doentes de DIP através de CLE. O principal objectivo do projecto é melhorar a técnica de CLE e aumentar a sua usabilidade para que no futuro possa contribuir para facilitar a estratificação de doentes com DIP e eventualmente reduzir o número de biópsias pulmonares nestes doentes. Como mencionado, o instrumento de Confocal Laser Endomicroscopy emite uma luz laser azul de 488nm, a qual é reflectida no tecido e reorientada para o sistema de detecção pela mesma lente, passando por um pequeno orifício (pinhole). Isto permite que a luz focada seja recolhida e que feixes provenientes de planos fora de foco sejam excluídos, originando uma resolução microscópica que permite imagens ao nível celular. Quando o CLE é aplicado a imagem pulmonar, é possível observar as paredes alveolares pela autofluorescência natural presente nas fibras de elastina. No estudo clínico subjacente a este estudo, o protocolo de CLE foi aplicado a 20 pacientes, embora 8 tenham sido posteriormente excluídos da análise. Os vídeos de CLE obtidos sofreram duas selecções: uma com base na região onde uma biópsia (usada como referência) foi tirada e outra com base na qualidade técnica das imagens. Depois, os dados foram pré-processados: geraram-se imagens mosaico com um campo de visão alargado e, paralelamente converteram-se as sequências de vídeo em frames. A qualidade da imagem foi melhorada, filtrando o ruído electrónico para que posteriormente pudesse ser aplicada a análise de imagem. Esta análise extraiu valores numéricos que reflectem o estado do espaço alveolar, nomeadamente, variáveis de textura e medições relacionadas com as fibras de elastina. As imagens de CLE obtidas mostraram-se muito interessantes. A resolução é superior à tomografia computorizada de alta resolução e a tridimensionalidade acrescenta informação às biópsias. O facto de permitir feedback em tempo real e observar ao vivo os movimentos naturais da respiração contribui para a análise do estado do doente. A análise de textura feita às imagens serviu-se de um algoritmo de extracção de variáveis de Haralick a partir de uma Gray-Level Co-occurence Matrix (GLCM). Foram extraídas as variáveis de textura Momento Angular Secundário (Energia), Entropia, Momento de Diferença Inversa, Contraste, Variação e Correlação. O algoritmo de Ridge Detection (detecção de linhas) identificou a maior parte das fibras de elastina detectáveis por um observador humano e mediu o Número de Fibras, o seu Comprimento e Largura e o Número de Junções entre fibras, permitindo também calcular a Soma dos Comprimentos de todas as fibras. Estes algoritmos devolveram valores consistentes num processo mais eficiente comparado com um observador humano, conseguindo avaliar em poucos segundos múltiplas variáveis para todo o conjunto de dados. As medições relacionadas com as fibras de elastina pretendiam ajudar a identificar os doentes fibróticos. Era esperado que as fibras dos doentes fibróticos fossem mais largas, mas isso não se observou. Também se previa que este grupo de doentes apresentasse maior número de fibras e junções, mas não houve uma diferença significativa entre grupos. No entanto, quando o grupo fibrótico foi segregado, o número de fibras e junções parece separar a fibrose moderada da fibrose severa. Este resultado é interessante na medida em que sugere que a monitorização do número de fibras/junções com CLE pode potencialmente ser usado como medida de eficácia de medicação anti-fibrótica. Em relação às variáveis de textura, esperava-se que os doentes fibróticos apresentassem valores mais elevados de Entropia, Contraste e Variância e valores inferiores de Momento de Diferença Inversa, dado que o seu tecido pulmonar deveria corresponder a imagens mais complexas e heterogéneas com mais arestas presentes. No entanto, ainda não foi possível estabelecer diferenças significativas entre grupos. Apesar dos resultados com o conjunto de dados usado não ter demonstrado correlações fortes entre as conclusões do CLE e da TCAR/histopatologia, os valores das variáveis em si já contribuem para o estudo das DIP, nomeadamente da sua fisiologia. De facto, a amostra de doentes deste estudo era reduzida, mas com uma amostra maior, espera-se que algumas das varáveis se correlacionem com outras técnicas usadas no diagnóstico e permitam segregar os pacientes em grupos e eventualmente aplicar classificação de dados. Neste momento, é possível especular que algumas variáveis seriam melhores candidatas para um classificador, nomeadamente os Números de Fibras e Junções, a Soma dos Comprimentos das fibras e as variáveis de Haralick Entropia e Energia. O projecto apresentado nesta dissertação foi desenvolvido através de um estágio de 6 meses no departamento de Pneumologia no Academic Medical Center em Amsterdão, Países Baixos. No Academic Medical Center (AMC), fui acompanhada pelos estudantes de doutoramento Lizzy Wijmans - médica - e Paul Brinkman - engenheiro biomédico - e supervisionada pelo Dr. Jouke Annema, MD, PhD, Professor de endoscopia pulmonar. Este grupo de investigação do AMC está focado em técnicas inovadoras de imagiologia do sistema pulmonar e teve a oportunidade de reunir com a empresa MKT –que produz a tecnologia de Confocal Laser Endomicroscopy –, o que enriqueceu a discussão aqui apresentada. Do Departamento de Física da Faculdade de Ciências da Universidade de Lisboa, fui orientada pelo Prof. Nuno Matela.Interstitial Lung Diseases (ILD) is a heterogeneous group of more than 200 diseases which affect the lung parenchyma. To identify the type of ILD a patient suffers from is a difficult process, and 10% of the patients are categorized as unclassifiable, mostly due to the absence of histopathological data associated with the risks of lung biopsies. The patient specific diagnosis is important because of its implications to the patient treatment and management, being particularly relevant to identify lung fibrosis. The Confocal Laser Endomicroscopy (CLE) can add information to this process. CLE allows to image the lung tissue with a micrometer resolution in a minimally invasive way, through a bronchoscopy. The elastin fibers from the lung alveoli are visible with this technique due to their autofluorescence. Since there is evidence that the amount of elastin fibers increases, and their architecture is altered in lung fibrosis, CLE should be used to extract values reflecting this condition. Thus, the main goal of this project was to improve the CLE technique and increase its usability, by extracting numerical values from the images which would reflect the state of the alveolar space, particularly the elastin fibers. The ILD patients recruited for the study had their lung alveoli imaged with CLE. The CLE movies were selected, pre-processed – were converted into frames, had their image quality enhanced and some mosaics were obtained – and then analyzed. The ridge detection algorithm detected most fibers recognized by a human observer. It allowed the measurement of the Number of Detected Fibers, their Length and Width, the Number of Junctions between fibers and to calculate the Sum from all Fibers’ Lengths. The Gray-Level Co-occurrence Matrix allowed the extraction of the Haralick texture features: Angular Second Moment (Energy), Entropy, Inverse Difference Moment, Contrast, Variance and Correlation. These algorithms produced consistent and unbiased numerical features, in an efficient process which can analyze the entire data set in a few seconds. Regarding the fiber related measurements, it was expected for the fibrotic patients to have wider fibers and a higher number of fibers and junctions. In terms of texture variables, it was expected from the fibrotic patients to present higher values of Entropy, Contrast and Variance, and lower values of Inverse Difference Moment, given their lung tissue should correspond to more complex and heterogeneous images with more ridges present. Due to the small sample size, it was still not possible to stratify patients with this data set. Nevertheless, the measurements presented here already contribute to the study of ILD, helping to understand the disease physiology. It is hoped that in the future, these measurements will aid the diagnosis process specially in those cases when patients cannot undergo a surgical biopsy. Additionally, CLE could potentially be used as an anti-fibrotic medication efficiency measurement tool