Medical imaging analysis with artificial neural networks

Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging

Special issue on signal processing and machine learning for biomedical data

This Special Issue is focused on advanced techniques in signal processing, analysis, modelling, and classification, applied to a variety of medical diagnostic problems. Biomedical data play a fundamental role in many fields of research and clinical practice. Very often the complexity of these data and their large volume makes it necessary to develop advanced analysis techniques and systems. Furthermore, the introduction of new techniques and methodologies for diagnostic purposes, especially in the field of medical imaging, requires new signal processing and machine learning methods. The recent progress in machine learning techniques, and in particular deep learning, revolutionized various fields of artificial vision, significantly pushing the state of the art of artificial vision systems into a wide range of high-level tasks. Such progress can help address problems in the analysis of biomedical data.This Special Issue placed particular emphasis on contributions dealing with practical, applications-led research, on the use of methods and devices in clinical diagnosis. The works that make up this special issue show a remarkable variety of applications for the detection and classification of medical imaging problems. In particular, the aforementioned works can be divided on the basis of types of techniques used, into three categories—signal processing (SP) methods, traditional machine learning (ML) methods, and deep learning (DL) methods

Microcalcifications Detection Using Image And Signal Processing Techniques For Early Detection Of Breast Cancer

Breast cancer has transformed into a severe health problem around the world. Early diagnosis is an important factor to survive this disease. The earliest detection signs of potential breast cancer that is distinguishable by current screening techniques are the presence of microcalcifications (MCs). MCs are small crystals of calcium apatite and their normal size ranges from 0.1mm to 0.5mm single crystals to groups up to a few centimeters in diameter. They are the first indication of breast cancer in more than 40% of all breast cancer cases, making their diagnosis critical. This dissertation proposes several segmentation techniques for detecting and isolating point microcalcifications: Otsu’s Method, Balanced Histogram Thresholding, Iterative Method, Maximum Entropy, Moment Preserving, and Genetic Algorithm. These methods were applied to medical images to detect microcalcifications. In this dissertation, results from the application of these techniques are presented and their efficiency for early detection of breast cancer is explained. This dissertation also explains theories and algorithms related to these techniques that can be used for breast cancer detection

Cluster analysis of the signal curves in perfusion DCE-MRI datasets

Pathological studies show that tumors consist of different sub-regions with more homogeneous vascular properties during their growth. In addition, destroying tumor's blood supply is the target of most cancer therapies. Finding the sub-regions in the tissue of interest with similar perfusion patterns provides us with valuable information about tissue structure and angiogenesis. This information on cancer therapy, for example, can be used in monitoring the response of the cancer treatment to the drug. Cluster analysis of perfusion curves assays to find sub-regions with a similar perfusion pattern. The present work focuses on the cluster analysis of perfusion curves, measured by dynamic contrast enhanced magnetic resonance imaging (DCE-MRI). The study, besides searching for the proper clustering method, follows two other major topics, the choice of an appropriate similarity measure, and determining the number of clusters. These three subjects are connected to each other in such a way that success in one direction will help solving the other problems. This work introduces a new similarity measure, parallelism measure (PM), for comparing the parallelism in the washout phase of the signal curves. Most of the previous works used the Euclidean distance as the measure of dissimilarity. However, the Euclidean distance does not take the patterns of the signal curves into account and therefore for comparing the signal curves is not sufficient. To combine the advantages of both measures a two-steps clustering is developed. The two-steps clustering uses two different similarity measures, the introduced PM measure and Euclidean distance in two consecutive steps. The results of two-steps clustering are compared with the results of other clustering methods. The two-steps clustering besides good performance has some other advantages. The granularity and the number of clusters are controlled by thresholds defined by considering the noise in signal curves. The method is easy to implement and is robust against noise. The focus of the work is mainly the cluster analysis of breast tumors in DCE-MRI datasets. The possibility to adopt the method for liver datasets is studied as well

H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological images

Over the past decades, histopathological cancer diagnostics has become more complex, and the increasing number of biopsies is a challenge for most pathology laboratories. Thus, development of automatic methods for evaluation of histopathological cancer sections would be of value. In this study, we used 624 whole slide images (WSIs) of breast cancer from a Norwegian cohort. We propose a cascaded convolutional neural network design, called H2G-Net, for segmentation of breast cancer region from gigapixel histopathological images. The design involves a detection stage using a patch-wise method, and a refinement stage using a convolutional autoencoder. To validate the design, we conducted an ablation study to assess the impact of selected components in the pipeline on tumor segmentation. Guiding segmentation, using hierarchical sampling and deep heatmap refinement, proved to be beneficial when segmenting the histopathological images. We found a significant improvement when using a refinement network for post-processing the generated tumor segmentation heatmaps. The overall best design achieved a Dice similarity coefficient of 0.933±0.069 on an independent test set of 90 WSIs. The design outperformed single-resolution approaches, such as cluster-guided, patch-wise high-resolution classification using MobileNetV2 (0.872±0.092) and a low-resolution U-Net (0.874±0.128). In addition, the design performed consistently on WSIs across all histological grades and segmentation on a representative × 400 WSI took ~ 58 s, using only the central processing unit. The findings demonstrate the potential of utilizing a refinement network to improve patch-wise predictions. The solution is efficient and does not require overlapping patch inference or ensembling. Furthermore, we showed that deep neural networks can be trained using a random sampling scheme that balances on multiple different labels simultaneously, without the need of storing patches on disk. Future work should involve more efficient patch generation and sampling, as well as improved clustering

H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological images

Over the past decades, histopathological cancer diagnostics has become more complex, and the increasing number of biopsies is a challenge for most pathology laboratories. Thus, development of automatic methods for evaluation of histopathological cancer sections would be of value. In this study, we used 624 whole slide images (WSIs) of breast cancer from a Norwegian cohort. We propose a cascaded convolutional neural network design, called H2G-Net, for segmentation of breast cancer region from gigapixel histopathological images. The design involves a detection stage using a patch-wise method, and a refinement stage using a convolutional autoencoder. To validate the design, we conducted an ablation study to assess the impact of selected components in the pipeline on tumor segmentation. Guiding segmentation, using hierarchical sampling and deep heatmap refinement, proved to be beneficial when segmenting the histopathological images. We found a significant improvement when using a refinement network for post-processing the generated tumor segmentation heatmaps. The overall best design achieved a Dice similarity coefficient of 0.933±0.069 on an independent test set of 90 WSIs. The design outperformed single-resolution approaches, such as cluster-guided, patch-wise high-resolution classification using MobileNetV2 (0.872±0.092) and a low-resolution U-Net (0.874±0.128). In addition, the design performed consistently on WSIs across all histological grades and segmentation on a representative × 400 WSI took ~ 58 s, using only the central processing unit. The findings demonstrate the potential of utilizing a refinement network to improve patch-wise predictions. The solution is efficient and does not require overlapping patch inference or ensembling. Furthermore, we showed that deep neural networks can be trained using a random sampling scheme that balances on multiple different labels simultaneously, without the need of storing patches on disk. Future work should involve more efficient patch generation and sampling, as well as improved clustering.publishedVersio

Modelling the head and neck region for microwave imaging of cervical lymph nodes

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Radiações em Diagnóstico e Terapia), Universidade de Lisboa, Faculdade de Ciências, 2020O termo “cancro da cabeça e pescoço” refere-se a um qualquer tipo de cancro com início nas células epiteliais das cavidades oral e nasal, seios perinasais, glândulas salivares, faringe e laringe. Estes tumores malignos apresentaram, em 2018, uma incidência mundial de cerca de 887.659 novos casos e taxa de mortalidade superior a 51%. Aproximadamente 80% dos novos casos diagnosticados nesse ano revelaram a proliferação de células cancerígenas dos tumores para outras regiões do corpo através dos vasos sanguíneos e linfáticos das redondezas. De forma a determinar o estado de desenvolvimento do cancro e as terapias a serem seguidas, é fundamental a avaliação dos primeiros gânglios linfáticos que recebem a drenagem do tumor primário – os gânglios sentinela – e que, por isso, apresentam maior probabilidade de se tornarem os primeiros alvos das células tumorais. Gânglios sentinela saudáveis implicam uma menor probabilidade de surgirem metástases, isto é, novos focos tumorais decorrentes da disseminação do cancro para outros órgãos. O procedimento standard que permite o diagnóstico dos gânglios linfáticos cervicais, gânglios que se encontram na região da cabeça e pescoço, e o estadiamento do cancro consiste na remoção cirúrgica destes gânglios e subsequente histopatologia. Para além de ser um procedimento invasivo, a excisão cirúrgica dos gânglios linfáticos representa perigos tanto para a saúde mental e física dos pacientes, como para a sua qualidade de vida. Dores, aparência física deformada (devido a cicatrizes), perda da fala ou da capacidade de deglutição são algumas das repercussões que poderão advir da remoção de gânglios linfáticos da região da cabeça e pescoço. Adicionalmente, o risco de infeção e linfedema – acumulação de linfa nos tecidos intersticiais – aumenta significativamente com a remoção de uma grande quantidade de gânglios linfáticos saudáveis. Também os encargos para os sistemas de saúde são elevados devido à necessidade de monitorização destes pacientes e subsequentes terapias e cuidados associados à morbilidade, como é o caso da drenagem linfática manual e da fisioterapia. O desenvolvimento de novas tecnologias de imagem da cabeça e pescoço requer o uso de modelos realistas que simulem o comportamento e propriedades dos tecidos biológicos. A imagem médica por micro-ondas é uma técnica promissora e não invasiva que utiliza radiação não ionizante, isto é, sinais com frequências na gama das micro-ondas cujo comportamento depende do contraste dielétrico entre os diferentes tecidos atravessados, pelo que é possível identificar regiões ou estruturas de interesse e, consequentemente, complementar o diagnóstico. No entanto, devido às suas características, este tipo de modalidade apenas poderá ser utilizado para a avaliação de regiões anatómicas pouco profundas. Estudos indicam que os gânglios linfáticos com células tumorais possuem propriedades dielétricas distintas dos gânglios linfáticos saudáveis. Por esta razão e juntamente pelo facto da sua localização pouco profunda, consideramos que os gânglios linfáticos da região da cabeça e pescoço constituem um excelente candidato para a utilização de imagem médica por radar na frequência das micro-ondas como ferramenta de diagnóstico. Até à data, não foram efetuados estudos de desenvolvimento de modelos da região da cabeça e pescoço focados em representar realisticamente os gânglios linfáticos cervicais. Por este motivo, este projeto consistiu no desenvolvimento de dois geradores de fantomas tridimensionais da região da cabeça e pescoço – um gerador de fantomas numéricos simples (gerador I) e um gerador de fantomas numéricos mais complexos e anatomicamente realistas, que foi derivado de imagens de ressonância magnética e que inclui as propriedades dielétricas realistas dos tecidos biológicos (gerador II). Ambos os geradores permitem obter fantomas com diferentes níveis de complexidade e assim acompanhar diferentes fases no processo de desenvolvimento de equipamentos médicos de imagiologia por micro-ondas. Todos os fantomas gerados, e principalmente os fantomas anatomicamente realistas, poderão ser mais tarde impressos a três dimensões. O processo de construção do gerador I compreendeu a modelação da região da cabeça e pescoço em concordância com a anatomia humana e distribuição dos principais tecidos, e a criação de uma interface para a personalização dos modelos (por exemplo, a inclusão ou remoção de alguns tecidos é dependente do propósito para o qual cada modelo é gerado). O estudo minucioso desta região levou à inclusão de tecidos ósseos, musculares e adiposos, pele e gânglios linfáticos nos modelos. Apesar destes fantomas serem bastante simples, são essenciais para o início do processo de desenvolvimento de dispositivos de imagem médica por micro-ondas dedicados ao diagnóstico dos gânglios linfáticos cervicais. O processo de construção do gerador II foi fracionado em 3 grandes etapas devido ao seu elevado grau de complexidade. A primeira etapa consistiu na criação de uma pipeline que permitiu o processamento das imagens de ressonância magnética. Esta pipeline incluiu: a normalização dos dados, a subtração do background com recurso a máscaras binárias manualmente construídas, o tratamento das imagens através do uso de filtros lineares (como por exemplo, filtros passa-baixo ideal, Gaussiano e Butterworth) e não-lineares (por exemplo, o filtro mediana), e o uso de algoritmos não supervisionados de machine learning para a segmentação dos vários tecidos biológicos presentes na região cervical, tais como o K-means, Agglomerative Hierarchical Clustering, DBSCAN e BIRCH. Visto que cada algoritmo não supervisionado de machine learning anteriormente referido requer diferentes hiperparâmetros, é necessário proceder a um estudo pormenorizado que permita a compreensão do modo de funcionamento de cada algoritmo individualmente e a sua interação / performance com o tipo de dados tratados neste projeto (isto é, dados de exames de ressonâncias magnéticas) com vista a escolher empiricamente o leque de valores de cada hiperparâmetro que deve ser considerado, e ainda as combinações que devem ser testadas. Após esta fase, segue-se a avaliação da combinação de hiperparâmetros que resulta na melhor segmentação das estruturas anatómicas. Para esta avaliação são consideradas duas metodologias que foram combinadas: a utilização de métricas que permitam avaliar a qualidade do clustering (como por exemplo, o Silhoeutte Coefficient, o índice de Davies-Bouldin e o índice de Calinski-Harabasz) e ainda a inspeção visual. A segunda etapa foi dedicada à introdução manual de algumas estruturas, como a pele e os gânglios linfáticos, que não foram segmentadas pelos algoritmos de machine learning devido à sua fina espessura e pequena dimensão, respetivamente. Finalmente, a última etapa consistiu na atribuição das propriedades dielétricas, para uma frequência pré-definida, aos tecidos biológicos através do Modelo de Cole-Cole de quatro pólos. Tal como no gerador I, foi criada uma interface que permitiu ao utilizador decidir que características pretende incluir no fantoma, tais como: os tecidos a incluir (tecido adiposo, tecido muscular, pele e / ou gânglios linfáticos), relativamente aos gânglios linfáticos o utilizador poderá ainda determinar o seu número, dimensões, localização em níveis e estado clínico (saudável ou metastizado) e finalmente, o valor de frequência para o qual pretende obter as propriedades dielétricas (permitividade relativa e condutividade) de cada tecido biológico. Este projeto resultou no desenvolvimento de um gerador de modelos realistas da região da cabeça e pescoço com foco nos gânglios linfáticos cervicais, que permite a inserção de tecidos biológicos, tais como o tecidos muscular e adiposo, pele e gânglios linfáticos e aos quais atribui as propriedades dielétricas para uma determinada frequência na gama de micro-ondas. Estes modelos computacionais resultantes do gerador II, e que poderão ser mais tarde impressos em 3D, podem vir a ter grande impacto no processo de desenvolvimento de dispositivos médicos de imagem por micro-ondas que visam diagnosticar gânglios linfáticos cervicais, e consequentemente, contribuir para um processo não invasivo de estadiamento do cancro da cabeça e pescoço.Head and neck cancer is a broad term referring to any epithelial malignancies arising in the paranasal sinuses, nasal and oral cavities, salivary glands, pharynx, and larynx. In 2018, approximately 80% of the newly diagnosed head and neck cancer cases resulted in tumour cells spreading to neighbouring lymph and blood vessels. In order to determine cancer staging and decide which follow-up exams and therapy to follow, physicians excise and assess the Lymph Nodes (LNs) closest to the primary site of the head and neck tumour – the sentinel nodes – which are the ones with highest probability of being targeted by cancer cells. The standard procedure to diagnose the Cervical Lymph Nodes (CLNs), i.e. lymph nodes within the head and neck region, and determine the cancer staging frequently involves their surgical removal and subsequent histopathology. Besides being invasive, the removal of the lymph nodes also has negative impact on patients’ quality of life, it can be health threatening, and it is costly to healthcare systems due to the patients’ needs for follow-up treatments/cares. Anatomically realistic phantoms are required to develop novel technologies tailored to image head and neck regions. Medical MicroWave Imaging (MWI) is a promising non-invasive approach which uses non-ionizing radiation to screen shallow body regions, therefore cervical lymph nodes are excellent candidates to this imaging modality. In this project, a three-dimensional (3D) numerical phantom generator (generator I) and a Magnetic Resonance Imaging (MRI)-derived anthropomorphic phantom generator (generator II) of the head and neck region were developed to create phantoms with different levels of complexity and realism, which can be later 3D printed to test medical MWI devices. The process of designing the numerical phantom generator included the modelling of the head and neck regions according to their anatomy and the distribution of their main tissues, and the creation of an interface which allowed the users to personalise the model (e.g. include or remove certain tissues, depending on the purpose of each generated model). To build the anthropomorphic phantom generator, the modelling process included the creation of a pipeline of data processing steps to be applied to MRIs of the head and neck, followed by the development of algorithms to introduce additional tissues to the models, such as skin and lymph nodes, and finally, the assignment of the dielectric properties to the biological tissues. Similarly, this generator allowed users to decide the features they wish to include in the phantoms. This project resulted in the creation of a generator of 3D anatomically realistic head and neck phantoms which allows the inclusion of biological tissues such as skin, muscle tissue, adipose tissue, and LNs, and assigns state-of-the-art dielectric properties to the tissues. These phantoms may have a great impact in the development process of MWI devices aimed at screening and diagnosing CLNs, and consequently, contribute to a non-invasive staging of the head and neck cancer

Computational Modeling for Abnormal Brain Tissue Segmentation, Brain Tumor Tracking, and Grading

This dissertation proposes novel texture feature-based computational models for quantitative analysis of abnormal tissues in two neurological disorders: brain tumor and stroke. Brain tumors are the cells with uncontrolled growth in the brain tissues and one of the major causes of death due to cancer. On the other hand, brain strokes occur due to the sudden interruption of the blood supply which damages the normal brain tissues and frequently causes death or persistent disability. Clinical management of these brain tumors and stroke lesions critically depends on robust quantitative analysis using different imaging modalities including Magnetic Resonance (MR) and Digital Pathology (DP) images. Due to uncontrolled growth and infiltration into the surrounding tissues, the tumor regions appear with a significant texture variation in the static MRI volume and also in the longitudinal imaging study. Consequently, this study developed computational models using novel texture features to segment abnormal brain tissues (tumor, and stroke lesions), tracking the change of tumor volume in longitudinal images, and tumor grading in MR images. Manual delineation and analysis of these abnormal tissues in large scale is tedious, error-prone, and often suffers from inter-observer variability. Therefore, efficient computational models for robust segmentation of different abnormal tissues is required to support the diagnosis and analysis processes. In this study, brain tissues are characterized with novel computational modeling of multi-fractal texture features for multi-class brain tumor tissue segmentation (BTS) and extend the method for ischemic stroke lesions in MRI. The robustness of the proposed segmentation methods is evaluated using a huge amount of private and public domain clinical data that offers competitive performance when compared with that of the state-of-the-art methods. Further, I analyze the dynamic texture behavior of tumor volume in longitudinal imaging and develop post-processing frame-work using three-dimensional (3D) texture features. These post-processing methods are shown to reduce the false positives in the BTS results and improve the overall segmentation result in longitudinal imaging. Furthermore, using this improved segmentation results the change of tumor volume has been quantified in three types such as stable, progress, and shrinkage as observed by the volumetric changes of different tumor tissues in longitudinal images. This study also investigates a novel non-invasive glioma grading, for the first time in literature, that uses structural MRI only. Such non-invasive glioma grading may be useful before an invasive biopsy is recommended. This study further developed an automatic glioma grading scheme using the invasive cell nuclei morphology in DP images for cross-validation with the same patients. In summary, the texture-based computational models proposed in this study are expected to facilitate the clinical management of patients with the brain tumors and strokes by automating large scale imaging data analysis, reducing human error, inter-observer variability, and producing repeatable brain tumor quantitation and grading