Brain Tumor Segmentation with Deep Neural Networks

In this paper, we present a fully automatic brain tumor segmentation method based on Deep Neural Networks (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. By their very nature, these tumors can appear anywhere in the brain and have almost any kind of shape, size, and contrast. These reasons motivate our exploration of a machine learning solution that exploits a flexible, high capacity DNN while being extremely efficient. Here, we give a description of different model choices that we've found to be necessary for obtaining competitive performance. We explore in particular different architectures based on Convolutional Neural Networks (CNN), i.e. DNNs specifically adapted to image data. We present a novel CNN architecture which differs from those traditionally used in computer vision. Our CNN exploits both local features as well as more global contextual features simultaneously. Also, different from most traditional uses of CNNs, our networks use a final layer that is a convolutional implementation of a fully connected layer which allows a 40 fold speed up. We also describe a 2-phase training procedure that allows us to tackle difficulties related to the imbalance of tumor labels. Finally, we explore a cascade architecture in which the output of a basic CNN is treated as an additional source of information for a subsequent CNN. Results reported on the 2013 BRATS test dataset reveal that our architecture improves over the currently published state-of-the-art while being over 30 times faster

Detection of brain tumour in 2d MRI: implementation and critical review of clustering-based image segmentation methods

Image segmentation can be defined as segregation or partitioning of images into multiple regions with the same predefined homogeneity criterion. Image segmentation is a crucial process in medical image analysis. This paper explores and investigates several unsupervised image segmentation approaches and their viability and performances in delineating tumour region in contrast enhanced T1-weighted brain MRI (Magnetic Resonance Imaging) scans. First and foremost, raw CE T1-weighted brain MR images are downloaded from a free online database. The images are then pre-processed and undergo an important process called skull stripping. Then, image segmentation techniques such as k-means clustering, Gaussian mixture model segmentation and fuzzy c-means are applied to the pre-processed MR images. The image segmentation results are evaluated using several performance measures, such as precision, recall, Tanimoto coefficient and Dice similarity index in reference to ground truth images. The highest average Dice coefficient is achieved by k-means (0.189) before post-processing and GMM (0.208) after post-processing. Unsupervised clustering-based brain tumour segmentation based on just image pixel intensity in single-spectral brain MRI without adaptive post-processing algorithm cannot achieve efficient and robust segmentation results

Computational Modeling for Abnormal Brain Tissue Segmentation, Brain Tumor Tracking, and Grading

This dissertation proposes novel texture feature-based computational models for quantitative analysis of abnormal tissues in two neurological disorders: brain tumor and stroke. Brain tumors are the cells with uncontrolled growth in the brain tissues and one of the major causes of death due to cancer. On the other hand, brain strokes occur due to the sudden interruption of the blood supply which damages the normal brain tissues and frequently causes death or persistent disability. Clinical management of these brain tumors and stroke lesions critically depends on robust quantitative analysis using different imaging modalities including Magnetic Resonance (MR) and Digital Pathology (DP) images. Due to uncontrolled growth and infiltration into the surrounding tissues, the tumor regions appear with a significant texture variation in the static MRI volume and also in the longitudinal imaging study. Consequently, this study developed computational models using novel texture features to segment abnormal brain tissues (tumor, and stroke lesions), tracking the change of tumor volume in longitudinal images, and tumor grading in MR images. Manual delineation and analysis of these abnormal tissues in large scale is tedious, error-prone, and often suffers from inter-observer variability. Therefore, efficient computational models for robust segmentation of different abnormal tissues is required to support the diagnosis and analysis processes. In this study, brain tissues are characterized with novel computational modeling of multi-fractal texture features for multi-class brain tumor tissue segmentation (BTS) and extend the method for ischemic stroke lesions in MRI. The robustness of the proposed segmentation methods is evaluated using a huge amount of private and public domain clinical data that offers competitive performance when compared with that of the state-of-the-art methods. Further, I analyze the dynamic texture behavior of tumor volume in longitudinal imaging and develop post-processing frame-work using three-dimensional (3D) texture features. These post-processing methods are shown to reduce the false positives in the BTS results and improve the overall segmentation result in longitudinal imaging. Furthermore, using this improved segmentation results the change of tumor volume has been quantified in three types such as stable, progress, and shrinkage as observed by the volumetric changes of different tumor tissues in longitudinal images. This study also investigates a novel non-invasive glioma grading, for the first time in literature, that uses structural MRI only. Such non-invasive glioma grading may be useful before an invasive biopsy is recommended. This study further developed an automatic glioma grading scheme using the invasive cell nuclei morphology in DP images for cross-validation with the same patients. In summary, the texture-based computational models proposed in this study are expected to facilitate the clinical management of patients with the brain tumors and strokes by automating large scale imaging data analysis, reducing human error, inter-observer variability, and producing repeatable brain tumor quantitation and grading

Longitudinal Brain Tumor Tracking, Tumor Grading, and Patient Survival Prediction Using MRI

This work aims to develop novel methods for brain tumor classification, longitudinal brain tumor tracking, and patient survival prediction. Consequently, this dissertation proposes three tasks. First, we develop a framework for brain tumor segmentation prediction in longitudinal multimodal magnetic resonance imaging (mMRI) scans, comprising two methods: feature fusion and joint label fusion (JLF). The first method fuses stochastic multi-resolution texture features with tumor cell density features, in order to obtain tumor segmentation predictions in follow-up scans from a baseline pre-operative timepoint. The second method utilizes JLF to combine segmentation labels obtained from (i) the stochastic texture feature-based and Random Forest (RF)-based tumor segmentation method; and (ii) another state-of-the-art tumor growth and segmentation method known as boosted Glioma Image Segmentation and Registration (GLISTRboost, or GB). With the advantages of feature fusion and label fusion, we achieve state-of-the-art brain tumor segmentation prediction. Second, we propose a deep neural network (DNN) learning-based method for brain tumor type and subtype grading using phenotypic and genotypic data, following the World Health Organization (WHO) criteria. In addition, the classification method integrates a cellularity feature which is derived from the morphology of a pathology image to improve classification performance. The proposed method achieves state-of-the-art performance for tumor grading following the new CNS tumor grading criteria. Finally, we investigate brain tumor volume segmentation, tumor subtype classification, and overall patient survival prediction, and then we propose a new context- aware deep learning method, known as the Context Aware Convolutional Neural Network (CANet). Using the proposed method, we participated in the Multimodal Brain Tumor Segmentation Challenge 2019 (BraTS 2019) for brain tumor volume segmentation and overall survival prediction tasks. In addition, we also participated in the Radiology-Pathology Challenge 2019 (CPM-RadPath 2019) for Brain Tumor Subtype Classification, organized by the Medical Image Computing & Computer Assisted Intervention (MICCAI) Society. The online evaluation results show that the proposed methods offer competitive performance from their use of state-of-the-art methods in tumor volume segmentation, promising performance on overall survival prediction, and state-of-the-art performance on tumor subtype classification. Moreover, our result was ranked second place in the testing phase of the CPM-RadPath 2019

"The Effect of Arylhydrazone of Active Methylene Compounds on GBM Cell Lines"

Glioblastoma (GBM), a WHO grade IV glioma, is the most common and one of the malignant types of central nervous system tumors in adults. The resistance of GBM cells to antineoplastic agents has been the biggest obstacle in the treatment of GBM tumors. The temozolomide, a frontline GBM chemotherapy drug, has increased the median survival of the patients approximately for one and a half years only. Therefore, there is a need for the development of a novel anti-GBM drug that increases the median survival of the GBM patients with fewer side effects. The hydrazone derivatives inhibit the growth and proliferation of GBM tumors. In this study, arylhydrazone of active methylene compounds (AHAMCs) was examined using GBM cell lines U87 and LN229. The antitumor activity of AHAMC compound R234 was investigated by performing cytotoxic assay and dose- and time-dependent assay. To ascertain whether the R234 compound obstructs the cell cycle progression, GBM cells were stained with the propidium iodide (PI) dye and analyzed using cellProfiler software. To improve the quality of the segmentation several combinations of thresholding strategies and thresholding methods were tested for both the cell lines. The global and adaptive thresholding strategies and Otsu’s and maximum correlation thresholding (MCT) methods that returned comparable segmentation results were selected. Further, segmentation results of the selected thresholding strategies and methods were compared by calculating the false negatives (FN) and false positives (FP) for each combination manually. Finally, the segmentation results obtained from the combination of global strategy and MCT method were chosen for the intensity measurement. The cells were then grouped into the different phases of the cell cycle from the histogram of PI intensities. The proportions of the cells in each phase of the cell cycle show that the R234 arrests the cell cycle at the G2/M phase in both cell lines. Overall, the results suggest that the R234 compound exerts its cytotoxic activity by arresting the cell cycle at the G2/M phase

Segmentação automática de tecidos cerebrais

No âmbito da cadeira de Projeto do Mestrado em Instrumentação Biomédica foi proposto o trabalho de segmentação automática de tecidos cerebrais de imagens de ressonância magnética para ser um classificador num sistema de segmentação já existente. O trabalho teve como objetivo principal obter um modelo estatístico de previsão das intensidades dos pixéis para os diferentes tipos de tecidos cerebrais, através de modelos de mistura Gaussianas por processos de Dirichlet, das imagens da ressonância magnética. Com isso, foi feito um levantamento dos modelos estatísticos de previsão, quer paramétricos quer não-paramétricos, onde o modelo de mistura Gaussianas por processos de Dirichlet se insere. Também foi estudado o modelo de mistura Gaussianas por processos de Dirichlet apresentando a sua estrutura e o seu procedimento para calcular a distribuição de previsão dos dados observados. O trabalho experimental realizado apresenta a capacidade do modelo de mistura Gaussianas por processos de Dirichlet fazer a previsão de dados amostrados a partir de distribuições conhecidas, onde revela que o modelo consegue estimar corretamente a distribuição preditiva dos dados e até para amostras de misturas de distribuições conhecidas. Uma vez conhecida a aptidão do modelo de mistura Gaussianas por processos de Dirichlet para estimar a distribuição preditiva dos dados, foi realizada a sua implementação na segmentação de tecidos cerebrais. O resultado obtido da segmentação feita por modelos de mistura Gaussianas por processos de Dirichlet para os tecidos cerebrais, em comparação com um método de previsão paramétrico, indica que a segmentação por modelos de mistura Gaussianas por processos de Dirichlet obtém melhores resultados embora com limitações. Verificando essas limitações, descobrimos que a sua causa se deve à sobreposição de intensidades dos tecidos cerebrais. Com isso, alterámos o modelo para que este se ajustasse às intensidades dos tecidos cerebrais a fim de diminuir as limitações. No caso do tecido líquido cerebrospinal, modificámos o modelo para trabalhar com a distribuição Riciana. Observou-se que com essa alteração que a segmentação proposta melhorou com uma pequena diferença. No caso do tecido substância branca, modificámos o modelo para trabalhar com novas distribuições: Genlogistic, Powernorm, Log Gama, Gumbel enviesada à esquerda e Johnson Su. Apenas o modelo que trabalha com a distribuição Johnson Su é que consegue melhorar a segmentação dos tecidos cerebrais embora com uma diferença mínima

MR görüntüleri ve MR spektroskopi verileri ile yapay öğrenme tabanlı beyin tümörü tespit yöntemi ve uygulaması

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Beyinde büyüyen ve gelişen kötü huylu tümörler son zamanlarda insan ölümlerinin en önde gelen nedenlerinden birisi olmaya başlamıştır. Beyin tümörleri için en uygun tedavi yönteminin belirlenmesi hekim tarafından tümörün türünün ve evresinin belirlenmesine bağlıdır. Beyin tümörünün tecrübeli radyologlar tarafından tam olarak teşhis edilebilmesi, Manyetik Rezonans (MR görüntüleri), MR spektroskopi verileri ve patolojik değerlendirmeleri içerisine alan karmaşık bir süreçtir. Genel olarak bir radyolog bu süreçle ilgili olarak önemli doğruluk ve hassaslıkta karar verebiliyor olsa da, hataları en aza indirebilmek için sürekli yeni yöntemler araştırılmaktadır. Bu yüzden radyolog ya da hekimlerin beyin tümörlerinin ayrımını yüksek oranda yapabilecek Bilgisayar Destekli Teşhis (Computer-Aided Detection, CAD / BDT) sistemlerinden yararlanması oldukça önemlidir. Bu tez çalışmasında, hem MR görüntüleri ile hem de MR Spektroskopi (MRS) verileri kullanarak, radyologların karar verme aşamalarında yardımcı olabilecek, beyin tümörlerinin tespitini başarılı bir şekilde yapan yeni bilgisayar destekli yaklaşımlar önerilmiştir. Tez kapsamında geliştirilen ilk yöntem MR görüntüleri üzerinde çalışmakta ve beyin tümörlerinin iyi/kötü huylu ayrımlarını görüntü işleme ve örüntü tanıma teknikleri ile gerçekleştirmektedir. Bu işlemi gerçekleştirmek amacıyla MR görüntüleri üzerinde kafatası kısmını çıkarma için yeni bir görüntü ön-işleme tekniği önerilmiştir. Ayrıca, tümör ayrımlarında sınıflandırıcı etkisini görebilmek için farklı sınıflandırıcıların başarımları kıyaslanmıştır. 188 adet MR görüntüsü üzerinde yapılan detaylı deney sonuçlarına göre, önerilen yöntem ile %96.81 doğruluk oranı ile beyin tümörlerinin iyi / kötü huylu ayrımı gerçekleştirilebilmiştir. Tez kapsamında önerilen bir diğer yöntemde ise, MR spektroskopi sinyalleri üzerinde çalışan ve Yapay Bağışıklık Sistemi (YBS) tabanlı yeni bir BDT yaklaşımı geliştirilmiştir. Önerilen yöntem ile MRS verileri kullanılarak iyi huylu / kötü huylu tümör ayrımı, beyin tümörünün evrelemesi, normal beyin dokusu ile beyin tümörünün ayrımı, metastaz beyin tümörleri ile birincil beyin tümörlerinin ayrımı ve sahte tümörlerin belirlenmesi yüksek başarımla mümkün olmuştur. Çok uluslu ve merkezli bir proje kapsamında elde edilen geniş bir veri seti ile gerçekleştirilen deney sonuçlarına göre sırasıyla %96.97, %100, %100, %98.33 ve %98.44 başarım elde edilmiştir.Malignant tumors growing and developing in the brain have recently become one of the leading causes of death in humans. Determination of the most suitable treatment for brain tumors depends on accurate detection of malignancy, type and grade of the tumor by the physician. Diagnosis of brain tumors by radiologists is a complex process which includes MR images, MR spectroscopy data and pathological assessments. Generally, a radiologist makes a decision with reasonable accuracy and specifity rates. However new methods have been investigated by the researchers to minimize the diagnosis mistakes. Therefore, it is crucial for radiologists or physicians to use a Computer-Aided Diagnosis (CAD) system which will help detection of brain tumors with high success rates. In this thesis, novel computer aided methods, which use MR images and MR Spectroscopy data, have been proposed for the detection of brain tumors to support decision process of the radiologists. The first method developed in the thesis differentiates brain tumors as benign or malignant by image processing and pattern recognition techniques on MR images. To perform this operation, a new image pre-processing technique has been proposed to strip the skull region. Moreover, to evaluate the effect of classifier performance on tumor differentiation, different classifiers have been compared. According to detailed test results performed on 188 MR images, benign or malignant differentiation of brain tumors can be detected with 96.81% accuracy rate by proposed method. In the second method, a novel Artificial Immune System (AIS) based computer-aided diagnosis system has been proposed. This system utilizes MR Spectroscopy signals to make a decision about brain tumors. The system can perform differentiation of benign / malign, metastatic / primary, pseudo / normal tumors and grading of brain tumors with high accuracy rates. According to the experimental results performed on large dataset obtained from an international and multi-center project, the detection performance has been achieved 96.97%, 100%, 100%, 98.33% and 98.44% success rates respectively

Supervised learning-based multimodal MRI brain image analysis

Medical imaging plays an important role in clinical procedures related to cancer, such as diagnosis, treatment selection, and therapy response evaluation. Magnetic resonance imaging (MRI) is one of the most popular acquisition modalities which is widely used in brain tumour analysis and can be acquired with different acquisition protocols, e.g. conventional and advanced. Automated segmentation of brain tumours in MR images is a difficult task due to their high variation in size, shape and appearance. Although many studies have been conducted, it still remains a challenging task and improving accuracy of tumour segmentation is an ongoing field. The aim of this thesis is to develop a fully automated method for detection and segmentation of the abnormal tissue associated with brain tumour (tumour core and oedema) from multimodal MRI images. In this thesis, firstly, the whole brain tumour is segmented from fluid attenuated inversion recovery (FLAIR) MRI, which is commonly acquired in clinics. The segmentation is achieved using region-wise classification, in which regions are derived from superpixels. Several image features including intensity-based, Gabor textons, fractal analysis and curvatures are calculated from each superpixel within the entire brain area in FLAIR MRI to ensure a robust classification. Extremely randomised trees (ERT) classifies each superpixel into tumour and non-tumour. Secondly, the method is extended to 3D supervoxel based learning for segmentation and classification of tumour tissue subtypes in multimodal MRI brain images. Supervoxels are generated using the information across the multimodal MRI data set. This is then followed by a random forests (RF) classifier to classify each supervoxel into tumour core, oedema or healthy brain tissue. The information from the advanced protocols of diffusion tensor imaging (DTI), i.e. isotropic (p) and anisotropic (q) components is also incorporated to the conventional MRI to improve segmentation accuracy. Thirdly, to further improve the segmentation of tumour tissue subtypes, the machine-learned features from fully convolutional neural network (FCN) are investigated and combined with hand-designed texton features to encode global information and local dependencies into feature representation. The score map with pixel-wise predictions is used as a feature map which is learned from multimodal MRI training dataset using the FCN. The machine-learned features, along with hand-designed texton features are then applied to random forests to classify each MRI image voxel into normal brain tissues and different parts of tumour. The methods are evaluated on two datasets: 1) clinical dataset, and 2) publicly available Multimodal Brain Tumour Image Segmentation Benchmark (BRATS) 2013 and 2017 dataset. The experimental results demonstrate the high detection and segmentation performance of the III single modal (FLAIR) method. The average detection sensitivity, balanced error rate (BER) and the Dice overlap measure for the segmented tumour against the ground truth for the clinical data are 89.48%, 6% and 0.91, respectively; whilst, for the BRATS dataset, the corresponding evaluation results are 88.09%, 6% and 0.88, respectively. The corresponding results for the tumour (including tumour core and oedema) in the case of multimodal MRI method are 86%, 7%, 0.84, for the clinical dataset and 96%, 2% and 0.89 for the BRATS 2013 dataset. The results of the FCN based method show that the application of the RF classifier to multimodal MRI images using machine-learned features based on FCN and hand-designed features based on textons provides promising segmentations. The Dice overlap measure for automatic brain tumor segmentation against ground truth for the BRATS 2013 dataset is 0.88, 0.80 and 0.73 for complete tumor, core and enhancing tumor, respectively, which is competitive to the state-of-the-art methods. The corresponding results for BRATS 2017 dataset are 0.86, 0.78 and 0.66 respectively. The methods demonstrate promising results in the segmentation of brain tumours. This provides a close match to expert delineation across all grades of glioma, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management. In the experiments, texton has demonstrated its advantages of providing significant information to distinguish various patterns in both 2D and 3D spaces. The segmentation accuracy has also been largely increased by fusing information from multimodal MRI images. Moreover, a unified framework is present which complementarily integrates hand-designed features with machine-learned features to produce more accurate segmentation. The hand-designed features from shallow network (with designable filters) encode the prior-knowledge and context while the machine-learned features from a deep network (with trainable filters) learn the intrinsic features. Both global and local information are combined using these two types of networks that improve the segmentation accuracy