Enlarged Training Dataset by Pairwise GANs for Molecular-Based Brain Tumor Classification

This paper addresses issues of brain tumor subtype classification using Magnetic Resonance Images (MRIs) from different scanner modalities like T1 weighted, T1 weighted with contrast-enhanced, T2 weighted and FLAIR images. Currently most available glioma datasets are relatively moderate in size,and often accompanied with incomplete MRIs in different modalities. To tackle the commonly encountered problems of insufficiently large brain tumor datasets and incomplete modality of image for deep learning, we propose to add augmented brain MR images to enlarge the training dataset by employing a pairwise Generative Adversarial Network (GAN) model. The pairwise GAN is able to generate synthetic MRIs across different modalities. To achieve the patient-level diagnostic result, we propose a post-processing strategy to combine the slice-level glioma subtype classification results by majority voting. A two-stage course-to-fine training strategy is proposed to learn the glioma feature using GAN-augmented MRIs followed by real MRIs. To evaluate the effectiveness of the proposed scheme, experiments have been conducted on a brain tumor dataset for classifying glioma molecular subtypes: isocitrate dehydrogenase 1 (IDH1) mutation and IDH1 wild-type. Our results on the dataset have shown good performance (with test accuracy 88.82%). Comparisons with several state-of-the-art methods are also included

Machine Learning Methods for Image Analysis in Medical Applications, from Alzheimer\u27s Disease, Brain Tumors, to Assisted Living

Healthcare has progressed greatly nowadays owing to technological advances, where machine learning plays an important role in processing and analyzing a large amount of medical data. This thesis investigates four healthcare-related issues (Alzheimer\u27s disease detection, glioma classification, human fall detection, and obstacle avoidance in prosthetic vision), where the underlying methodologies are associated with machine learning and computer vision. For Alzheimer’s disease (AD) diagnosis, apart from symptoms of patients, Magnetic Resonance Images (MRIs) also play an important role. Inspired by the success of deep learning, a new multi-stream multi-scale Convolutional Neural Network (CNN) architecture is proposed for AD detection from MRIs, where AD features are characterized in both the tissue level and the scale level for improved feature learning. Good classification performance is obtained for AD/NC (normal control) classification with test accuracy 94.74%. In glioma subtype classification, biopsies are usually needed for determining different molecular-based glioma subtypes. We investigate non-invasive glioma subtype prediction from MRIs by using deep learning. A 2D multi-stream CNN architecture is used to learn the features of gliomas from multi-modal MRIs, where the training dataset is enlarged with synthetic brain MRIs generated by pairwise Generative Adversarial Networks (GANs). Test accuracy 88.82% has been achieved for IDH mutation (a molecular-based subtype) prediction. A new deep semi-supervised learning method is also proposed to tackle the problem of missing molecular-related labels in training datasets for improving the performance of glioma classification. In other two applications, we also address video-based human fall detection by using co-saliency-enhanced Recurrent Convolutional Networks (RCNs), as well as obstacle avoidance in prosthetic vision by characterizing obstacle-related video features using a Spiking Neural Network (SNN). These investigations can benefit future research, where artificial intelligence/deep learning may open a new way for real medical applications

Deep semi-supervised learning for brain tumor classification

Background: This paper addresses issues of brain tumor, glioma, classification from four modalities of Magnetic Resonance Image (MRI) scans (i.e., T1 weighted MRI, T1 weighted MRI with contrast-enhanced, T2 weighted MRI and FLAIR). Currently, many available glioma datasets often contain some unlabeled brain scans, and many datasets are moderate in size. Methods: We propose to exploit deep semi-supervised learning to make full use of the unlabeled data. Deep CNN features were incorporated into a new graph-based semi-supervised learning framework for learning the labels of the unlabeled data, where a new 3D-2D consistent constraint is added to make consistent classifications for the 2D slices from the same 3D brain scan. A deep-learning classifier is then trained to classify different glioma types using both labeled and unlabeled data with estimated labels. To alleviate the overfitting caused by moderate-size datasets, synthetic MRIs generated by Generative Adversarial Networks (GANs) are added in the training of CNNs. Results: The proposed scheme has been tested on two glioma datasets, TCGA dataset for IDH-mutation prediction (molecular-based glioma subtype classification) and MICCAI dataset for glioma grading. Our results have shown good performance (with test accuracies 86.53% on TCGA dataset and 90.70% on MICCAI dataset). Conclusions: The proposed scheme is effective for glioma IDH-mutation prediction and glioma grading, and its performance is comparable to the state-of-the-art

GENERATIVE ADVERSARIAL NETWORKS IN TUMOR-RELATED RESEARCH: A REVIEW AND AGENDA FOR MOVING FORWARD

Recent advances in Generative Adversarial Networks (GANs) have led to many new variants and uses of GANs. The latest advancements have allowed researchers and practitioners to apply this technique to tumor-related problems with limited data. One of the trends in this problem domain is to develop different variants of GANs suited explicitly to particular problems. The variants of GANs are numerous but share a common characteristic of expanding the dataset by creating synthetic data from the original dataset. This paper aims to develop a research agenda through a systematic literature review that investigates practitioners\u27 and researchers\u27 emerging issues and current works on the topic. Emerging implementation trends and limitations of GANs in tumor-related problems are explored

General Adversarial Networks in Tumor-related Research: A Review and Agenda for Moving Forward

Recent advances in Generative Adversarial Networks (GANs) have led to many new variants and uses of GANs. The latest advancements have allowed researchers and practitioners to apply this technique to tumor-related problems with limited data. One of the trends in this problem domain is to develop different variants of GANs suited explicitly to particular problems. The variants of GANs are numerous but share a common characteristic of expanding the dataset by creating synthetic data from the original dataset. This paper aims to develop a research agenda through a systematic literature review that investigates practitioners\u27 and researchers\u27 emerging issues and current works on the topic. Emerging implementation trends and limitations of GANs in tumor-related problems are explored

Deep learning based Brain Tumour Classification based on Recursive Sigmoid Neural Network based on Multi-Scale Neural Segmentation

Brain tumours are malignant tissues in which cells replicate rapidly and indefinitely, and tumours grow out of control. Deep learning has the potential to overcome challenges associated with brain tumour diagnosis and intervention. It is well known that segmentation methods can be used to remove abnormal tumour areas in the brain. It is one of the advanced technology classification and detection tools. Can effectively achieve early diagnosis of the disease or brain tumours through reliable and advanced neural network classification algorithms. Previous algorithm has some drawbacks, an automatic and reliable method for segmentation is needed. However, the large spatial and structural heterogeneity between brain tumors makes automated segmentation a challenging problem. Image tumors have irregular shapes and are spatially located in any part of the brain, making their segmentation is inaccurate for clinical purposes a challenging task. In this work, propose a method Recursive SigmoidNeural Network based on Multi-scale Neural Segmentation (RSN2-MSNS) for image proper segmentation. Initially collets the image dataset from standard repository for brain tumour classification.  Next, pre-processing method that targets only a small part of an image rather than the entire image. This approach reduces computational time and overcomes the over complication. Second stage, segmenting the images based on the Enhanced Deep Clustering U-net (EDCU-net) for estimating the boundary points in the brain tumour images. This method can successfully colour histogram values are evaluating segment complex images that contain both textured and non-textured regions. Third stage, Feature extraction for extracts the features from segmenting images using Convolution Deep Feature Spectral Similarity (CDFS2) scaled the values from images extracting the relevant weights based on its threshold limits. Then selecting the features from extracting stage, this selection is based on the relational weights. And finally classified the features based on the Recursive Sigmoid Neural Network based on Multi-scale Neural Segmentation (RSN2-MSNS) for evaluating the proposed brain tumour classification model consists of 1500 trainable images and the proposed method achieves 97.0% accuracy. The sensitivity, specificity, detection accuracy and F1 measures were 96.4%, 952%, and 95.9%, respectively

A novel federated deep learning scheme for glioma and its subtype classification

Background:\ua0Deep learning (DL) has shown promising results in molecular-based classification of glioma subtypes from MR images. DL requires a large number of training data for achieving good generalization performance. Since brain tumor datasets are usually small in size, combination of such datasets from different hospitals are needed. Data privacy issue from hospitals often poses a constraint on such a practice. Federated learning (FL) has gained much attention lately as it trains a central DL model without requiring data sharing from different hospitals.Method:\ua0We propose a novel 3D FL scheme for glioma and its molecular subtype classification. In the scheme, a slice-based DL classifier, EtFedDyn, is exploited which is an extension of FedDyn, with the key differences on using focal loss cost function to tackle severe class imbalances in the datasets, and on multi-stream network to exploit MRIs in different modalities. By combining EtFedDyn with domain mapping as the pre-processing and 3D scan-based post-processing, the proposed scheme makes 3D brain scan-based classification on datasets from different dataset owners. To examine whether the FL scheme could replace the central learning (CL) one, we then compare the classification performance between the proposed FL and the corresponding CL schemes. Furthermore, detailed empirical-based analysis were also conducted to exam the effect of using domain mapping, 3D scan-based post-processing, different cost functions and different FL schemes.Results:\ua0Experiments were done on two case studies: classification of glioma subtypes (IDH mutation and wild-type on TCGA and US datasets in case A) and glioma grades (high/low grade glioma HGG and LGG on MICCAI dataset in case B). The proposed FL scheme has obtained good performance on the test sets (85.46%, 75.56%) for IDH subtypes and (89.28%, 90.72%) for glioma LGG/HGG all averaged on five runs. Comparing with the corresponding CL scheme, the drop in test accuracy from the proposed FL scheme is small (−1.17%, −0.83%), indicating its good potential to replace the CL scheme. Furthermore, the empirically tests have shown that an increased classification test accuracy by applying: domain mapping (0.4%, 1.85%) in case A; focal loss function (1.66%, 3.25%) in case A and (1.19%, 1.85%) in case B; 3D post-processing (2.11%, 2.23%) in case A and (1.81%, 2.39%) in case B and EtFedDyn over FedAvg classifier (1.05%, 1.55%) in case A and (1.23%, 1.81%) in case B with fast convergence, which all contributed to the improvement of overall performance in the proposed FL scheme.Conclusion:\ua0The proposed FL scheme is shown to be effective in predicting glioma and its subtypes by using MR images from test sets, with great potential of replacing the conventional CL approaches for training deep networks. This could help hospitals to maintain their data privacy, while using a federated trained classifier with nearly similar performance as that from a centrally trained one. Further detailed experiments have shown that different parts in the proposed 3D FL scheme, such as domain mapping (make datasets more uniform) and post-processing (scan-based classification), are essential

Deep Learning Methods for Classification of Gliomas and Their Molecular Subtypes, From Central Learning to Federated Learning

The most common type of brain cancer in adults are gliomas. Under the updated 2016 World Health Organization (WHO) tumor classification in central nervous system (CNS), identification of molecular subtypes of gliomas is important. For low grade gliomas (LGGs), prediction of molecular subtypes by observing magnetic resonance imaging (MRI) scans might be difficult without taking biopsy. With the development of machine learning (ML) methods such as deep learning (DL), molecular based classification methods have shown promising results from MRI scans that may assist clinicians for prognosis and deciding on a treatment strategy. However, DL requires large amount of training datasets with tumor class labels and tumor boundary annotations. Manual annotation of tumor boundary is a time consuming and expensive process.The thesis is based on the work developed in five papers on gliomas and their molecular subtypes. We propose novel methods that provide improved performance. \ua0The proposed methods consist of a multi-stream convolutional autoencoder (CAE)-based classifier, a deep convolutional generative adversarial network (DCGAN) to enlarge the training dataset, a CycleGAN to handle domain shift, a novel federated learning (FL) scheme to allow local client-based training with dataset protection, and employing bounding boxes to MRIs when tumor boundary annotations are not available.Experimental results showed that DCGAN generated MRIs have enlarged the original training dataset size and have improved the classification performance on test sets. CycleGAN showed good domain adaptation on multiple source datasets and improved the classification performance. The proposed FL scheme showed a slightly degraded performance as compare to that of central learning (CL) approach while protecting dataset privacy. Using tumor bounding boxes showed to be an alternative approach to tumor boundary annotation for tumor classification and segmentation, with a trade-off between a slight decrease in performance and saving time in manual marking by clinicians. The proposed methods may benefit the future research in bringing DL tools into clinical practice for assisting tumor diagnosis and help the decision making process

Explainability of deep neural networks for MRI analysis of brain tumors

Purpose Artificial intelligence (AI), in particular deep neural networks, has achieved remarkable results for medical image analysis in several applications. Yet the lack of explainability of deep neural models is considered the principal restriction before applying these methods in clinical practice. Methods In this study, we propose a NeuroXAI framework for explainable AI of deep learning networks to increase the trust of medical experts. NeuroXAI implements seven state-of-the-art explanation methods providing visualization maps to help make deep learning models transparent. Results NeuroXAI has been applied to two applications of the most widely investigated problems in brain imaging analysis, i.e., image classification and segmentation using magnetic resonance (MR) modality. Visual attention maps of multiple XAI methods have been generated and compared for both applications. Another experiment demonstrated that NeuroXAI can provide information flow visualization on internal layers of a segmentation CNN. Conclusion Due to its open architecture, ease of implementation, and scalability to new XAI methods, NeuroXAI could be utilized to assist radiologists and medical professionals in the detection and diagnosis of brain tumors in the clinical routine of cancer patients. The code of NeuroXAI is publicly accessible at https://github.com/razeineldin/NeuroXAI

IMPLEMENTASI MASK-RCNN PADA DATASET KECIL CITRA SEL DARAH MERAH BERDASARKAN KRITERIA WARNA SEL

Examination of red blood cell morphology is one of the diagnostic aids for several diseases, one of which is anemia. The development of the application of digital image processing technology, artificial intelligence, and computer-assisted diagnosis opens opportunities to solve various problems related to medical images. Red blood cells sticking together or overlapping is a challenge in the red blood cell segmentation process which ultimately affects the results of cell type identification. A method that can perform instance segmentation is needed to overcome this problem. This study aims to implement the Mask-RCNN algorithm on a small red blood cell image dataset and evaluate the prediction results' performance. Based on the research results, the attached red blood cells can be detected individually by the model, and the accuracy of the cell detection results is 68.27%. Mask-RCNN can be used for blood cell segmentation instances and blood cell detection on small datasets, but the model accuracy still needs to be improved. Therefore it is necessary to do further research by increasing the number of datasets used.Pemeriksaan morfologi sel darah merah merupakan salah satu alat bantu penegakan diagnosis pada beberapa penyakit, salah satunya anemia. Perkembangan penerapan teknologi pengolahan citra digital, kecerdasan artifisial dan computer-aided diagnosis membuka peluang untuk menyelesaikan berbagai permasalahan terkait citra medis. Sel darah merah yang saling menempel atau bertumpuk merupakan tantangan dalam proses segmentasi sel darah merah yang pada akhirnya berpengaruh pada hasil pengenalan jenis sel. Metode yang dapat melakukan instance segmentation sangat diperlukan untuk mengatasi masalah tersebut. Penelitian ini bertujuan untuk mengimplementasikan algoritma Mask-RCNN pada dataset kecil citra sel darah merah dan mengevaluasi performa hasil prediksi. Berdasarkan hasil penelitian sel-sel darah merah yang menempel dapat dideteksi secara individual oleh model dan akurasi hasil deteksi sel adalah 68,27%. Mask-RCNN dapat digunakan untuk instance segmentasi sel darah dan deteksi sel darah pada dataset kecil namun akurasi model masih perlu ditingkatkan. oleh sebab itu perlu dilakukan penelitian selanjutnya dengan menambah jumlah dataset yang digunakan