3D U-Net Based Brain Tumor Segmentation and Survival Days Prediction

Past few years have witnessed the prevalence of deep learning in many application scenarios, among which is medical image processing. Diagnosis and treatment of brain tumors requires an accurate and reliable segmentation of brain tumors as a prerequisite. However, such work conventionally requires brain surgeons significant amount of time. Computer vision techniques could provide surgeons a relief from the tedious marking procedure. In this paper, a 3D U-net based deep learning model has been trained with the help of brain-wise normalization and patching strategies for the brain tumor segmentation task in the BraTS 2019 competition. Dice coefficients for enhancing tumor, tumor core, and the whole tumor are 0.737, 0.807 and 0.894 respectively on the validation dataset. These three values on the test dataset are 0.778, 0.798 and 0.852. Furthermore, numerical features including ratio of tumor size to brain size and the area of tumor surface as well as age of subjects are extracted from predicted tumor labels and have been used for the overall survival days prediction task. The accuracy could be 0.448 on the validation dataset, and 0.551 on the final test dataset.Comment: Third place award of the 2019 MICCAI BraTS challenge survival task [BraTS 2019](https://www.med.upenn.edu/cbica/brats2019.html

A Feasibility Study on Deep Learning Based Brain Tumor Segmentation Using 2D Ellipse Box Areas

In most deep learning-based brain tumor segmentation methods, training the deep network requires annotated tumor areas. However, accurate tumor annotation puts high demands on medical personnel. The aim of this study is to train a deep network for segmentation by using ellipse box areas surrounding the tumors. In the proposed method, the deep network is trained by using a large number of unannotated tumor images with foreground (FG) and background (BG) ellipse box areas surrounding the tumor and background, and a small number of patients (<20) with annotated tumors. The training is conducted by initial training on two ellipse boxes on unannotated MRIs, followed by refined training on a small number of annotated MRIs. We use a multi-stream U-Net for conducting our experiments, which is an extension of the conventional U-Net. This enables the use of complementary information from multi-modality (e.g., T1, T1ce, T2, and FLAIR) MRIs. To test the feasibility of the proposed approach, experiments and evaluation were conducted on two datasets for glioma segmentation. Segmentation performance on the test sets is then compared with those used on the same network but trained entirely by annotated MRIs. Our experiments show that the proposed method has obtained good tumor segmentation results on the test sets, wherein the dice score on tumor areas is (0.8407, 0.9104), and segmentation accuracy on tumor areas is (83.88%, 88.47%) for the MICCAI BraTS’17 and US datasets, respectively. Comparing the segmented results by using the network trained by all annotated tumors, the drop in the segmentation performance from the proposed approach is (0.0594, 0.0159) in the dice score, and (8.78%, 2.61%) in segmented tumor accuracy for MICCAI and US test sets, which is relatively small. Our case studies have demonstrated that training the network for segmentation by using ellipse box areas in place of all annotated tumors is feasible, and can be considered as an alternative, which is a trade-off between saving medical experts’ time annotating tumors and a small drop in segmentation performance

Artificial intelligence in radiation oncology: A review of its current status and potential application for the radiotherapy workforce

Objective Radiation oncology is a continually evolving speciality. With the development of new imaging modalities and advanced imaging processing techniques, there is an increasing amount of data available to practitioners. In this narrative review, Artificial Intelligence (AI) is used as a reference to machine learning, and its potential, along with current problems in the field of radiation oncology, are considered from a technical position. Key Findings AI has the potential to harness the availability of data for improving patient outcomes, reducing toxicity, and easing clinical burdens. However, problems including the requirement of complexity of data, undefined core outcomes and limited generalisability are apparent. Conclusion This original review highlights considerations for the radiotherapy workforce, particularly therapeutic radiographers, as there will be an increasing requirement for their familiarity with AI due to their unique position as the interface between imaging technology and patients. Implications for practice Collaboration between AI experts and the radiotherapy workforce are required to overcome current issues before clinical adoption. The development of educational resources and standardised reporting of AI studies may help facilitate this

Machine Learning and Radiomic Features to Predict Overall Survival Time for Glioblastoma Patients

Glioblastoma is an aggressive brain tumor with a low survival rate. Understanding tumor behavior by predicting prognosis outcomes is a crucial factor in deciding a proper treatment plan. In this paper, an automatic overall survival time prediction system (OST) for glioblastoma patients is developed on the basis of radiomic features and machine learning (ML). This system is designed to predict prognosis outcomes by classifying a glioblastoma patient into one of three survival groups: short-term, mid-term, and long-term. To develop the prediction system, a medical dataset based on imaging information from magnetic resonance imaging (MRI) and non-imaging information is used. A novel radiomic feature extraction method is proposed and developed on the basis of volumetric and location information of brain tumor subregions extracted from MRI scans. This method is based on calculating the volumetric features from two brain sub-volumes obtained from the whole brain volume in MRI images using brain sectional planes (sagittal, coronal, and horizontal). Many experiments are conducted on the basis of various ML methods and combinations of feature extraction methods to develop the best OST system. In addition, the feature fusions of both radiomic and non-imaging features are examined to improve the accuracy of the prediction system. The best performance was achieved by the neural network and feature fusions

Classification of glioma grading in brain MRI

Táto práca sa zaoberá klasifikáciou mozgových gliómových nádorov na nízko a vysoko agresívne nádory a predikciou doby prežitia pacientov po úplnej resekcii nádoru na základe obrazových dát dostupných z magnetickej rezonancie. Použité obrazy pochádzajú z výzvy BRATS challenge 2019 a každý súbor dát obsahoval informáciu zo štyroch váhovacích sekvencií. Práca je implementovaná v jazyku PYTHON a programovom prostredí Jupyter Notebooks. Pri výpočte obrazových príznakov bola použitá knižnica PyRadiomics. Cieľom práce bolo zistiť, z ktorej oblasti nádoru a váhovacej sekvencie je najvýhodnejšie počítať príznaky a následne nájsť súbor príznakov s ktorým by bola získaná najvyššia úspešnosť klasifikácie a predikcie prežitia. V práci sa nachádza aj vyhodnotenie predikcie prežitia na základe štatistických metód analýzy prežitia, konkrétne Coxovou regresiou.This thesis deals with a classification of glioma grade in high and low aggressive tumours and overall survival prediction based on magnetic resonance imaging. Data used in this work is from BRATS challenge 2019 and each set contains information from 4 weighting sequences of MRI. Thesis is implemented in PYTHON programming language and Jupyter Notebooks environment. Software PyRadiomics is used for calculation of image features. Goal of this work is to determine best tumour region and weighting sequence for calculation of image features and consequently select set of features that are the best ones for classification of tumour grade and survival prediction. Part of thesis is dedicated to survival prediction using set of statistical tests, specifically Cox regression

Radiomic Features to Predict Overall Survival Time for Patients with Glioblastoma Brain Tumors Based on Machine Learning and Deep Learning Methods

Machine Learning (ML) methods including Deep Learning (DL) Methods have been employed in the medical field to improve diagnosis process and patient’s prognosis outcomes. Glioblastoma multiforme is an extremely aggressive Glioma brain tumor that has a poor survival rate. Understanding the behavior of the Glioblastoma brain tumor is still uncertain and some factors are still unrecognized. In fact, the tumor behavior is important to decide a proper treatment plan and to improve a patient’s health. The aim of this dissertation is to develop a Computer-Aided-Diagnosis system (CADiag) based on ML/DL methods to automatically estimate the Overall Survival Time (OST) for patients with Glioblastoma brain tumors from medical imaging and non-imaging data. This system is developed to enhance and speed-up the diagnosis process, as well as to increase understanding of the behavior of Glioblastoma brain tumors. The proposed OST prediction system is developed based on a classification process to categorize a GBM patient into one of the following three survival time groups: short-term (months), mid-term (10-15 months), and long-term (\u3e15 months). The Brain Tumor Segmentation challenge (BraTS) dataset is used to develop the automatic OST prediction system. This dataset consists of multimodal preoperative Magnetic Resonance Imaging (mpMRI) data, and clinical data. The training data is relatively small in size to train an accurate OST prediction model based on DL method. Therefore, traditional ML methods such as Support Vector Machine (SVM), Neural Network, K-Nearest Neighbor (KNN), Decision Tree (DT) were used to develop the OST prediction model for GBM patients. The main contributions in the perspective of ML field include: developing and evaluating five novel radiomic feature extraction methods to produce an automatic and reliable OST prediction system based on classification task. These methods are volumetric, shape, location, texture, histogram-based, and DL features. Some of these radiomic features can be extracted directly from MRI images, such as statistical texture features and histogram-based features. However, preprocessing methods are required to extract automatically other radiomic features from MRI images such as the volume, shape, and location information of the GBM brain tumors. Therefore, a three-dimension (3D) segmentation DL model based on modified U-Net architecture is developed to identify and localize the three glioma brain tumor subregions, peritumoral edematous/invaded tissue (ED), GD-enhancing tumor (ET), and the necrotic tumor core (NCR), in multi MRI scans. The segmentation results are used to calculate the volume, location and shape information of a GBM tumor. Two novel approaches based on volumetric, shape, and location information, are proposed and evaluated in this dissertation. To improve the performance of the OST prediction system, information fusion strategies based on data-fusion, features-fusion and decision-fusion are involved. The best prediction model was developed based on feature fusions and ensemble models using NN classifiers. The proposed OST prediction system achieved competitive results in the BraTS 2020 with accuracy 55.2% and 55.1% on the BraTS 2020 validation and test datasets, respectively. In sum, developing automatic CADiag systems based on robust features and ML methods, such as our developed OST prediction system, enhances the diagnosis process in terms of cost, accuracy, and time. Our OST prediction system was evaluated from the perspective of the ML field. In addition, preprocessing steps are essential to improve not only the quality of the features but also boost the performance of the prediction system. To test the effectiveness of our developed OST system in medical decisions, we suggest more evaluations from the perspective of biology and medical decisions, to be then involved in the diagnosis process as a fast, inexpensive and automatic diagnosis method. To improve the performance of our developed OST prediction system, we believe it is required to increase the size of the training data, involve multi-modal data, and/or provide any uncertain or missing information to the data (such as patients\u27 resection statuses, gender, etc.). The DL structure is able to extract numerous meaningful low-level and high-level radiomic features during the training process without any feature type nominations by researchers. We thus believe that DL methods could achieve better predictions than ML methods if large size and proper data is available