BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks

We present a simple and effective framework for simultaneous semantic segmentation and instance segmentation with Fully Convolutional Networks (FCNs). The method, called BiSeg, predicts instance segmentation as a posterior in Bayesian inference, where semantic segmentation is used as a prior. We extend the idea of position-sensitive score maps used in recent methods to a fusion of multiple score maps at different scales and partition modes, and adopt it as a robust likelihood for instance segmentation inference. As both Bayesian inference and map fusion are performed per pixel, BiSeg is a fully convolutional end-to-end solution that inherits all the advantages of FCNs. We demonstrate state-of-the-art instance segmentation accuracy on PASCAL VOC.Comment: BMVC201

Automatic Brain Tumor Segmentation by Deep Convolutional Networks and Graph Cuts

Brain tumor segmentation in magnetic resonance imaging (MRI) is helpful for diagnostics, growth rate prediction, tumor volume measurements and treatment planning of brain tumor. The difficulties for brain tumor segmentation are mainly due to high variation of brain tumors in size, shape, regularity, location, and their heterogeneous appearance (e.g., contrast, intensity and texture variation for different tumors). Due to recent advances in deep convolutional neural networks for semantic image segmentation, automatic brain tumor segmentation is a promising research direction. This thesis investigates automatic brain tumor segmentation by combining deep convolutional neural network with regularization by a graph cut. We investigate several deep convolutional network structures that have been successful in semantic and medical image segmentation. Since the tumor pixels account for a very small portion in the whole brain slice, segmenting the tumor from the background is a highly imbalanced dense prediction task. We use a loss function that takes the imbalance of the training data into consideration. In the second part of the thesis, we improve the segmentation results of a deep neural network by using optimization framework with graph cuts. The graph cut framework can improve segmentation boundaries by making them more smooth and regular. The main issue when using the segmentation results of convolutional neural networks for the graph cut optimization framework is to convert tumor probabilities learned by a convolutional network into data terms. We investigate several possible ways that take into consideration the segmentation artifacts by convolutional neural networks. In experiments, we present the segmentation results by different deep convolutional neural network structures, e.g., fully convolutional neural network, dilated residual network and UNet. Also, we compare the combination of U-Net with different data terms for graph cut regularization to improve the neural network segmentation results. Experimental results show that the U-Net performs best with the intersection over union (IoU) for tumors of 0.7286. The IoU for tumors is improved to 0.7530 by training on three slices. Also, the IoU for tumors is improved to 0.7713 by U-Net with balanced loss function. The IoU for tumors is further improved to 0.8078 by graph cut regularization