Saliency-guided Adaptive Seeding for Supervoxel Segmentation

We propose a new saliency-guided method for generating supervoxels in 3D space. Rather than using an evenly distributed spatial seeding procedure, our method uses visual saliency to guide the process of supervoxel generation. This results in densely distributed, small, and precise supervoxels in salient regions which often contain objects, and larger supervoxels in less salient regions that often correspond to background. Our approach largely improves the quality of the resulting supervoxel segmentation in terms of boundary recall and under-segmentation error on publicly available benchmarks.Comment: 6 pages, accepted to IROS201

Evaluation on the compactness of supervoxels

Supervoxels are perceptually meaningful atomic spatiotemporal regions in videos, which has great potential to reduce the computational complexity of downstream video applications. Many methods have been proposed for generating supervoxels. To effectively evaluate these methods, a novel supervoxel library and benchmark called LIBSVX with seven collected metrics was recently established. In this paper, we propose a new compactness metric which measures the shape regularity of supervoxels and is served as a necessary complement to the existing metrics. To demonstrate its necessity, we first explore the relations between the new metric and existing ones. Correlation analysis shows that the new metric has a weak correlation with (i.e., nearly independent of) existing metrics, and so reflects a new characteristic of supervoxel quality. Second, we investigate two real-world video applications. Experimental results show that the new metric can effectively predict some important application performance, while most existing metrics cannot do so

Feature-aware uniform tessellations on video manifold for content-sensitive supervoxels

Over-segmenting a video into supervoxels has strong potential to reduce the complexity of computer vision applications. Content-sensitive supervoxels (CSS) are typically smaller in content-dense regionsand larger in content-sparse regions. In this paper, we propose to compute feature-aware CSS (FCSS) that are regularly shaped 3D primitive volumes well aligned with local object/region/motion boundaries in video.To compute FCSS, we map a video to a 3-dimensional manifold, in which the volume elements of video manifold give a good measure of the video content density. Then any uniform tessellation on manifold can induce CSS. Our idea is that among all possible uniform tessellations, FCSS find one whose cell boundaries well align with local video boundaries. To achieve this goal, we propose a novel tessellation method that simultaneously minimizes the tessellation energy and maximizes the average boundary distance.Theoretically our method has an optimal competitive ratio O(1). We also present a simple extension of FCSS to streaming FCSS for processing long videos that cannot be loaded into main memory at once. We evaluate FCSS, streaming FCSS and ten representative supervoxel methods on four video datasets and two novel video applications. The results show that our method simultaneously achieves state-of-the-art performance with respect to various evaluation criteria

Fast global interactive volume segmentation with regional supervoxel descriptors

In this paper we propose a novel approach towards fast multi-class volume segmentation that exploits supervoxels in order to reduce complexity, time and memory requirements. Current methods for biomedical image segmentation typically require either complex mathematical models with slow convergence, or expensive-to-calculate image features, which makes them non-feasible for large volumes with many objects (tens to hundreds) of different classes, as is typical in modern medical and biological datasets. Recently, graphical models such as Markov Random Fields (MRF) or Conditional Random Fields (CRF) are having a huge impact in different computer vision areas (e.g. image parsing, object detection, object recognition) as they provide global regularization for multiclass problems over an energy minimization framework. These models have yet to find impact in biomedical imaging due to complexities in training and slow inference in 3D images due to the very large number of voxels. Here, we define an interactive segmentation approach over a supervoxel space by first defining novel, robust and fast regional descriptors for supervoxels. Then, a hierarchical segmentation approach is adopted by training Contextual Extremely Random Forests in a user-defined label hierarchy where the classification output of the previous layer is used as additional features to train a new classifier to refine more detailed label information. This hierarchical model yields final class likelihoods for supervoxels which are finally refined by a MRF model for 3D segmentation. Results demonstrate the effectiveness on a challenging cryo-soft X-ray tomography dataset by segmenting cell areas with only a few user scribbles as the input for our algorithm. Further results demonstrate the effectiveness of our method to fully extract different organelles from the cell volume with another few seconds of user interaction. © (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only

Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels

BACKGROUND: Accurate segmentation of brain tumour in magnetic resonance images (MRI) is a difficult task due to various tumour types. Using information and features from multimodal MRI including structural MRI and isotropic (p) and anisotropic (q) components derived from the diffusion tensor imaging (DTI) may result in a more accurate analysis of brain images. METHODS: We propose a novel 3D supervoxel based learning method for segmentation of tumour in multimodal MRI brain images (conventional MRI and DTI). Supervoxels are generated using the information across the multimodal MRI dataset. For each supervoxel, a variety of features including histograms of texton descriptor, calculated using a set of Gabor filters with different sizes and orientations, and first order intensity statistical features are extracted. Those features are fed into a random forests (RF) classifier to classify each supervoxel into tumour core, oedema or healthy brain tissue. RESULTS: The method is evaluated on two datasets: 1) Our clinical dataset: 11 multimodal images of patients and 2) BRATS 2013 clinical dataset: 30 multimodal images. For our clinical dataset, the average detection sensitivity of tumour (including tumour core and oedema) using multimodal MRI is 86% with balanced error rate (BER) 7%; while the Dice score for automatic tumour segmentation against ground truth is 0.84. The corresponding results of the BRATS 2013 dataset are 96%, 2% and 0.89, respectively. CONCLUSION: The method demonstrates promising results in the segmentation of brain tumour. Adding features from multimodal MRI images can largely increase the segmentation accuracy. The method provides a close match to expert delineation across all tumour grades, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management

Human Pose Estimation with Supervoxels

This thesis investigates how segmentation as a preprocessing step can reduce both the search space as well as complexity of human pose estimation in the context of smart environments. A 3D reconstruction is computed with a voxel carving algorithm. Based on a superpixel algorithm, these voxels are segmented into supervoxels that are then applied to pictorial structures in 3D to efficiently estimate the human pose. Both static and dynamic gesture recognition applications were developed

Human Pose Estimation with Supervoxels

This thesis investigates how segmentation as a preprocessing step can reduce both the search space as well as complexity of human pose estimation in the context of smart environments. A 3D reconstruction is computed with a voxel carving algorithm. Based on a superpixel algorithm, these voxels are segmented into supervoxels that are then applied to pictorial structures in 3D to efficiently estimate the human pose. Both static and dynamic gesture recognition applications were developed