Saliency Methods for Object Discovery Based on Image and Depth Segmentation

Object discovery is a recent paradigm in computer and robotic vision where the process of interpreting an image starts by proposing a set of candidate regions that potentially correspond to objects; these candidates can be validated later on by object recognition modules or by robot interaction. In this thesis, we propose a novel method for object discovery that works on single RGB-D images and aims at achieving higher recall than current state-of-the-art methods with fewer candidates. Our approach uses saliency as a cue to roughly estimate the location and extent of the objects, and segmentation processes in order to identify the candidates' precise boundaries. We investigate the performance of four different segmentation methods based on colour, depth, an early and a late fusion of colour and depth, and conclude that the late fusion is the most successful. The object candidates are sorted according to a novel ranking strategy based on a combination of features such as 3D convexity and saliency. We evaluate our method and compare it to other state-of-the-art approaches in object discovery on challenging real world sequences from three different public datasets containing a high degree of clutter. The results show that our approach consistently outperforms the other methods. In the second part of this thesis, we turn to streams of images. Here, our goal is to generate as few object candidates per frame as necessary in order to find as many objects as possible throughout the sequence. Therefore, we propose to extend our object discovery system with a so called spatial inhibition of return mechanism to inhibit object candidates that correspond to objects that have already been generated in the past. The challenge here is to inhibit the candidates consistently with viewpoint change, and therefore, we root our inhibition of return mechanism in 3D spatial coordinates. In the final part of this thesis we show an application of our object discovery method to the task of salient object segmentation. The results show that our method achieves state-of-the-art performance

Saliency-guided Adaptive Seeding for Supervoxel Segmentation

We propose a new saliency-guided method for generating supervoxels in 3D space. Rather than using an evenly distributed spatial seeding procedure, our method uses visual saliency to guide the process of supervoxel generation. This results in densely distributed, small, and precise supervoxels in salient regions which often contain objects, and larger supervoxels in less salient regions that often correspond to background. Our approach largely improves the quality of the resulting supervoxel segmentation in terms of boundary recall and under-segmentation error on publicly available benchmarks.Comment: 6 pages, accepted to IROS201

Object Discovery From a Single Unlabeled Image by Mining Frequent Itemset With Multi-scale Features

TThe goal of our work is to discover dominant objects in a very general setting where only a single unlabeled image is given. This is far more challenge than typical co-localization or weakly-supervised localization tasks. To tackle this problem, we propose a simple but effective pattern mining-based method, called Object Location Mining (OLM), which exploits the advantages of data mining and feature representation of pre-trained convolutional neural networks (CNNs). Specifically, we first convert the feature maps from a pre-trained CNN model into a set of transactions, and then discovers frequent patterns from transaction database through pattern mining techniques. We observe that those discovered patterns, i.e., co-occurrence highlighted regions, typically hold appearance and spatial consistency. Motivated by this observation, we can easily discover and localize possible objects by merging relevant meaningful patterns. Extensive experiments on a variety of benchmarks demonstrate that OLM achieves competitive localization performance compared with the state-of-the-art methods. We also evaluate our approach compared with unsupervised saliency detection methods and achieves competitive results on seven benchmark datasets. Moreover, we conduct experiments on fine-grained classification to show that our proposed method can locate the entire object and parts accurately, which can benefit to improving the classification results significantly