Salient object subitizing

We study the problem of salient object subitizing, i.e. predicting the existence and the number of salient objects in an image using holistic cues. This task is inspired by the ability of people to quickly and accurately identify the number of items within the subitizing range (1–4). To this end, we present a salient object subitizing image dataset of about 14 K everyday images which are annotated using an online crowdsourcing marketplace. We show that using an end-to-end trained convolutional neural network (CNN) model, we achieve prediction accuracy comparable to human performance in identifying images with zero or one salient object. For images with multiple salient objects, our model also provides significantly better than chance performance without requiring any localization process. Moreover, we propose a method to improve the training of the CNN subitizing model by leveraging synthetic images. In experiments, we demonstrate the accuracy and generalizability of our CNN subitizing model and its applications in salient object detection and image retrieval.This research was supported in part by US NSF Grants 0910908 and 1029430, and gifts from Adobe and NVIDIA. (0910908 - US NSF; 1029430 - US NSF)https://arxiv.org/abs/1607.07525https://arxiv.org/pdf/1607.07525.pdfAccepted manuscrip

Outdoor view recognition based on landmark grouping and logistic regression

Vision-based robot localization outdoors has remained more elusive than its indoors counterpart. Drastic illumination changes and the scarceness of suitable landmarks are the main difficulties. This paper attempts to surmount them by deviating from the main trend of using local features. Instead, a global descriptor called landmark-view is defined, which aggregates the most visually-salient landmarks present in each scene. Thus, landmark co-occurrence and spatial and saliency relationships between them are added to the single landmark characterization, based on saliency and color distribution. A suitable framework to compare landmark-views is developed, and it is shown how this remarkably enhances the recognition performance, compared against single landmark recognition. A view-matching model is constructed using logistic regression. Experimentation using 45 views, acquired outdoors, containing 273 landmarks, yielded good recognition results. The overall percentage of correct view classification obtained was 80.6%, indicating the adequacy of the approach.Peer ReviewedPostprint (author’s final draft

Unconstrained salient object detection via proposal subset optimization

We aim at detecting salient objects in unconstrained images. In unconstrained images, the number of salient objects (if any) varies from image to image, and is not given. We present a salient object detection system that directly outputs a compact set of detection windows, if any, for an input image. Our system leverages a Convolutional-Neural-Network model to generate location proposals of salient objects. Location proposals tend to be highly overlapping and noisy. Based on the Maximum a Posteriori principle, we propose a novel subset optimization framework to generate a compact set of detection windows out of noisy proposals. In experiments, we show that our subset optimization formulation greatly enhances the performance of our system, and our system attains 16-34% relative improvement in Average Precision compared with the state-of-the-art on three challenging salient object datasets.http://openaccess.thecvf.com/content_cvpr_2016/html/Zhang_Unconstrained_Salient_Object_CVPR_2016_paper.htmlPublished versio

Towards the Success Rate of One: Real-time Unconstrained Salient Object Detection

In this work, we propose an efficient and effective approach for unconstrained salient object detection in images using deep convolutional neural networks. Instead of generating thousands of candidate bounding boxes and refining them, our network directly learns to generate the saliency map containing the exact number of salient objects. During training, we convert the ground-truth rectangular boxes to Gaussian distributions that better capture the ROI regarding individual salient objects. During inference, the network predicts Gaussian distributions centered at salient objects with an appropriate covariance, from which bounding boxes are easily inferred. Notably, our network performs saliency map prediction without pixel-level annotations, salient object detection without object proposals, and salient object subitizing simultaneously, all in a single pass within a unified framework. Extensive experiments show that our approach outperforms existing methods on various datasets by a large margin, and achieves more than 100 fps with VGG16 network on a single GPU during inference

Color constancy for landmark detection in outdoor environments

European Workshop on Advanced Mobile Robots (EUROBOT), 2001, Lund (Suecia)This work presents an evaluation of three color constancy techniques applied to a landmark detection system designed for a walking robot, which has to operate in unknown and unstructured outdoor environments. The first technique is the well-known image conversion to a chromaticity space, and the second technique is based on successive lighting intensity and illuminant color normalizations. Based on a differential model of color constancy, we propose the third technique, based on color ratios, which unifies the processes of color constancy and landmark detection. The approach used to detect potential landmarks, which is common to all evaluated systems, is based on visual saliency concepts using multiscale color opponent features to identify salient regions in the images. These regions are selected as landmark candidates, and they are further characterized by their features for identification and recognition.This work was supported by the project 'Navegación autónoma de robots guiados por objetivos visuales' (070-720).Peer Reviewe

Detecting salient cues through illumination-invariant color ratios

This work presents a novel technique for embedding color constancy into a saliency-based system for detecting potential landmarks in outdoor environments. Since multiscale color opponencies are among the ingredients determining saliency, the idea is to make such opponencies directly invariant to illumination variations, rather than enforcing the invariance of colors themselves. The new technique is compared against the alternative approach of preprocessing the images with a color constancy procedure before entering the saliency system. The first procedure used in the experimental comparison is the well-known image conversion to chromaticity space, and the second one is based on successive lighting intensity and illuminant color normalizations. The proposed technique offers significant advantages over the preceding two ones since, at a lower computational cost, it exhibits higher stability in front of illumination variations and even of slight viewpoint changes, resulting in a better correspondence of visual saliency to potential landmark elements.This work was supported by the project 'Sistema reconfigurable para la navegación basada en visión de robots caminantes y rodantes en entornos naturales.' (00). The authors would like to the support obtained from the Forschungszentrum Informatik and Institut für Prozessrechentechnik, Automation und Robotik , Karlsruhe University, Germany. This work is partially supported by the Spanish Science and Technology Directorate, in the scope of the project “Reconfigurable system for vision-based navigation of legged and wheeled robots in natural environments (SIRVENT)”, grant DPI2003-05193-C02-01.Peer Reviewe