Information Acquisition with Sensing Robots: Algorithms and Error Bounds

Utilizing the capabilities of configurable sensing systems requires addressing difficult information gathering problems. Near-optimal approaches exist for sensing systems without internal states. However, when it comes to optimizing the trajectories of mobile sensors the solutions are often greedy and rarely provide performance guarantees. Notably, under linear Gaussian assumptions, the problem becomes deterministic and can be solved off-line. Approaches based on submodularity have been applied by ignoring the sensor dynamics and greedily selecting informative locations in the environment. This paper presents a non-greedy algorithm with suboptimality guarantees, which does not rely on submodularity and takes the sensor dynamics into account. Our method performs provably better than the widely used greedy one. Coupled with linearization and model predictive control, it can be used to generate adaptive policies for mobile sensors with non-linear sensing models. Applications in gas concentration mapping and target tracking are presented.Comment: 9 pages (two-column); 2 figures; Manuscript submitted to the 2014 IEEE International Conference on Robotics and Automatio

Localization from semantic observations via the matrix permanent

Most approaches to robot localization rely on low-level geometric features such as points, lines, and planes. In this paper, we use object recognition to obtain semantic information from the robot’s sensors and consider the task of localizing the robot within a prior map of landmarks, which are annotated with semantic labels. As object recognition algorithms miss detections and produce false alarms, correct data association between the detections and the landmarks on the map is central to the semantic localization problem. Instead of the traditional vector-based representation, we propose a sensor model, which encodes the semantic observations via random finite sets and enables a unified treatment of missed detections, false alarms, and data association. Our second contribution is to reduce the problem of computing the likelihood of a set-valued observation to the problem of computing a matrix permanent. It is this crucial transformation that allows us to solve the semantic localization problem with a polynomial-time approximation to the set-based Bayes filter. Finally, we address the active semantic localization problem, in which the observer’s trajectory is planned in order to improve the accuracy and efficiency of the localization process. The performance of our approach is demonstrated in simulation and in real environments using deformable-part-model-based object detectors. Robust global localization from semantic observations is demonstrated for a mobile robot, for the Project Tango phone, and on the KITTI visual odometry dataset. Comparisons are made with the traditional lidar-based geometric Monte Carlo localization

Recognizing Fine-Grained Object Instances for Robotics Applications

State-of-the-art object recognition systems, and computer vision methods in general, are getting better and better at a variety of tasks. Many of these techniques focus on general object categories, like person or bottle, and use many hundreds or thousands of training examples per category. Some domains, such as robotics, require recognition of fine-grained object instances, such as a 16oz. bottle of Coke, and only have access to limited training data. In this work we look at how to build data and adapt successful computer vision techniques for fine-grained object recognition tasks. Our Active Vision Dataset is the first deep-learning scale dataset for instance detection and motion simulation using real images. Our Target Driven Instance Detection method uses a siamese convolutional neural network to focus on specific fine-grained object instances, and has the capacity to generalize to instances outside its training set. Our SymGan method used the Generative Adversarial Network paradigm to train a 3D object orientation predictor that is robust to object symmetries.Doctor of Philosoph

Controlled Recognition Bounds for Visual Learning and Exploration

We describe the tradeoff between the performance in a visual recognition problem and the control authority that the agent can exercise on the sensing process. We focus on the problem of “visual search ” of an object in an otherwise known and static scene, propose a measure of control authority, and relate it to the expected risk and its proxy (conditional entropy of the posterior density). We show this analytically, as well as empirically by simulation using the simplest known model that captures the phenomenology of image formation, including scaling and occlusions. We show that a “passive ” agent given a training set can provide no guarantees on performance beyond what is afforded by the priors, and that an “omnipotent ” agent, capable of infinite control authority, can achieve arbitrarily good performance (asymptotically). In between these limiting cases, the tradeoff can be characterized empirically.