Localization from semantic observations via the matrix permanent

Most approaches to robot localization rely on low-level geometric features such as points, lines, and planes. In this paper, we use object recognition to obtain semantic information from the robot’s sensors and consider the task of localizing the robot within a prior map of landmarks, which are annotated with semantic labels. As object recognition algorithms miss detections and produce false alarms, correct data association between the detections and the landmarks on the map is central to the semantic localization problem. Instead of the traditional vector-based representation, we propose a sensor model, which encodes the semantic observations via random finite sets and enables a unified treatment of missed detections, false alarms, and data association. Our second contribution is to reduce the problem of computing the likelihood of a set-valued observation to the problem of computing a matrix permanent. It is this crucial transformation that allows us to solve the semantic localization problem with a polynomial-time approximation to the set-based Bayes filter. Finally, we address the active semantic localization problem, in which the observer’s trajectory is planned in order to improve the accuracy and efficiency of the localization process. The performance of our approach is demonstrated in simulation and in real environments using deformable-part-model-based object detectors. Robust global localization from semantic observations is demonstrated for a mobile robot, for the Project Tango phone, and on the KITTI visual odometry dataset. Comparisons are made with the traditional lidar-based geometric Monte Carlo localization

Active Information Acquisition With Mobile Robots

The recent proliferation of sensors and robots has potential to transform fields as diverse as environmental monitoring, security and surveillance, localization and mapping, and structure inspection. One of the great technical challenges in these scenarios is to control the sensors and robots in order to extract accurate information about various physical phenomena autonomously. The goal of this dissertation is to provide a unified approach for active information acquisition with a team of sensing robots. We formulate a decision problem for maximizing relevant information measures, constrained by the motion capabilities and sensing modalities of the robots, and focus on the design of a scalable control strategy for the robot team. The first part of the dissertation studies the active information acquisition problem in the special case of linear Gaussian sensing and mobility models. We show that the classical principle of separation between estimation and control holds in this case. It enables us to reduce the original stochastic optimal control problem to a deterministic version and to provide an optimal centralized solution. Unfortunately, the complexity of obtaining the optimal solution scales exponentially with the length of the planning horizon and the number of robots. We develop approximation algorithms to manage the complexity in both of these factors and provide theoretical performance guarantees. Applications in gas concentration mapping, joint localization and vehicle tracking in sensor networks, and active multi-robot localization and mapping are presented. Coupled with linearization and model predictive control, our algorithms can even generate adaptive control policies for nonlinear sensing and mobility models. Linear Gaussian information seeking, however, cannot be applied directly in the presence of sensing nuisances such as missed detections, false alarms, and ambiguous data association or when some sensor observations are discrete (e.g., object classes, medical alarms) or, even worse, when the sensing and target models are entirely unknown. The second part of the dissertation considers these complications in the context of two applications: active localization from semantic observations (e.g, recognized objects) and radio signal source seeking. The complexity of the target inference problem forces us to resort to greedy planning of the sensor trajectories. Non-greedy closed-loop information acquisition with general discrete models is achieved in the final part of the dissertation via dynamic programming and Monte Carlo tree search algorithms. Applications in active object recognition and pose estimation are presented. The techniques developed in this thesis offer an effective and scalable approach for controlled information acquisition with multiple sensing robots and have broad applications to environmental monitoring, search and rescue, security and surveillance, localization and mapping, precision agriculture, and structure inspection

Mutual information-based gradient-ascent control for distributed robotics

Thesis (Ph. D.)--Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 167-179).This thesis presents the derivation, analysis, and implementation of a novel class of decentralized mutual information-based gradient-ascent controllers that continuously move robots equipped with sensors to better observe their environment. We begin with the fundamental problem of deploying a single ground robot equipped with a range sensor and tasked to build an occupancy grid map. The desired explorative behaviors of the robot for occupancy grid mapping highlight the correlation between the information content and the spatial realization of the robot's range measurements. We prove that any occupancy grid controller tasked to maximize a mutual information reward function is eventually attracted to unexplored space, i.e., areas of highest uncertainty. We show that mutual information encodes geometric relationships that are fundamental to robot control and yields geometrically relevant reward surfaces on which robots can navigate. Taking inspiration from geometric-based approaches to distributed robot coordination, we show that many multi-robot inference tasks can be cast in terms of an optimization problem. This optimization problem defines the task of minimizing the conditional entropy associated with the robots' inferred beliefs of the environment, which is equivalent to maximizing the mutual information between the environment state and the robots' next joint observation. Given simple robot dynamics and few probabilistic assumptions, none of which involve Gaussianity, we derive a gradientascent solution approach to these optimization problems that is convergent between sensor observations and locally optimal. More formally, we invoke LaSalle's Invariance Principle to prove that, given enough time between consecutive joint observations, robots following the gradient of mutual information will converge to goal positions that locally maximize the expected information gain resulting from the next observation. We show that the algorithmic implementation of the generalized gradient-ascent controller is not readily distributed among multiple robots, and thus sample-based methods are introduced to distributively approximate the likelihoods of the robots' joint observations. Not only are the involved non-parametric representations compatible with any type of Bayesian filter, but the computational complexities of the resulting decentralized controllers are independent with respect to the number of robots. Concerning the distributed approximations, we give two example consensus-based algorithms that run on an undirected network graph. The first consensus-based algorithm approximates discrete measurement probabilities, while the second approximates continuous likelihood distributions. We show that these anytime approximations provably converge to the correct values on a static and connected network graph without knowledge of the number of robots in the network or the corresponding graph's topology. Lastly, we incorporate the resulting consensus-based algorithms into both a hardware system and a simulation environment to allow for decentralized controller evaluation under non-ideal network settings. For the hardware experiments, the task is to infer the state of a bounded, planar environment by deploying five quadrotor flying robots with simulated sensors in both indoor and outdoor settings. For the numerical simulations, Monte Carlo-based analyses are performed for 100 robots, where each robot is simulated on an independent computer node within a computer cluster system. Simulations are also performed for 1000 robots using a single workstation computer equipped with a multicore GPU-enabled graphics card. The results from both the hardware experiments and numerical simulations validate our theoretical and computational claims throughout the thesis.by Brian John Julian.Ph.D