Stop! planner time: metareasoning for probabilistic planning using learned performance profiles

The metareasoning framework aims to enable autonomous agents to factor in planning costs when making decisions. In this work, we develop the first non-myopic metareasoning algorithm for planning with Markov decision processes. Our method learns the behaviour of anytime probabilistic planning algorithms from performance data. Specifically, we propose a novel model for metareasoning, based on contextual performance profiles that predict the value of the planner’s current solution given the time spent planning, the state of the planning algorithm’s internal parameters, and the difficulty of the planning problem being solved. This model removes the need to assume that the current solution quality is always known, broadening the class of metareasoning problems that can be addressed. We then employ deep reinforcement learning to learn a policy that decides, at each timestep, whether to continue planning or start executing the current plan, and how to set hyperparameters of the planner to enhance its performance. We demonstrate our algorithm’s ability to perform effective metareasoning in two domains

Learning When to Quit: Meta-Reasoning for Motion Planning

Anytime motion planners are widely used in robotics. However, the relationship between their solution quality and computation time is not well understood, and thus, determining when to quit planning and start execution is unclear. In this paper, we address the problem of deciding when to stop deliberation under bounded computational capacity, so called meta-reasoning, for anytime motion planning. We propose data-driven learning methods, model-based and model-free meta-reasoning, that are applicable to different environment distributions and agnostic to the choice of anytime motion planners. As a part of the framework, we design a convolutional neural network-based optimal solution predictor that predicts the optimal path length from a given 2D workspace image. We empirically evaluate the performance of the proposed methods in simulation in comparison with baselines.Comment: 8 pages, 5 figures, Submitted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 202

Real-time Planning as Decision-making Under Uncertainty

In real-time planning, an agent must select the next action to take within a fixed time bound. Many popular real-time heuristic search methods approach this by expanding nodes using time-limited A* and selecting the action leading toward the frontier node with the lowest f value. In this thesis, we reconsider real-time planning as a problem of decision-making under uncertainty. We treat heuristic values as uncertain evidence and we explore several backup methods for aggregating this evidence. We then propose a novel lookahead strategy that expands nodes to minimize risk, the expected regret in case a non-optimal action is chosen. We evaluate these methods in a simple synthetic benchmark and the sliding tile puzzle and find that they outperform previous methods. This work illustrates how uncertainty can arise even when solving deterministic planning problems, due to the inherent ignorance of time-limited search algorithms about those portions of the state space that they have not computed, and how an agent can benefit from explicitly meta-reasoning about this uncertainty

Using Metareasoning on a Mobile Ground Robot to Recover from Path Planning Failures

Autonomous mobile ground robots use global and local path planners to determine the routes that they should follow to achieve mission goals while avoiding obstacles. Although many path planners have been developed, no single one is best for all situations. This paper describes metareasoning approaches that enable a robot to select a new path planning algorithm when the current planning algorithm cannot find a feasible solution. We implemented the approaches within a ROS-based autonomy stack and conducted simulation experiments to evaluate their performance in multiple scenarios. The results show that these metareasoning approaches reduce the frequency of failures and reduce the time required to complete the mission.This work was supported by the U.S. Army Research Laboratory (Award W911NF2120076)

Case based reasoning as a model for cognitive artificial intelligence.

Cognitive Systems understand the world through learning and experience. Case Based Reasoning (CBR) systems naturally capture knowledge as experiences in memory and they are able to learn new experiences to retain in their memory. CBR's retrieve and reuse reasoning is also knowledge-rich because of its nearest neighbour retrieval and analogy-based adaptation of retrieved solutions. CBR is particularly suited to domains where there is no well-defined theory, because they have a memory of experiences of what happened, rather than why/how it happened. CBR's assumption that 'similar problems have similar solutions' enables it to understand the contexts for its experiences and the 'bigger picture' from clusters of cases, but also where its similarity assumption is challenged. Here we explore cognition and meta-cognition for CBR through self-refl ection and introspection of both memory and retrieve and reuse reasoning. Our idea is to embed and exploit cognitive functionality such as insight, intuition and curiosity within CBR to drive robust, and even explainable, intelligence that will achieve problemsolving in challenging, complex, dynamic domains

Anomaly Detection for Symbolic Representations

A fully autonomous agent recognizes new problems, explains what causes such problems, and generates its own goals to solve these problems. Our approach to this goal-driven model of autonomy uses a methodology called the Note-Assess-Guide procedure. It instantiates a monitoring process in which an agent notes an anomaly in the world, assesses the nature and cause of that anomaly, and guides appropriate modifications to behavior. This report describes a novel approach to the note phase of that procedure. A-distance, a sliding-window statistical distance metric, is applied to numerical vector representations of intermediate states from plans generated for two symbolic domains. Using these representations, the metric is able to detect anomalous world states caused by restricting the actions available to the planner

Natively probabilistic computation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2009.Includes bibliographical references (leaves 129-135).I introduce a new set of natively probabilistic computing abstractions, including probabilistic generalizations of Boolean circuits, backtracking search and pure Lisp. I show how these tools let one compactly specify probabilistic generative models, generalize and parallelize widely used sampling algorithms like rejection sampling and Markov chain Monte Carlo, and solve difficult Bayesian inference problems. I first introduce Church, a probabilistic programming language for describing probabilistic generative processes that induce distributions, which generalizes Lisp, a language for describing deterministic procedures that induce functions. I highlight the ways randomness meshes with the reflectiveness of Lisp to support the representation of structured, uncertain knowledge, including nonparametric Bayesian models from the current literature, programs for decision making under uncertainty, and programs that learn very simple programs from data. I then introduce systematic stochastic search, a recursive algorithm for exact and approximate sampling that generalizes a popular form of backtracking search to the broader setting of stochastic simulation and recovers widely used particle filters as a special case. I use it to solve probabilistic reasoning problems from statistical physics, causal reasoning and stereo vision. Finally, I introduce stochastic digital circuits that model the probability algebra just as traditional Boolean circuits model the Boolean algebra.(cont.) I show how these circuits can be used to build massively parallel, fault-tolerant machines for sampling and allow one to efficiently run Markov chain Monte Carlo methods on models with hundreds of thousands of variables in real time. I emphasize the ways in which these ideas fit together into a coherent software and hardware stack for natively probabilistic computing, organized around distributions and samplers rather than deterministic functions. I argue that by building uncertainty and randomness into the foundations of our programming languages and computing machines, we may arrive at ones that are more powerful, flexible and efficient than deterministic designs, and are in better alignment with the needs of computational science, statistics and artificial intelligence.by Vikash Kumar Mansinghka.Ph.D

Effect of Emotion and Personality on Deviation from Purely Rational Decision-Making

Human decision-making has consistently demonstrated deviation from "pure" rationality. Emotions are a primary driver of human actions and the current study investigates how perceived emotions and personality traits may affect decision-making during the Ultimatum Game (UG). We manipulated emotions by showing images with emotional connotation while participants decided how to split money with a second player. Event-related potentials (ERPs) from scalp electrodes were recorded during the whole decision-making process. We observed significant differences in the activity of central and frontal areas when participants offered money with respect to when they accepted or rejected an offer. We found that participants were more likely to offer a higher amount of money when making their decision in association with negative emotions. Furthermore, participants were more likely to accept offers when making their decision in association with positive emotions. Honest, conscientious, and introverted participants were more likely to accept offers. Our results suggest that factors others than a rational strategy may predict economic decision-making in the UG