Moment-Sum-Of-Squares Approach For Fast Risk Estimation In Uncertain Environments

In this paper, we address the risk estimation problem where one aims at estimating the probability of violation of safety constraints for a robot in the presence of bounded uncertainties with arbitrary probability distributions. In this problem, an unsafe set is described by level sets of polynomials that is, in general, a non-convex set. Uncertainty arises due to the probabilistic parameters of the unsafe set and probabilistic states of the robot. To solve this problem, we use a moment-based representation of probability distributions. We describe upper and lower bounds of the risk in terms of a linear weighted sum of the moments. Weights are coefficients of a univariate Chebyshev polynomial obtained by solving a sum-of-squares optimization problem in the offline step. Hence, given a finite number of moments of probability distributions, risk can be estimated in real-time. We demonstrate the performance of the provided approach by solving probabilistic collision checking problems where we aim to find the probability of collision of a robot with a non-convex obstacle in the presence of probabilistic uncertainties in the location of the robot and size, location, and geometry of the obstacle.Comment: 57th IEEE Conference on Decision and Control 201

Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a model-based RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into long-term predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.Comment: Accepted at AISTATS 2018