597 research outputs found
How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equations
We present a framework and algorithms to learn controlled dynamics models
using neural stochastic differential equations (SDEs) -- SDEs whose drift and
diffusion terms are both parametrized by neural networks. We construct the
drift term to leverage a priori physics knowledge as inductive bias, and we
design the diffusion term to represent a distance-aware estimate of the
uncertainty in the learned model's predictions -- it matches the system's
underlying stochasticity when evaluated on states near those from the training
dataset, and it predicts highly stochastic dynamics when evaluated on states
beyond the training regime. The proposed neural SDEs can be evaluated quickly
enough for use in model predictive control algorithms, or they can be used as
simulators for model-based reinforcement learning. Furthermore, they make
accurate predictions over long time horizons, even when trained on small
datasets that cover limited regions of the state space. We demonstrate these
capabilities through experiments on simulated robotic systems, as well as by
using them to model and control a hexacopter's flight dynamics: A neural SDE
trained using only three minutes of manually collected flight data results in a
model-based control policy that accurately tracks aggressive trajectories that
push the hexacopter's velocity and Euler angles to nearly double the maximum
values observed in the training dataset.Comment: Final submission to CoRL 202
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
We develop a new Low-level, First-order Probabilistic Programming Language
(LF-PPL) suited for models containing a mix of continuous, discrete, and/or
piecewise-continuous variables. The key success of this language and its
compilation scheme is in its ability to automatically distinguish parameters
the density function is discontinuous with respect to, while further providing
runtime checks for boundary crossings. This enables the introduction of new
inference engines that are able to exploit gradient information, while
remaining efficient for models which are not everywhere differentiable. We
demonstrate this ability by incorporating a discontinuous Hamiltonian Monte
Carlo (DHMC) inference engine that is able to deliver automated and efficient
inference for non-differentiable models. Our system is backed up by a
mathematical formalism that ensures that any model expressed in this language
has a density with measure zero discontinuities to maintain the validity of the
inference engine.Comment: Published in the proceedings of the 22nd International Conference on
Artificial Intelligence and Statistics (AISTATS
Autonomous Exploration over Continuous Domains
Motion planning is an essential aspect of robot autonomy, and as such it has been studied for decades, producing a wide range of planning methodologies. Path planners are generally categorised as either trajectory optimisers or sampling-based planners. The latter is the predominant planning paradigm as it can resolve a path efficiently while explicitly reasoning about path safety. Yet, with a limited budget, the resulting paths are far from optimal. In contrast, state-of-the-art trajectory optimisers explicitly trade-off between path safety and efficiency to produce locally optimal paths. However, these planners cannot incorporate updates from a partially observed model such as an occupancy map and fail in planning around information gaps caused by incomplete sensor coverage. Autonomous exploration adds another twist to path planning. The objective of exploration is to safely and efficiently traverse through an unknown environment in order to map it. The desired output of such a process is a sequence of paths that efficiently and safely minimise the uncertainty of the map. However, optimising over the entire space of trajectories is computationally intractable. Therefore, most exploration algorithms relax the general formulation by optimising a simpler one, for example finding the single next best view, resulting in suboptimal performance. This thesis investigates methodologies for optimal and safe exploration over continuous paths. Contrary to existing exploration algorithms that break exploration into independent sub-problems of finding goal points and planning safe paths to these points, our holistic approach simultaneously optimises the coupled problems of where and how to explore. Thus, offering a shift in paradigm from next best view to next best path. With exploration defined as an optimisation problem over continuous paths, this thesis explores two different optimisation paradigms; Bayesian and functional
Sampling constrained probability distributions using Spherical Augmentation
Statistical models with constrained probability distributions are abundant in
machine learning. Some examples include regression models with norm constraints
(e.g., Lasso), probit, many copula models, and latent Dirichlet allocation
(LDA). Bayesian inference involving probability distributions confined to
constrained domains could be quite challenging for commonly used sampling
algorithms. In this paper, we propose a novel augmentation technique that
handles a wide range of constraints by mapping the constrained domain to a
sphere in the augmented space. By moving freely on the surface of this sphere,
sampling algorithms handle constraints implicitly and generate proposals that
remain within boundaries when mapped back to the original space. Our proposed
method, called {Spherical Augmentation}, provides a mathematically natural and
computationally efficient framework for sampling from constrained probability
distributions. We show the advantages of our method over state-of-the-art
sampling algorithms, such as exact Hamiltonian Monte Carlo, using several
examples including truncated Gaussian distributions, Bayesian Lasso, Bayesian
bridge regression, reconstruction of quantized stationary Gaussian process, and
LDA for topic modeling.Comment: 41 pages, 13 figure
Data-Driven Passivity-Based Control of Underactuated Robotic Systems
Classical control strategies for robotic systems are based on the idea that feedback control can be used to override the natural dynamics of the machines. Passivity-based control (Pbc) is a branch of nonlinear control theory that follows a similar approach, where the natural dynamics is modified based on the overall energy of the system. This method involves transforming a nonlinear control system, through a suitable control input, into another fictitious system that has desirable stability characteristics. The majority of Pbc techniques require the discovery of a reasonable storage function, which acts as a Lyapunov function candidate that can be used to certify stability.
There are several challenges in the design of a suitable storage function, including: 1) what a reasonable choice for the function is for a given control system, and 2) the control synthesis requires a closed-form solution to a set of nonlinear partial differential equations. The latter is in general difficult to overcome, especially for systems with high degrees of freedom, limiting the applicability of Pbc techniques.
A machine learning framework that automatically determines the storage function for underactuated robotic systems is introduced in this dissertation. This framework combines the expressive power of neural networks with the systematic methods of the Pbc paradigm, bridging the gap between controllers derived from learning algorithms and nonlinear control theory. A series of experiments demonstrates the efficacy and applicability of this framework for a family of underactuated robots
Recommended from our members
Game-Theoretic Safety Assurance for Human-Centered Robotic Systems
In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence
Probabilistic models for data efficient reinforcement learning
Trial-and-error based reinforcement learning (RL) has seen rapid advancements
in recent times, especially with the advent of deep neural networks. However, the
standard deep learning methods often overlook the progress made in control theory
by treating systems as black-box. We propose a model-based RL framework based
on probabilistic Model Predictive Control (MPC). In particular, we propose to learn
a probabilistic transition model using Gaussian Processes (GPs) to incorporate model
uncertainty into long-term predictions, thereby, reducing the impact of model errors. We
provide theoretical guarantees for first-order optimality in the GP-based transition models
with deterministic approximate inference for long-term planning. We demonstrate that
our approach not only achieves the state-of-the-art data efficiency, but also is a principled
way for RL in constrained environments.
When the true state of the dynamical system cannot be fully observed the standard
model based methods cannot be directly applied. For these systems an additional step of
state estimation is needed. We propose distributed message passing for state estimation in
non-linear dynamical systems. In particular, we propose to use expectation propagation
(EP) to iteratively refine the state estimate, i.e., the Gaussian posterior distribution on the
latent state. We show two things: (a) Classical Rauch-Tung-Striebel (RTS) smoothers,
such as the extended Kalman smoother (EKS) or the unscented Kalman smoother (UKS),
are special cases of our message passing scheme; (b) running the message passing
scheme more than once can lead to significant improvements over the classical RTS
smoothers. We show the explicit connection between message passing with EP and
well-known RTS smoothers and provide a practical implementation of the suggested
algorithm. Furthermore, we address convergence issues of EP by generalising this
framework to damped updates and the consideration of general -divergences.
Probabilistic models can also be used to generate synthetic data. In model based RL
we use ’synthetic’ data as a proxy to real environments and in order to achieve high data
efficiency. The ability to generate high-fidelity synthetic data is crucial when available
(real) data is limited as in RL or where privacy and data protection standards allow
only for limited use of the given data, e.g., in medical and financial data-sets. Current
state-of-the-art methods for synthetic data generation are based on generative models,
such as Generative Adversarial Networks (GANs). Even though GANs have achieved
remarkable results in synthetic data generation, they are often challenging to interpret.
Furthermore, GAN-based methods can suffer when used with mixed real and categorical
variables. Moreover, the loss function (discriminator loss) design itself is problem
specific, i.e., the generative model may not be useful for tasks it was not explicitly trained
for. In this paper, we propose to use a probabilistic model as a synthetic data generator.
Learning the probabilistic model for the data is equivalent to estimating the density of
the data. Based on the copula theory, we divide the density estimation task into two parts,
i.e., estimating univariate marginals and estimating the multivariate copula density over
the univariate marginals. We use normalising flows to learn both the copula density and
univariate marginals. We benchmark our method on both simulated and real data-sets in
terms of density estimation as well as the ability to generate high-fidelity synthetic data.Open Acces
- …