6,423 research outputs found
Real-time information processing of environmental sensor network data using Bayesian Gaussian processes
In this article, we consider the problem faced by a sensor network operator who must infer, in real time, the value of some environmental parameter that is being monitored at discrete points in space and time by a sensor network. We describe a powerful and generic approach built upon an efficient multi-output Gaussian process that facilitates this information acquisition and processing. Our algorithm allows effective inference even with minimal domain knowledge, and we further introduce a formulation of Bayesian Monte Carlo to permit the principled management of the hyperparameters introduced by our flexible models. We demonstrate how our methods can be applied in cases where the data is delayed, intermittently missing, censored, and/or correlated. We validate our approach using data collected from three networks of weather sensors and show that it yields better inference performance than both conventional independent Gaussian processes and the Kalman filter. Finally, we show that our formalism efficiently reuses previous computations by following an online update procedure as new data sequentially arrives, and that this results in a four-fold increase in computational speed in the largest cases considered
Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics
The most data-efficient algorithms for reinforcement learning in robotics are
model-based policy search algorithms, which alternate between learning a
dynamical model of the robot and optimizing a policy to maximize the expected
return given the model and its uncertainties. Among the few proposed
approaches, the recently introduced Black-DROPS algorithm exploits a black-box
optimization algorithm to achieve both high data-efficiency and good
computation times when several cores are used; nevertheless, like all
model-based policy search approaches, Black-DROPS does not scale to high
dimensional state/action spaces. In this paper, we introduce a new model
learning procedure in Black-DROPS that leverages parameterized black-box priors
to (1) scale up to high-dimensional systems, and (2) be robust to large
inaccuracies of the prior information. We demonstrate the effectiveness of our
approach with the "pendubot" swing-up task in simulation and with a physical
hexapod robot (48D state space, 18D action space) that has to walk forward as
fast as possible. The results show that our new algorithm is more
data-efficient than previous model-based policy search algorithms (with and
without priors) and that it can allow a physical 6-legged robot to learn new
gaits in only 16 to 30 seconds of interaction time.Comment: Accepted at ICRA 2018; 8 pages, 4 figures, 2 algorithms, 1 table;
Video at https://youtu.be/HFkZkhGGzTo ; Spotlight ICRA presentation at
https://youtu.be/_MZYDhfWeL
Chance, long tails, and inference: a non-Gaussian, Bayesian theory of vocal learning in songbirds
Traditional theories of sensorimotor learning posit that animals use sensory
error signals to find the optimal motor command in the face of Gaussian sensory
and motor noise. However, most such theories cannot explain common behavioral
observations, for example that smaller sensory errors are more readily
corrected than larger errors and that large abrupt (but not gradually
introduced) errors lead to weak learning. Here we propose a new theory of
sensorimotor learning that explains these observations. The theory posits that
the animal learns an entire probability distribution of motor commands rather
than trying to arrive at a single optimal command, and that learning arises via
Bayesian inference when new sensory information becomes available. We test this
theory using data from a songbird, the Bengalese finch, that is adapting the
pitch (fundamental frequency) of its song following perturbations of auditory
feedback using miniature headphones. We observe the distribution of the sung
pitches to have long, non-Gaussian tails, which, within our theory, explains
the observed dynamics of learning. Further, the theory makes surprising
predictions about the dynamics of the shape of the pitch distribution, which we
confirm experimentally
- …