28,055 research outputs found
Machine Learning for Fluid Mechanics
The field of fluid mechanics is rapidly advancing, driven by unprecedented
volumes of data from field measurements, experiments and large-scale
simulations at multiple spatiotemporal scales. Machine learning offers a wealth
of techniques to extract information from data that could be translated into
knowledge about the underlying fluid mechanics. Moreover, machine learning
algorithms can augment domain knowledge and automate tasks related to flow
control and optimization. This article presents an overview of past history,
current developments, and emerging opportunities of machine learning for fluid
mechanics. It outlines fundamental machine learning methodologies and discusses
their uses for understanding, modeling, optimizing, and controlling fluid
flows. The strengths and limitations of these methods are addressed from the
perspective of scientific inquiry that considers data as an inherent part of
modeling, experimentation, and simulation. Machine learning provides a powerful
information processing framework that can enrich, and possibly even transform,
current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202
VIME: Variational Information Maximizing Exploration
Scalable and effective exploration remains a key challenge in reinforcement
learning (RL). While there are methods with optimality guarantees in the
setting of discrete state and action spaces, these methods cannot be applied in
high-dimensional deep RL scenarios. As such, most contemporary RL relies on
simple heuristics such as epsilon-greedy exploration or adding Gaussian noise
to the controls. This paper introduces Variational Information Maximizing
Exploration (VIME), an exploration strategy based on maximization of
information gain about the agent's belief of environment dynamics. We propose a
practical implementation, using variational inference in Bayesian neural
networks which efficiently handles continuous state and action spaces. VIME
modifies the MDP reward function, and can be applied with several different
underlying RL algorithms. We demonstrate that VIME achieves significantly
better performance compared to heuristic exploration methods across a variety
of continuous control tasks and algorithms, including tasks with very sparse
rewards.Comment: Published in Advances in Neural Information Processing Systems 29
(NIPS), pages 1109-111
Bayesian model predictive control: Efficient model exploration and regret bounds using posterior sampling
Tight performance specifications in combination with operational constraints
make model predictive control (MPC) the method of choice in various industries.
As the performance of an MPC controller depends on a sufficiently accurate
objective and prediction model of the process, a significant effort in the MPC
design procedure is dedicated to modeling and identification. Driven by the
increasing amount of available system data and advances in the field of machine
learning, data-driven MPC techniques have been developed to facilitate the MPC
controller design. While these methods are able to leverage available data,
they typically do not provide principled mechanisms to automatically trade off
exploitation of available data and exploration to improve and update the
objective and prediction model. To this end, we present a learning-based MPC
formulation using posterior sampling techniques, which provides finite-time
regret bounds on the learning performance while being simple to implement using
off-the-shelf MPC software and algorithms. The performance analysis of the
method is based on posterior sampling theory and its practical efficiency is
illustrated using a numerical example of a highly nonlinear dynamical
car-trailer system
Performance-oriented model learning for data-driven MPC design
Model Predictive Control (MPC) is an enabling technology in applications
requiring controlling physical processes in an optimized way under constraints
on inputs and outputs. However, in MPC closed-loop performance is pushed to the
limits only if the plant under control is accurately modeled; otherwise, robust
architectures need to be employed, at the price of reduced performance due to
worst-case conservative assumptions. In this paper, instead of adapting the
controller to handle uncertainty, we adapt the learning procedure so that the
prediction model is selected to provide the best closed-loop performance. More
specifically, we apply for the first time the above "identification for
control" rationale to hierarchical MPC using data-driven methods and Bayesian
optimization.Comment: Accepted for publication in the IEEE Control Systems Letters (L-CSS
Task-Driven Estimation and Control via Information Bottlenecks
Our goal is to develop a principled and general algorithmic framework for
task-driven estimation and control for robotic systems. State-of-the-art
approaches for controlling robotic systems typically rely heavily on accurately
estimating the full state of the robot (e.g., a running robot might estimate
joint angles and velocities, torso state, and position relative to a goal).
However, full state representations are often excessively rich for the specific
task at hand and can lead to significant computational inefficiency and
brittleness to errors in state estimation. In contrast, we present an approach
that eschews such rich representations and seeks to create task-driven
representations. The key technical insight is to leverage the theory of
information bottlenecks}to formalize the notion of a "task-driven
representation" in terms of information theoretic quantities that measure the
minimality of a representation. We propose novel iterative algorithms for
automatically synthesizing (offline) a task-driven representation (given in
terms of a set of task-relevant variables (TRVs)) and a performant control
policy that is a function of the TRVs. We present online algorithms for
estimating the TRVs in order to apply the control policy. We demonstrate that
our approach results in significant robustness to unmodeled measurement
uncertainty both theoretically and via thorough simulation experiments
including a spring-loaded inverted pendulum running to a goal location.Comment: 9 pages, 4 figures, abridged version accepted to ICRA2019;
Incorporates changes in final conference submissio
An Empirical Bayes Approach for Distributed Estimation of Spatial Fields
In this paper we consider a network of spatially distributed sensors which
collect measurement samples of a spatial field, and aim at estimating in a
distributed way (without any central coordinator) the entire field by suitably
fusing all network data. We propose a general probabilistic model that can
handle both partial knowledge of the physics generating the spatial field as
well as a purely data-driven inference. Specifically, we adopt an Empirical
Bayes approach in which the spatial field is modeled as a Gaussian Process,
whose mean function is described by means of parametrized equations. We
characterize the Empirical Bayes estimator when nodes are heterogeneous, i.e.,
perform a different number of measurements. Moreover, by exploiting the
sparsity of both the covariance and the (parametrized) mean function of the
Gaussian Process, we are able to design a distributed spatial field estimator.
We corroborate the theoretical results with two numerical simulations: a
stationary temperature field estimation in which the field is described by a
partial differential (heat) equation, and a data driven inference in which the
mean is parametrized by a cubic spline
- …