49,592 research outputs found
Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing
Within the context of autonomous driving a model-based reinforcement learning
algorithm is proposed for the design of neural network-parameterized
controllers. Classical model-based control methods, which include sampling- and
lattice-based algorithms and model predictive control, suffer from the
trade-off between model complexity and computational burden required for the
online solution of expensive optimization or search problems at every short
sampling time. To circumvent this trade-off, a 2-step procedure is motivated:
first learning of a controller during offline training based on an arbitrarily
complicated mathematical system model, before online fast feedforward
evaluation of the trained controller. The contribution of this paper is the
proposition of a simple gradient-free and model-based algorithm for deep
reinforcement learning using task separation with hill climbing (TSHC). In
particular, (i) simultaneous training on separate deterministic tasks with the
purpose of encoding many motion primitives in a neural network, and (ii) the
employment of maximally sparse rewards in combination with virtual velocity
constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl
Sequential Design for Optimal Stopping Problems
We propose a new approach to solve optimal stopping problems via simulation.
Working within the backward dynamic programming/Snell envelope framework, we
augment the methodology of Longstaff-Schwartz that focuses on approximating the
stopping strategy. Namely, we introduce adaptive generation of the stochastic
grids anchoring the simulated sample paths of the underlying state process.
This allows for active learning of the classifiers partitioning the state space
into the continuation and stopping regions. To this end, we examine sequential
design schemes that adaptively place new design points close to the stopping
boundaries. We then discuss dynamic regression algorithms that can implement
such recursive estimation and local refinement of the classifiers. The new
algorithm is illustrated with a variety of numerical experiments, showing that
an order of magnitude savings in terms of design size can be achieved. We also
compare with existing benchmarks in the context of pricing multi-dimensional
Bermudan options.Comment: 24 page
The role of learning on industrial simulation design and analysis
The capability of modeling real-world system operations has turned simulation into an indispensable problemsolving methodology for business system design and analysis. Today, simulation supports decisions ranging
from sourcing to operations to finance, starting at the strategic level and proceeding towards tactical and
operational levels of decision-making. In such a dynamic setting, the practice of simulation goes beyond
being a static problem-solving exercise and requires integration with learning. This article discusses the role
of learning in simulation design and analysis motivated by the needs of industrial problems and describes
how selected tools of statistical learning can be utilized for this purpose
Multilevel Double Loop Monte Carlo and Stochastic Collocation Methods with Importance Sampling for Bayesian Optimal Experimental Design
An optimal experimental set-up maximizes the value of data for statistical
inferences and predictions. The efficiency of strategies for finding optimal
experimental set-ups is particularly important for experiments that are
time-consuming or expensive to perform. For instance, in the situation when the
experiments are modeled by Partial Differential Equations (PDEs), multilevel
methods have been proven to dramatically reduce the computational complexity of
their single-level counterparts when estimating expected values. For a setting
where PDEs can model experiments, we propose two multilevel methods for
estimating a popular design criterion known as the expected information gain in
simulation-based Bayesian optimal experimental design. The expected information
gain criterion is of a nested expectation form, and only a handful of
multilevel methods have been proposed for problems of such form. We propose a
Multilevel Double Loop Monte Carlo (MLDLMC), which is a multilevel strategy
with Double Loop Monte Carlo (DLMC), and a Multilevel Double Loop Stochastic
Collocation (MLDLSC), which performs a high-dimensional integration by
deterministic quadrature on sparse grids. For both methods, the Laplace
approximation is used for importance sampling that significantly reduces the
computational work of estimating inner expectations. The optimal values of the
method parameters are determined by minimizing the average computational work,
subject to satisfying the desired error tolerance. The computational
efficiencies of the methods are demonstrated by estimating the expected
information gain for Bayesian inference of the fiber orientation in composite
laminate materials from an electrical impedance tomography experiment. MLDLSC
performs better than MLDLMC when the regularity of the quantity of interest,
with respect to the additive noise and the unknown parameters, can be
exploited
- …