704,879 research outputs found
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
Model-free deep reinforcement learning algorithms have been shown to be
capable of learning a wide range of robotic skills, but typically require a
very large number of samples to achieve good performance. Model-based
algorithms, in principle, can provide for much more efficient learning, but
have proven difficult to extend to expressive, high-capacity models such as
deep neural networks. In this work, we demonstrate that medium-sized neural
network models can in fact be combined with model predictive control (MPC) to
achieve excellent sample complexity in a model-based reinforcement learning
algorithm, producing stable and plausible gaits to accomplish various complex
locomotion tasks. We also propose using deep neural network dynamics models to
initialize a model-free learner, in order to combine the sample efficiency of
model-based approaches with the high task-specific performance of model-free
methods. We empirically demonstrate on MuJoCo locomotion tasks that our pure
model-based approach trained on just random action data can follow arbitrary
trajectories with excellent sample efficiency, and that our hybrid algorithm
can accelerate model-free learning on high-speed benchmark tasks, achieving
sample efficiency gains of 3-5x on swimmer, cheetah, hopper, and ant agents.
Videos can be found at https://sites.google.com/view/mbm
Discriminative Segmental Cascades for Feature-Rich Phone Recognition
Discriminative segmental models, such as segmental conditional random fields
(SCRFs) and segmental structured support vector machines (SSVMs), have had
success in speech recognition via both lattice rescoring and first-pass
decoding. However, such models suffer from slow decoding, hampering the use of
computationally expensive features, such as segment neural networks or other
high-order features. A typical solution is to use approximate decoding, either
by beam pruning in a single pass or by beam pruning to generate a lattice
followed by a second pass. In this work, we study discriminative segmental
models trained with a hinge loss (i.e., segmental structured SVMs). We show
that beam search is not suitable for learning rescoring models in this
approach, though it gives good approximate decoding performance when the model
is already well-trained. Instead, we consider an approach inspired by
structured prediction cascades, which use max-marginal pruning to generate
lattices. We obtain a high-accuracy phonetic recognition system with several
expensive feature types: a segment neural network, a second-order language
model, and second-order phone boundary features
Modeling of complex-valued Wiener systems using B-spline neural network
In this brief, a new complex-valued B-spline neural network is introduced in order to model the complex-valued Wiener system using observational input/output data. The complex-valued nonlinear static function in the Wiener system is represented using the tensor product from two univariate Bspline neural networks, using the real and imaginary parts of the system input. Following the use of a simple least squares parameter initialization scheme, the GaussāNewton algorithm is applied for the parameter estimation, which incorporates the De Boor algorithm, including both the B-spline curve and the first-order derivatives recursion. Numerical examples, including a nonlinear high-power amplifier model in communication systems, are used to demonstrate the efficacy of the proposed approaches
Deep convolutional neural networks for estimating porous material parameters with ultrasound tomography
We study the feasibility of data based machine learning applied to ultrasound
tomography to estimate water-saturated porous material parameters. In this
work, the data to train the neural networks is simulated by solving wave
propagation in coupled poroviscoelastic-viscoelastic-acoustic media. As the
forward model, we consider a high-order discontinuous Galerkin method while
deep convolutional neural networks are used to solve the parameter estimation
problem. In the numerical experiment, we estimate the material porosity and
tortuosity while the remaining parameters which are of less interest are
successfully marginalized in the neural networks-based inversion. Computational
examples confirms the feasibility and accuracy of this approach
Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
We introduce a data-driven forecasting method for high-dimensional chaotic
systems using long short-term memory (LSTM) recurrent neural networks. The
proposed LSTM neural networks perform inference of high-dimensional dynamical
systems in their reduced order space and are shown to be an effective set of
nonlinear approximators of their attractor. We demonstrate the forecasting
performance of the LSTM and compare it with Gaussian processes (GPs) in time
series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation
and a prototype climate model. The LSTM networks outperform the GPs in
short-term forecasting accuracy in all applications considered. A hybrid
architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is
proposed to ensure convergence to the invariant measure. This novel hybrid
method is fully data-driven and extends the forecasting capabilities of LSTM
networks.Comment: 31 page
- ā¦