46,035 research outputs found
Control of Complex Dynamic Systems by Neural Networks
This paper considers the use of neural networks (NN's) in controlling a nonlinear, stochastic system with unknown process equations. The NN is used to model the resulting unknown control law. The approach here is based on using the output error of the system to train the NN controller without the need to construct a separate model (NN or other type) for the unknown process dynamics. To implement such a direct adaptive control approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (back-propagation-type) weight estimation algorithms. Therefore, this paper considers the use of a new stochastic approximation algorithm for this weight estimation, which is based on a 'simultaneous perturbation' gradient approximation that only requires the system output error. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations
Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization
The stochastic gradient Langevin Dynamics is one of the most fundamental
algorithms to solve sampling problems and non-convex optimization appearing in
several machine learning applications. Especially, its variance reduced
versions have nowadays gained particular attention. In this paper, we study two
variants of this kind, namely, the Stochastic Variance Reduced Gradient
Langevin Dynamics and the Stochastic Recursive Gradient Langevin Dynamics. We
prove their convergence to the objective distribution in terms of KL-divergence
under the sole assumptions of smoothness and Log-Sobolev inequality which are
weaker conditions than those used in prior works for these algorithms. With the
batch size and the inner loop length set to , the gradient complexity
to achieve an -precision is
, which is an
improvement from any previous analyses. We also show some essential
applications of our result to non-convex optimization
Generalisation under gradient descent via deterministic PAC-Bayes
We establish disintegrated PAC-Bayesian generalisation bounds for models
trained with gradient descent methods or continuous gradient flows. Contrary to
standard practice in the PAC-Bayesian setting, our result applies to
optimisation algorithms that are deterministic, without requiring any
de-randomisation step. Our bounds are fully computable, depending on the
density of the initial distribution and the Hessian of the training objective
over the trajectory. We show that our framework can be applied to a variety of
iterative optimisation algorithms, including stochastic gradient descent (SGD),
momentum-based schemes, and damped Hamiltonian dynamics
- …