13,503 research outputs found
Differentiable Game Mechanics
Deep learning is built on the foundational guarantee that gradient descent on
an objective function converges to local minima. Unfortunately, this guarantee
fails in settings, such as generative adversarial nets, that exhibit multiple
interacting losses. The behavior of gradient-based methods in games is not well
understood -- and is becoming increasingly important as adversarial and
multi-objective architectures proliferate. In this paper, we develop new tools
to understand and control the dynamics in n-player differentiable games.
The key result is to decompose the game Jacobian into two components. The
first, symmetric component, is related to potential games, which reduce to
gradient descent on an implicit function. The second, antisymmetric component,
relates to Hamiltonian games, a new class of games that obey a conservation law
akin to conservation laws in classical mechanical systems. The decomposition
motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding
stable fixed points in differentiable games. Basic experiments show SGA is
competitive with recently proposed algorithms for finding stable fixed points
in GANs -- while at the same time being applicable to, and having guarantees
in, much more general cases.Comment: JMLR 2019, journal version of arXiv:1802.0564
A Variational Perspective on Accelerated Methods in Optimization
Accelerated gradient methods play a central role in optimization, achieving
optimal rates in many settings. While many generalizations and extensions of
Nesterov's original acceleration method have been proposed, it is not yet clear
what is the natural scope of the acceleration concept. In this paper, we study
accelerated methods from a continuous-time perspective. We show that there is a
Lagrangian functional that we call the \emph{Bregman Lagrangian} which
generates a large class of accelerated methods in continuous time, including
(but not limited to) accelerated gradient descent, its non-Euclidean extension,
and accelerated higher-order gradient methods. We show that the continuous-time
limit of all of these methods correspond to traveling the same curve in
spacetime at different speeds. From this perspective, Nesterov's technique and
many of its generalizations can be viewed as a systematic way to go from the
continuous-time curves generated by the Bregman Lagrangian to a family of
discrete-time accelerated algorithms.Comment: 38 pages. Subsumes an earlier working draft arXiv:1509.0361
Generalisation under gradient descent via deterministic PAC-Bayes
We establish disintegrated PAC-Bayesian generalisation bounds for models
trained with gradient descent methods or continuous gradient flows. Contrary to
standard practice in the PAC-Bayesian setting, our result applies to
optimisation algorithms that are deterministic, without requiring any
de-randomisation step. Our bounds are fully computable, depending on the
density of the initial distribution and the Hessian of the training objective
over the trajectory. We show that our framework can be applied to a variety of
iterative optimisation algorithms, including stochastic gradient descent (SGD),
momentum-based schemes, and damped Hamiltonian dynamics
Variational ansatz-based quantum simulation of imaginary time evolution
Imaginary time evolution is a powerful tool for studying quantum systems.
While it is possible to simulate with a classical computer, the time and memory
requirements generally scale exponentially with the system size. Conversely,
quantum computers can efficiently simulate quantum systems, but not non-unitary
imaginary time evolution. We propose a variational algorithm for simulating
imaginary time evolution on a hybrid quantum computer. We use this algorithm to
find the ground-state energy of many-particle systems; specifically molecular
hydrogen and lithium hydride, finding the ground state with high probability.
Our method can also be applied to general optimisation problems and quantum
machine learning. As our algorithm is hybrid, suitable for error mitigation and
can exploit shallow quantum circuits, it can be implemented with current
quantum computers.Comment: 14 page
Variational method for locating invariant tori
We formulate a variational fictitious-time flow which drives an initial guess
torus to a torus invariant under given dynamics. The method is general and
applies in principle to continuous time flows and discrete time maps in
arbitrary dimension, and to both Hamiltonian and dissipative systems.Comment: 10 page
Trajectory-Based Off-Policy Deep Reinforcement Learning
Policy gradient methods are powerful reinforcement learning algorithms and
have been demonstrated to solve many complex tasks. However, these methods are
also data-inefficient, afflicted with high variance gradient estimates, and
frequently get stuck in local optima. This work addresses these weaknesses by
combining recent improvements in the reuse of off-policy data and exploration
in parameter space with deterministic behavioral policies. The resulting
objective is amenable to standard neural network optimization strategies like
stochastic gradient descent or stochastic gradient Hamiltonian Monte Carlo.
Incorporation of previous rollouts via importance sampling greatly improves
data-efficiency, whilst stochastic optimization schemes facilitate the escape
from local optima. We evaluate the proposed approach on a series of continuous
control benchmark tasks. The results show that the proposed algorithm is able
to successfully and reliably learn solutions using fewer system interactions
than standard policy gradient methods.Comment: Includes appendix. Accepted for ICML 201
- …