9 research outputs found
Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization
Stochastic gradient-based optimization is crucial to optimize neural
networks. While popular approaches heuristically adapt the step size and
direction by rescaling gradients, a more principled approach to improve
optimizers requires second-order information. Such methods precondition the
gradient using the objective's Hessian. Yet, computing the Hessian is usually
expensive and effectively using second-order information in the stochastic
gradient setting is non-trivial. We propose using Information-Theoretic Trust
Region Optimization (arTuRO) for improved updates with uncertain second-order
information. By modeling the network parameters as a Gaussian distribution and
using a Kullback-Leibler divergence-based trust region, our approach takes
bounded steps accounting for the objective's curvature and uncertainty in the
parameters. Before each update, it solves the trust region problem for an
optimal step size, resulting in a more stable and faster optimization process.
We approximate the diagonal elements of the Hessian from stochastic gradients
using a simple recursive least squares approach, constructing a model of the
expected Hessian over time using only first-order information. We show that
arTuRO combines the fast convergence of adaptive moment-based optimization with
the generalization capabilities of SGD
Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization
Stochastic gradient-based optimization is crucial to optimize neural networks. While popular approaches heuristically adapt the step size and direction by rescaling gradients, a more principled
approach to improve optimizers requires second-order information. Such methods precondition
the gradient using the objective’s Hessian. Yet, computing the Hessian is usually expensive and
effectively using second-order information in the stochastic gradient setting is non-trivial. We propose using Information-Theoretic Trust Region Optimization (arTuRO) for improved updates with
uncertain second-order information. By modeling the network parameters as a Gaussian distribution and using a Kullback-Leibler divergence-based trust region, our approach takes bounded steps
accounting for the objective’s curvature and uncertainty in the parameters. Before each update, it
solves the trust region problem for an optimal step size, resulting in a more stable and faster optimization process. We approximate the diagonal elements of the Hessian from stochastic gradients
using a simple recursive least squares approach, constructing a model of the expected Hessian over
time using only first-order information. We show that arTuRO combines the fast convergence of
adaptive moment-based optimization with the generalization capabilities of SGD
Swarm Reinforcement Learning For Adaptive Mesh Refinement
Adaptive Mesh Refinement (AMR) enhances the Finite Element Method, an important technique for simulating complex problems in engineering, by dynamically refining mesh regions, enabling a favorable trade-off between computational speed and simulation accuracy. Classical methods for AMR depend on heuristics or expensive error estimators, hindering their use for complex simulations. Recent learning-based AMR methods tackle these issues, but so far scale only to simple toy examples. We formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh is modeled as a system of simple collaborating agents that may split into multiple new agents. This framework allows for a spatial reward formulation that simplifies the credit assignment problem, which we combine with Message Passing Networks to propagate information between neighboring mesh elements. We experimentally validate our approach, Adaptive Swarm Mesh Refinement (ASMR), on challenging refinement tasks. Our approach learns reliable and efficient refinement strategies that can robustly generalize to different domains during inference. Additionally, it achieves a speedup of up to orders of magnitude compared to uniform refinements in more demanding simulations. We outperform learned baselines and heuristics, achieving a refinement quality that is on par with costly error-based oracle AMR strategies
Swarm Reinforcement Learning For Adaptive Mesh Refinement
The Finite Element Method, an important technique in engineering, is aided by
Adaptive Mesh Refinement (AMR), which dynamically refines mesh regions to allow
for a favorable trade-off between computational speed and simulation accuracy.
Classical methods for AMR depend on task-specific heuristics or expensive error
estimators, hindering their use for complex simulations. Recent learned AMR
methods tackle these problems, but so far scale only to simple toy examples. We
formulate AMR as a novel Adaptive Swarm Markov Decision Process in which a mesh
is modeled as a system of simple collaborating agents that may split into
multiple new agents. This framework allows for a spatial reward formulation
that simplifies the credit assignment problem, which we combine with Message
Passing Networks to propagate information between neighboring mesh elements. We
experimentally validate the effectiveness of our approach, Adaptive Swarm Mesh
Refinement (ASMR), showing that it learns reliable, scalable, and efficient
refinement strategies on a set of challenging problems. Our approach
significantly speeds up computation, achieving up to 30-fold improvement
compared to uniform refinements in complex simulations. Additionally, we
outperform learned baselines and achieve a refinement quality that is on par
with a traditional error-based AMR strategy without expensive oracle
information about the error signal.Comment: Version 1 of this paper is a preliminary workshop version that was
accepted as a workshop paper in the ICLR 2023 Workshop on Physics for Machine
Learnin
Preventing traffic accidents with in-vehicle decision support systems - The impact of accident hotspot warnings on driver behaviour
ISSN:0167-9236ISSN:1873-579
A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Models
Variational inference with Gaussian mixture models (GMMs) enables learning of
highly tractable yet multi-modal approximations of intractable target
distributions with up to a few hundred dimensions. The two currently most
effective methods for GMM-based variational inference, VIPS and iBayes-GMM,
both employ independent natural gradient updates for the individual components
and their weights. We show for the first time, that their derived updates are
equivalent, although their practical implementations and theoretical guarantees
differ. We identify several design choices that distinguish both approaches,
namely with respect to sample selection, natural gradient estimation, stepsize
adaptation, and whether trust regions are enforced or the number of components
adapted. We argue that for both approaches, the quality of the learned
approximations can heavily suffer from the respective design choices: By
updating the individual components using samples from the mixture model,
iBayes-GMM often fails to produce meaningful updates to low-weight components,
and by using a zero-order method for estimating the natural gradient, VIPS
scales badly to higher-dimensional problems. Furthermore, we show that
information-geometric trust-regions (used by VIPS) are effective even when
using first-order natural gradient estimates, and often outperform the improved
Bayesian learning rule (iBLR) update used by iBayes-GMM. We systematically
evaluate the effects of design choices and show that a hybrid approach
significantly outperforms both prior works. Along with this work, we publish
our highly modular and efficient implementation for natural gradient
variational inference with Gaussian mixture models, which supports 432
different combinations of design choices, facilitates the reproduction of all
our experiments, and may prove valuable for the practitioner.Comment: This version corresponds to the camera ready version published at
Transactions of Machine Learning Research (TMLR).
https://openreview.net/forum?id=tLBjsX4tj