Least-biased correction of extended dynamical systems using observational data

We consider dynamical systems evolving near an equilibrium statistical state where the interest is in modelling long term behavior that is consistent with thermodynamic constraints. We adjust the distribution using an entropy-optimizing formulation that can be computed on-the- fly, making possible partial corrections using incomplete information, for example measured data or data computed from a different model (or the same model at a different scale). We employ a thermostatting technique to sample the target distribution with the aim of capturing relavant statistical features while introducing mild dynamical perturbation (thermostats). The method is tested for a point vortex fluid model on the sphere, and we demonstrate both convergence of equilibrium quantities and the ability of the formulation to balance stationary and transient- regime errors.Comment: 27 page

Multirate Training of Neural Networks

We propose multirate training of neural networks: partitioning neural network parameters into "fast" and "slow" parts which are trained simultaneously using different learning rates. By choosing appropriate partitionings we can obtain large computational speed-ups for transfer learning tasks. We show that for various transfer learning applications in vision and NLP we can fine-tune deep neural networks in almost half the time, without reducing the generalization performance of the resulting model. We also discuss other splitting choices for the neural network parameters which are beneficial in enhancing generalization performance in settings where neural networks are trained from scratch. Finally, we propose an additional multirate technique which can learn different features present in the data by training the full network on different time scales simultaneously. The benefits of using this approach are illustrated for ResNet architectures on image data. Our paper unlocks the potential of using multirate techniques for neural network training and provides many starting points for future work in this area

Generating Generalized Distributions from Dynamical Simulation

We present a general molecular-dynamics simulation scheme, based on the Nose' thermostat, for sampling according to arbitrary phase space distributions. We formulate numerical methods based on both Nose'-Hoover and Nose'-Poincare' thermostats for two specific classes of distributions; namely, those that are functions of the system Hamiltonian and those for which position and momentum are statistically independent. As an example, we propose a generalized variable temperature distribution that designed to accelerate sampling in molecular systems.Comment: 10 pages, 3 figure

A molecular-dynamics algorithm for mixed hard-core/continuous potentials

We present a new molecular-dynamics algorithm for integrating the equations of motion for a system of particles interacting with mixed continuous/impulsive forces. This method, which we call Impulsive Verlet, is constructed using operator splitting techniques similar to those that have been used successfully to generate a variety molecular-dynamics integrators. In numerical experiments, the Impulsive Verlet method is shown to be superior to previous methods with respect to stability and energy conservation in long simulations.Comment: 18 pages, 6 postscript figures, uses rotate.st

The adaptive Verlet method

This is the published version, also available here: http://dx.doi.org/10.1137/S1064827595284658.We discuss the integration of autonomous Hamiltonian systems via dynamical rescaling of the vector field (reparameterization of time). Appropriate rescalings (e.g., based on normalization of the vector field or on minimum particle separation in an N-body problem) do not alter the time-reversal symmetry of the flow, and it is desirable to maintain this symmetry under discretization. For standard form mechanical systems without rescaling, this can be achieved by using the explicit leapfrog--Verlet method; we show that explicit time-reversible integration of the reparameterized equations is also possible if the parameterization depends on positions or velocities only. For general rescalings, a scalar nonlinear equation must be solved at each step, but only one force evaluation is needed. The new method also conserves the angular momentum for an N-body problem. The use of reversible schemes, together with a step control based on normalization of the vector field (arclength reparameterization), is demonstrated in several numerical experiments, including a double pendulum, the Kepler problem, and a three-body problem

Semi-geostrophic particle motion and exponentially accurate normal forms

We give an exponentially-accurate normal form for a Lagrangian particle moving in a rotating shallow-water system in the semi-geostrophic limit, which describes the motion in the region of an exponentially-accurate slow manifold (a region of phase space for which dynamics on the fast scale are exponentially small in the Rossby number). The result extends to numerical solutions of this problem via backward error analysis, and extends to the Hamiltonian Particle-Mesh (HPM) method for the shallow-water equations where the result shows that HPM stays close to balance for exponentially-long times in the semi-geostrophic limit. We show how this result is related to the variational asymptotics approach of [Oliver, 2005]; the difference being that on the Hamiltonian side it is possible to obtain strong bounds on the growth of fast motion away from (but near to) the slow manifold