On the nonlinear stability of symplectic integrators

The modified Hamiltonian is used to study the nonlinear stability of symplectic integrators, especially for nonlinear oscillators. We give conditions under which an initial condition on a compact energy surface will remain bounded for exponentially long times for sufficiently small time steps. For example, the implicit midpoint rule achieves this for the critical energy surface of the H´enon- Heiles system, while the leapfrog method does not. We construct explicit methods which are nonlinearly stable for all simple mechanical systems for exponentially long times. We also address questions of topological stability, finding conditions under which the original and modified energy surfaces are topologically equivalent

An asynchronous leapfrog method II

A second order explicit one-step numerical method for the initial value problem of the general ordinary differential equation is proposed. It is obtained by natural modifications of the well-known leapfrog method, which is a second order, two-step, explicit method. According to the latter method, the input data for an integration step are two system states, which refer to different times. The usage of two states instead of a single one can be seen as the reason for the robustness of the method. Since the time step size thus is part of the step input data, it is complicated to change this size during the computation of a discrete trajectory. This is a serious drawback when one needs to implement automatic time step control. The proposed modification transforms one of the two input states into a velocity and thus gets rid of the time step dependency in the step input data. For these new step input data, the leapfrog method gives a unique prescription how to evolve them stepwise. The stability properties of this modified method are the same as for the original one: the set of absolute stability is the interval [-i,+i] on the imaginary axis. This implies exponential growth of trajectories in situations where the exact trajectory has an asymptote. By considering new evolution steps that are composed of two consecutive old evolution steps we can average over the velocities of the sub-steps and get an integrator with a much larger set of absolute stability, which is immune to the asymptote problem. The method is exemplified with the equation of motion of a one-dimensional non-linear oscillator describing the radial motion in the Kepler problem.Comment: 41 pages, 25 figure

Reversible Architectures for Arbitrarily Deep Residual Neural Networks

Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper and wider architectures. In this work, we interpret deep residual networks as ordinary differential equations (ODEs), which have long been studied in mathematics and physics with rich theoretical and empirical success. From this interpretation, we develop a theoretical framework on stability and reversibility of deep neural networks, and derive three reversible neural network architectures that can go arbitrarily deep in theory. The reversibility property allows a memory-efficient implementation, which does not need to store the activations for most hidden layers. Together with the stability of our architectures, this enables training deeper networks using only modest computational resources. We provide both theoretical analyses and empirical results. Experimental results demonstrate the efficacy of our architectures against several strong baselines on CIFAR-10, CIFAR-100 and STL-10 with superior or on-par state-of-the-art performance. Furthermore, we show our architectures yield superior results when trained using fewer training data.Comment: Accepted at AAAI 201

Linear multistep methods for integrating reversible differential equations

This paper studies multistep methods for the integration of reversible dynamical systems, with particular emphasis on the planar Kepler problem. It has previously been shown by Cano & Sanz-Serna that reversible linear multisteps for first-order differential equations are generally unstable. Here, we report on a subset of these methods -- the zero-growth methods -- that evade these instabilities. We provide an algorithm for identifying these rare methods. We find and study all zero-growth, reversible multisteps with six or fewer steps. This select group includes two well-known second-order multisteps (the trapezoidal and explicit midpoint methods), as well as three new fourth-order multisteps -- one of which is explicit. Variable timesteps can be readily implemented without spoiling the reversibility. Tests on Keplerian orbits show that these new reversible multisteps work well on orbits with low or moderate eccentricity, although at least 100 steps/radian are required for stability.Comment: 31 pages, 9 figures, in press at The Astronomical Journa

Symplectic integrators with adaptive time steps

In recent decades, there have been many attempts to construct symplectic integrators with variable time steps, with rather disappointing results. In this paper we identify the causes for this lack of performance, and find that they fall into two categories. In the first, the time step is considered a function of time alone, \Delta=\Delta(t). In this case, backwards error analysis shows that while the algorithms remain symplectic, parametric instabilities arise because of resonance between oscillations of \Delta(t) and the orbital motion. In the second category the time step is a function of phase space variables \Delta=\Delta(q,p). In this case, the system of equations to be solved is analyzed by introducing a new time variable \tau with dt=\Delta(q,p) d\tau. The transformed equations are no longer in Hamiltonian form, and thus are not guaranteed to be stable even when integrated using a method which is symplectic for constant \Delta. We analyze two methods for integrating the transformed equations which do, however, preserve the structure of the original equations. The first is an extended phase space method, which has been successfully used in previous studies of adaptive time step symplectic integrators. The second, novel, method is based on a non-canonical mixed-variable generating function. Numerical trials for both of these methods show good results, without parametric instabilities or spurious growth or damping. It is then shown how to adapt the time step to an error estimate found by backward error analysis, in order to optimize the time-stepping scheme. Numerical results are obtained using this formulation and compared with other time-stepping schemes for the extended phase space symplectic method.Comment: 23 pages, 9 figures, submitted to Plasma Phys. Control. Fusio

Degenerate Variational Integrators for Magnetic Field Line Flow and Guiding Center Trajectories

Symplectic integrators offer many advantages for the numerical solution of Hamiltonian differential equations, including bounded energy error and the preservation of invariant sets. Two of the central Hamiltonian systems encountered in plasma physics --- the flow of magnetic field lines and the guiding center motion of magnetized charged particles --- resist symplectic integration by conventional means because the dynamics are most naturally formulated in non-canonical coordinates, i.e., coordinates lacking the familiar $(q, p)$ partitioning. Recent efforts made progress toward non-canonical symplectic integration of these systems by appealing to the variational integration framework; however, those integrators were multistep methods and later found to be numerically unstable due to parasitic mode instabilities. This work eliminates the multistep character and, therefore, the parasitic mode instabilities via an adaptation of the variational integration formalism that we deem ``degenerate variational integration''. Both the magnetic field line and guiding center Lagrangians are degenerate in the sense that their resultant Euler-Lagrange equations are systems of first-order ODEs. We show that retaining the same degree of degeneracy when constructing a discrete Lagrangian yields one-step variational integrators preserving a non-canonical symplectic structure on the original Hamiltonian phase space. The advantages of the new algorithms are demonstrated via numerical examples, demonstrating superior stability compared to existing variational integrators for these systems and superior qualitative behavior compared to non-conservative algorithms