Herding as a Learning System with Edge-of-Chaos Dynamics

Herding defines a deterministic dynamical system at the edge of chaos. It generates a sequence of model states and parameters by alternating parameter perturbations with state maximizations, where the sequence of states can be interpreted as "samples" from an associated MRF model. Herding differs from maximum likelihood estimation in that the sequence of parameters does not converge to a fixed point and differs from an MCMC posterior sampling approach in that the sequence of states is generated deterministically. Herding may be interpreted as a"perturb and map" method where the parameter perturbations are generated using a deterministic nonlinear dynamical system rather than randomly from a Gumbel distribution. This chapter studies the distinct statistical characteristics of the herding algorithm and shows that the fast convergence rate of the controlled moments may be attributed to edge of chaos dynamics. The herding algorithm can also be generalized to models with latent variables and to a discriminative learning setting. The perceptron cycling theorem ensures that the fast moment matching property is preserved in the more general framework

Stochastic collective dynamics of charged--particle beams in the stability regime

We introduce a description of the collective transverse dynamics of charged (proton) beams in the stability regime by suitable classical stochastic fluctuations. In this scheme, the collective beam dynamics is described by time--reversal invariant diffusion processes deduced by stochastic variational principles (Nelson processes). By general arguments, we show that the diffusion coefficient, expressed in units of length, is given by $\lambda_c\sqrt{N}$, where $N$ is the number of particles in the beam and $\lambda_c$ the Compton wavelength of a single constituent. This diffusion coefficient represents an effective unit of beam emittance. The hydrodynamic equations of the stochastic dynamics can be easily recast in the form of a Schr\"odinger equation, with the unit of emittance replacing the Planck action constant. This fact provides a natural connection to the so--called ``quantum--like approaches'' to beam dynamics. The transition probabilities associated to Nelson processes can be exploited to model evolutions suitable to control the transverse beam dynamics. In particular we show how to control, in the quadrupole approximation to the beam--field interaction, both the focusing and the transverse oscillations of the beam, either together or independently.Comment: 15 pages, 9 figure

Non-intrusive and structure preserving multiscale integration of stiff ODEs, SDEs and Hamiltonian systems with hidden slow dynamics via flow averaging

We introduce a new class of integrators for stiff ODEs as well as SDEs. These integrators are (i) {\it Multiscale}: they are based on flow averaging and so do not fully resolve the fast variables and have a computational cost determined by slow variables (ii) {\it Versatile}: the method is based on averaging the flows of the given dynamical system (which may have hidden slow and fast processes) instead of averaging the instantaneous drift of assumed separated slow and fast processes. This bypasses the need for identifying explicitly (or numerically) the slow or fast variables (iii) {\it Nonintrusive}: A pre-existing numerical scheme resolving the microscopic time scale can be used as a black box and easily turned into one of the integrators in this paper by turning the large coefficients on over a microscopic timescale and off during a mesoscopic timescale (iv) {\it Convergent over two scales}: strongly over slow processes and in the sense of measures over fast ones. We introduce the related notion of two-scale flow convergence and analyze the convergence of these integrators under the induced topology (v) {\it Structure preserving}: for stiff Hamiltonian systems (possibly on manifolds), they can be made to be symplectic, time-reversible, and symmetry preserving (symmetries are group actions that leave the system invariant) in all variables. They are explicit and applicable to arbitrary stiff potentials (that need not be quadratic). Their application to the Fermi-Pasta-Ulam problems shows accuracy and stability over four orders of magnitude of time scales. For stiff Langevin equations, they are symmetry preserving, time-reversible and Boltzmann-Gibbs reversible, quasi-symplectic on all variables and conformally symplectic with isotropic friction.Comment: 69 pages, 21 figure

Role of stochastic noise and generalization error in the time propagation of neural-network quantum states

Neural-network quantum states (NQS) have been shown to be a suitable variational ansatz to simulate out-of-equilibrium dynamics in two-dimensional systems using time-dependent variational Monte Carlo (t-VMC). In particular, stable and accurate time propagation over long time scales has been observed in the square-lattice Heisenberg model using the Restricted Boltzmann machine architecture. However, achieving similar performance in other systems has proven to be more challenging. In this article, we focus on the two-leg Heisenberg ladder driven out of equilibrium by a pulsed excitation as a benchmark system. We demonstrate that unmitigated noise is strongly amplified by the nonlinear equations of motion for the network parameters, which causes numerical instabilities in the time evolution. As a consequence, the achievable accuracy of the simulated dynamics is a result of the interplay between network expressiveness and measures required to remedy these instabilities. We show that stability can be greatly improved by appropriate choice of regularization. This is particularly useful as tuning of the regularization typically imposes no additional computational cost. Inspired by machine learning practice, we propose a validation-set based diagnostic tool to help determining optimal regularization hyperparameters for t-VMC based propagation schemes. For our benchmark, we show that stable and accurate time propagation can be achieved in regimes of sufficiently regularized variational dynamics

Nonlinear wave-particle resonance in deterministic and stochastic kinetic plasmas

In kinetic plasma physics, BGK modes are ubiquitous solutions to the Vlasov equation, with particles travelling along orbits where the single particle energy is conserved. Approximate extensions of these exact solutions have been successfully used in the past to understand the formation and evolution of ‘holes’ and ‘clumps’, coherent structures on the particle distribution function which under certain conditions form in the nonlinear phase of the evolution of kinetic plasmas. In this thesis, analytical results are shown which consider perturbations and deformations to BGK orbits, allowing one to robustly construct more exotic orbits that allow for mode growth and frequency chirping. Computational results produced using the DARK code are presented, examining stochastic and deterministic populations in a 1D electrostatic plasma, and how they affect electrostatic waves exhibiting Landau resonance, based on Berk-Breizman models. A model is presented for parametric mode-mode destabilisation via holes and clumps interacting via the background distribution. Finally, work using the machine learning framework ERICSON is presented, analysing frequency spectrograms of magnetic perturbations in Alfvénic and sub-Alfvénic frequency ranges

Deterministic-Kinetic Computational Analyses of Expansion Flows and Current-Carrying Plasmas

Spacecraft electric propulsion (EP) takes advantage of the ability of electric and magnetic fields to accelerate plasmas to high velocities to generate efficient thrust. The thermionic hollow cathode is a critical component to both gridded-ion and Hall-effect thrusters, the state-of-the-art devices of the EP discipline. However, experiments demonstrate that the hollow cathode is plagued by erosion of its surfaces by the plasma, which may eventually cause premature failure of the device. This erosion has been linked to the ion-acoustic instability (IAI), a kinetic plasma instability which operates in the cathode plume. Existence of this kinetic instability has prevented numerical simulation from predicting the operating characteristics and lifetime of the hollow cathode device. Therefore, this thesis utilizes deterministic-kinetic (DK) simulation of gas and plasma flows to further the understanding of the IAI as it relates to the hollow cathode plume and to ultimately develop a predictive hollow cathode simulation platform. Towards these goals, two approaches to applying the DK simulation method to the hollow cathode plasma are undertaken: hybrid-kinetic simulation and fully-kinetic simulation. Hybrid-kinetic simulations utilize a kinetic description of the heavy propellant particles while using a reduced-order, fluid approach for the light electrons. Two unique two-dimensional, axisymmetric kinetic schemes are developed, one for neutral particles and one for ions; the schemes are verified by comparison with solutions obtained using the direct-simulation Monte Carlo method and with an analytic solution for a rarefied neutral jet flow. Assuming quasi-neutrality in the hollow cathode plasma and using the Boltzmann relation for the plasma potential, the hybrid-kinetic solver is applied to the problem of NASA's NSTAR discharge hollow cathode. Partial validation is achieved through agreement with experimental Langmuir probe data in the near-orifice region, while shortcomings of the solver such as use of a simplified electron model are discussed. Fully-kinetic simulations, where all species are considered kinetically, are carried out to study the IAI. The anomalous resistivity generated by the IAI is measured from one-dimensional fully-kinetic simulations and compared with a closure model commonly used in hollow cathode fluid codes, finding that the agreement with the closure model varies based on simulation domain size and electron Mach number. Further, the formation of high-energy tails in the ion velocity distribution function is observed near the transition to the Buneman instability, another instability of current-carrying plasmas. Two-dimensional kinetic simulations of current-carrying instabilities are carried out, finding that the nature of nonlinear saturation of the IAI differs significantly from that shown in one-dimensional simulations. A phenomenon known as the off-axis instability generates waves propagating normal to the current direction which eventually reach energy levels close to that of the waves along the current direction. Further fully-kinetic simulations demonstrate the formation of weak plasma double layers, regions of plasma which sustain a potential gradient, in the nonlinear saturation stage of the IAI. These double layers are found to be ubiquitous in all plasma species considered, even the heavy xenon ions commonly used in hollow cathodes. Phase space analysis suggests the double layers form from ion-acoustic wave packets which grow into ion phase space holes. Spectral analysis demonstrates a shift towards smaller wavenumbers which marks this transition. An electron two-stream instability is spawned due to the potential well of the double layer, where spectral analyses demonstrate that a simple theoretical expression well-predicts the resulting wave phase velocity.PHDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169806/1/vazsonyi_1.pd