Towards provably efficient quantum algorithms for large-scale machine-learning models

Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as $\mathcal{O}(T^2 \times \text{polylog}(n))$, where $n$ is the size of the models and $T$ is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.Comment: 7+30 pages, 3+5 figure

Quantum Computing for Fusion Energy Science Applications

This is a review of recent research exploring and extending present-day quantum computing capabilities for fusion energy science applications. We begin with a brief tutorial on both ideal and open quantum dynamics, universal quantum computation, and quantum algorithms. Then, we explore the topic of using quantum computers to simulate both linear and nonlinear dynamics in greater detail. Because quantum computers can only efficiently perform linear operations on the quantum state, it is challenging to perform nonlinear operations that are generically required to describe the nonlinear differential equations of interest. In this work, we extend previous results on embedding nonlinear systems within linear systems by explicitly deriving the connection between the Koopman evolution operator, the Perron-Frobenius evolution operator, and the Koopman-von Neumann evolution (KvN) operator. We also explicitly derive the connection between the Koopman and Carleman approaches to embedding. Extension of the KvN framework to the complex-analytic setting relevant to Carleman embedding, and the proof that different choices of complex analytic reproducing kernel Hilbert spaces depend on the choice of Hilbert space metric are covered in the appendices. Finally, we conclude with a review of recent quantum hardware implementations of algorithms on present-day quantum hardware platforms that may one day be accelerated through Hamiltonian simulation. We discuss the simulation of toy models of wave-particle interactions through the simulation of quantum maps and of wave-wave interactions important in nonlinear plasma dynamics.Comment: 42 pages; 12 figures; invited paper at the 2021-2022 International Sherwood Fusion Theory Conferenc

Solution Approximation for Atmospheric Flight Dynamics Using Volterra Theory

This dissertation introduces a set of novel approaches in order to facilitate and enrich Volterra theory as a nonlinear approximation technique for constructing mathematical solutions from the governing relationships describing aircraft dynamic behavior. These approaches reconnect Volterra theory and flight mechanics research, which has not been addressed in the technical literature for over twenty years. Volterra theory is known to be viable in modeling weak nonlinearities, but is not particularly well suited for directly describing high performance aircraft dynamics. In order to overcome these obstacles and restrictions of Volterra theory, the global Piecewise Volterra Approach has been developed. This new approach decomposes a strong nonlinearity into weaker components in several sub-regions, which individually only require a low order truncated series. A novel Cause-and-Effect Analysis of these low order truncated series has also been developed. This new technique in turn allows system prediction before employing computer simulation, as well as decomposition of existing simulation results. For a computationally complex and large envelope airframe system, a Volterra Parameter-Varying Model Approach has also been developed as a systematically efficient approach to track the aircraft dynamic model and its response across a wide range of operating conditions. The analytical and numerical solutions based on the proposed methodology show the ability of Volterra theory to help predict, understand, and analyze nonlinear aircraft behavior beyond that attainable by linear theory, or more difficult to extract from nonlinear simulation, which in turn leads to a more efficient nonlinear preliminary design tool

Mini-Workshop: Applied Koopmanism

Koopman and Perron–Frobenius operators are linear operators that encapsulate dynamics of nonlinear dynamical systems without loss of information. This is accomplished by embedding the dynamics into a larger infinite-dimensional space where the focus of study is shifted from trajectory curves to measurement functions evaluated along trajectories and densities of trajectories evolving in time. Operator-theoretic approach to dynamics shares many features with an optimization technique: the Lasserre moment–sums-of-squares (SOS) hierarchies, which was developed for numerically solving non-convex optimization problems with semialgebraic data. This technique embeds the optimization problem into a larger primal semidefinite programming (SDP) problem consisting of measure optimization over the set of globally optimal solutions, where measures are manipulated through their truncated moment sequences. The dual SDP problem uses SOS representations to certify bounds on the global optimum. This workshop highlighted the common threads between the operator-theoretic dynamical systems and moment–SOS hierarchies in optimization and explored the future directions where the synergy of the two techniques could yield results in fluid dynamics, control theory, optimization, and spectral theory

On study of deterministic conservative solvers for the nonlinear boltzmann and landau transport equations

textThe Boltzmann Transport Equation (BTE) has been the keystone of the kinetic theory, which is at the center of Statistical Mechanics bridging the gap between the atomic structures and the continuum-like behaviors. The existence of solutions has been a great mathematical challenge and still remains elusive. As a grazing limit of the Boltzmann operator, the Fokker-Planck-Landau (FPL) operator is of primary importance for collisional plasmas. We have worked on the following three different projects regarding the most important kinetic models, the BTE and the FPL Equations. (1). A Discontinuous Galerkin Solver for Nonlinear BTE. We propose a deterministic numerical solver based on Discontinuous Galerkin (DG) methods, which has been rarely studied. As the key part, the weak form of the collision operator is approximated within subspaces of piecewise polynomials. To save the tremendous computational cost with increasing order of polynomials and number of mesh nodes, as well as to resolve loss of conservations due to domain truncations, the following combined procedures are applied. First, the collision operator is projected onto a subspace of basis polynomials up to first order. Then, at every time step, a conservation routine is employed to enforce the preservation of desired moments (mass, momentum and/or energy), with only linear complexity. The asymptotic error analysis shows the validity and guarantees the accuracy of these two procedures. We applied the property of ``shifting symmetries" in the weight matrix, which consists in finding a minimal set of basis matrices that can exactly reconstruct the complete family of collision weight matrix. This procedure, together with showing the sparsity of the weight matrix, reduces the computation and storage of the collision matrix from O(N3) down to O(N^2). (2). Spectral Gap for Linearized Boltzmann Operator. Spectral gaps provide information on the relaxation to equilibrium. This is a pioneer field currently unexplored form the computational viewpoint. This work, for the first time, provides numerical evidence on the existence of spectral gaps and corresponding approximate values. The linearized Boltzmann operator is projected onto a Discontinuous Galerkin mesh, resulting in a ``collision matrix". The original spectral gap problem is then approximated by a constrained minimization problem, with objective function the Rayleigh quotient of the "collision matrix" and with constraints the conservation laws. A conservation correction then applies. We also study the convergence of the approximate Rayleigh quotient to the real spectral gap. (3). A Conservative Scheme for Approximating Collisional Plasmas. We have developed a deterministic conservative solver for the inhomogeneous Fokker-Planck-Landau equations coupled with Poisson equations. The original problem is splitted into two subproblems: collisonless Vlasov problem and collisonal homogeneous Fokker-Planck-Landau problem. They are handled with different numerical schemes. The former is approximated using Runge-Kutta Discontinuous Galerkin (RKDG) scheme with a piecewise polynomial basis subspace covering all collision invariants; while the latter is solved by a conservative spectral method. To link the two different computing grids, a special conservation routine is also developed. All the projects are implemented with hybrid MPI and OpenMP. Numerical results and applications are provided.Computational Science, Engineering, and Mathematic

System- and Data-Driven Methods and Algorithms

An increasing complexity of models used to predict real-world systems leads to the need for algorithms to replace complex models with far simpler ones, while preserving the accuracy of the predictions. This two-volume handbook covers methods as well as applications. This first volume focuses on real-time control theory, data assimilation, real-time visualization, high-dimensional state spaces and interaction of different reduction techniques