Ab-initio solution of the many-electron Schrödinger equation with deep neural networks

Given access to accurate solutions of the many-electron Schr\"odinger equation, nearly all chemistry could be derived from first principles. Exact wavefunctions of interesting chemical systems are out of reach because they are NP-hard to compute in general, but approximations can be found using polynomially-scaling algorithms. The key challenge for many of these algorithms is the choice of wavefunction approximation, or Ansatz, which must trade off between efficiency and accuracy. Neural networks have shown impressive power as accurate practical function approximators and promise as a compact wavefunction Ansatz for spin systems, but problems in electronic structure require wavefunctions that obey Fermi-Dirac statistics. Here we introduce a novel deep learning architecture, the Fermionic Neural Network, as a powerful wavefunction Ansatz for many-electron systems. The Fermionic Neural Network is able to achieve accuracy beyond other variational quantum Monte Carlo Ans\"atze on a variety of atoms and small molecules. Using no data other than atomic positions and charges, we predict the dissociation curves of the nitrogen molecule and hydrogen chain, two challenging strongly-correlated systems, to significantly higher accuracy than the coupled cluster method, widely considered the most accurate scalable method for quantum chemistry at equilibrium geometry. This demonstrates that deep neural networks can improve the accuracy of variational quantum Monte Carlo to the point where it outperforms other ab-initio quantum chemistry methods, opening the possibility of accurate direct optimisation of wavefunctions for previously intractable molecules and solids

Ab-initio quantum chemistry with neural-network wavefunctions

Machine learning and specifically deep-learning methods have outperformed human capabilities in many pattern recognition and data processing problems, in game playing, and now also play an increasingly important role in scientific discovery. A key application of machine learning in the molecular sciences is to learn potential energy surfaces or force fields from ab-initio solutions of the electronic Schr\"odinger equation using datasets obtained with density functional theory, coupled cluster, or other quantum chemistry methods. Here we review a recent and complementary approach: using machine learning to aid the direct solution of quantum chemistry problems from first principles. Specifically, we focus on quantum Monte Carlo (QMC) methods that use neural network ansatz functions in order to solve the electronic Schr\"odinger equation, both in first and second quantization, computing ground and excited states, and generalizing over multiple nuclear configurations. Compared to existing quantum chemistry methods, these new deep QMC methods have the potential to generate highly accurate solutions of the Schr\"odinger equation at relatively modest computational cost.Comment: review, 17 pages, 6 figure

Solution of Schrödinger Equation for Quantum Systems via Physics-Informed Neural Networks

openThe numerous successes achieved by machine learning techniques in many technical areas have sparked interest in the scientific community for their application in science. By merging the knowledge of machine learning experts and computational scientists, the field of scientific machine learning has shown its ability to greatly improve the performance of existing computational methods. One possible approach to developing physics-aware machine learning is the inclusion of physical constraints in the training of a machine learning model. Physics-Informed Neural Networks are an example of such an approach, as they can incorporate prior physical knowledge into their architecture, enabling them to learn and simulate complex phenomena while respecting the underlying physics principles. Possible constraints are physical laws, symmetries, and conservation laws. Compared to other machine learning models, Physics-Informed Neural Networks do not require substantial input data, with the exception of initial and boundary conditions to correctly formalize the problem. In this Thesis, we exploit the advantages of Physics-Informed Neural Networks to efficiently simulate one-electron quantum systems. The simulations rely on the direct solution of the eigenvalue equation represented by the Schrödinger equation. Traditional methods for solving the Schrödinger equation often rely on approximations and can become computationally expensive for nontrivial systems. The mesh-free Physics-Informed Neural Networks approach avoids the need for discretization, as the residuals computed with respect to the physical constraints are minimized during training for a given set of points within the domain. The solution of the Schrödinger equation allows one to calculate important physical quantities of the physical system under study, such as the ground state energy, the electronic wavefunction, and the associated electron density. These quantities are compared with the estimations present in the quantum chemistry literature to assess the performance of the Physics-Informed Machine Learning approach.The numerous successes achieved by machine learning techniques in many technical areas have sparked interest in the scientific community for their application in science. By merging the knowledge of machine learning experts and computational scientists, the field of scientific machine learning has shown its ability to greatly improve the performance of existing computational methods. One possible approach to developing physics-aware machine learning is the inclusion of physical constraints in the training of a machine learning model. Physics-Informed Neural Networks are an example of such an approach, as they can incorporate prior physical knowledge into their architecture, enabling them to learn and simulate complex phenomena while respecting the underlying physics principles. Possible constraints are physical laws, symmetries, and conservation laws. Compared to other machine learning models, Physics-Informed Neural Networks do not require substantial input data, with the exception of initial and boundary conditions to correctly formalize the problem. In this Thesis, we exploit the advantages of Physics-Informed Neural Networks to efficiently simulate one-electron quantum systems. The simulations rely on the direct solution of the eigenvalue equation represented by the Schrödinger equation. Traditional methods for solving the Schrödinger equation often rely on approximations and can become computationally expensive for nontrivial systems. The mesh-free Physics-Informed Neural Networks approach avoids the need for discretization, as the residuals computed with respect to the physical constraints are minimized during training for a given set of points within the domain. The solution of the Schrödinger equation allows one to calculate important physical quantities of the physical system under study, such as the ground state energy, the electronic wavefunction, and the associated electron density. These quantities are compared with the estimations present in the quantum chemistry literature to assess the performance of the Physics-Informed Machine Learning approach

Variational Monte Carlo on a Budget -- Fine-tuning pre-trained Neural Wavefunctions

Obtaining accurate solutions to the Schr\"odinger equation is the key challenge in computational quantum chemistry. Deep-learning-based Variational Monte Carlo (DL-VMC) has recently outperformed conventional approaches in terms of accuracy, but only at large computational cost. Whereas in many domains models are trained once and subsequently applied for inference, accurate DL-VMC so far requires a full optimization for every new problem instance, consuming thousands of GPUhs even for small molecules. We instead propose a DL-VMC model which has been pre-trained using self-supervised wavefunction optimization on a large and chemically diverse set of molecules. Applying this model to new molecules without any optimization, yields wavefunctions and absolute energies that outperform established methods such as CCSD(T)-2Z. To obtain accurate relative energies, only few fine-tuning steps of this base model are required. We accomplish this with a fully end-to-end machine-learned model, consisting of an improved geometry embedding architecture and an existing SE(3)-equivariant model to represent molecular orbitals. Combining this architecture with continuous sampling of geometries, we improve zero-shot accuracy by two orders of magnitude compared to the state of the art. We extensively evaluate the accuracy, scalability and limitations of our base model on a wide variety of test systems

Electronic excited states in deep variational Monte Carlo

Obtaining accurate ground and low-lying excited states of electronic systems is crucial in a multitude of important applications. One ab initio method for solving the Schrödinger equation that scales favorably for large systems is variational quantum Monte Carlo (QMC). The recently introduced deep QMC approach uses ansatzes represented by deep neural networks and generates nearly exact ground-state solutions for molecules containing up to a few dozen electrons, with the potential to scale to much larger systems where other highly accurate methods are not feasible. In this paper, we extend one such ansatz (PauliNet) to compute electronic excited states. We demonstrate our method on various small atoms and molecules and consistently achieve high accuracy for low-lying states. To highlight the method’s potential, we compute the first excited state of the much larger benzene molecule, as well as the conical intersection of ethylene, with PauliNet matching results of more expensive high-level methods

Sampling-free Inference for Ab-Initio Potential Energy Surface Networks

Recently, it has been shown that neural networks not only approximate the ground-state wave functions of a single molecular system well but can also generalize to multiple geometries. While such generalization significantly speeds up training, each energy evaluation still requires Monte Carlo integration which limits the evaluation to a few geometries. In this work, we address the inference shortcomings by proposing the Potential learning from ab-initio Networks (PlaNet) framework, in which we simultaneously train a surrogate model in addition to the neural wave function. At inference time, the surrogate avoids expensive Monte-Carlo integration by directly estimating the energy, accelerating the process from hours to milliseconds. In this way, we can accurately model high-resolution multi-dimensional energy surfaces for larger systems that previously were unobtainable via neural wave functions. Finally, we explore an additional inductive bias by introducing physically-motivated restricted neural wave function models. We implement such a function with several additional improvements in the new PESNet++ model. In our experimental evaluation, PlaNet accelerates inference by 7 orders of magnitude for larger molecules like ethanol while preserving accuracy. Compared to previous energy surface networks, PESNet++ reduces energy errors by up to 74%

Wasserstein Quantum Monte Carlo: A Novel Approach for Solving the Quantum Many-Body Schr\"odinger Equation

Solving the quantum many-body Schr\"odinger equation is a fundamental and challenging problem in the fields of quantum physics, quantum chemistry, and material sciences. One of the common computational approaches to this problem is Quantum Variational Monte Carlo (QVMC), in which ground-state solutions are obtained by minimizing the energy of the system within a restricted family of parameterized wave functions. Deep learning methods partially address the limitations of traditional QVMC by representing a rich family of wave functions in terms of neural networks. However, the optimization objective in QVMC remains notoriously hard to minimize and requires second-order optimization methods such as natural gradient. In this paper, we first reformulate energy functional minimization in the space of Born distributions corresponding to particle-permutation (anti-)symmetric wave functions, rather than the space of wave functions. We then interpret QVMC as the Fisher--Rao gradient flow in this distributional space, followed by a projection step onto the variational manifold. This perspective provides us with a principled framework to derive new QMC algorithms, by endowing the distributional space with better metrics, and following the projected gradient flow induced by those metrics. More specifically, we propose "Wasserstein Quantum Monte Carlo" (WQMC), which uses the gradient flow induced by the Wasserstein metric, rather than Fisher--Rao metric, and corresponds to transporting the probability mass, rather than teleporting it. We demonstrate empirically that the dynamics of WQMC results in faster convergence to the ground state of molecular systems

A Non-stochastic Optimization Algorithm for Neural-network Quantum States

Neural-network quantum states (NQS) employ artificial neural networks to encode many-body wave functions in second quantization through variational Monte Carlo (VMC). They have recently been applied to accurately describe electronic wave functions of molecules and have shown the challenges in efficiency comparing with traditional quantum chemistry methods. Here we introduce a general non-stochastic optimization algorithm for NQS in chemical systems, which deterministically generates a selected set of important configurations simultaneously with energy evaluation of NQS. This method bypasses the need for Markov-chain Monte Carlo within the VMC framework, thereby accelerating the entire optimization process. Furthermore, this newly-developed non-stochastic optimization algorithm for NQS offers comparable or superior accuracy compared to its stochastic counterpart and ensures more stable convergence. The application of this model to test molecules exhibiting strong electron correlations provides further insight into the performance of NQS in chemical systems and opens avenues for future enhancements.Comment: 30 pages, 7 figures, and 1 tabl