A Massively Parallel Algorithm for the Approximate Calculation of Inverse p-th Roots of Large Sparse Matrices

We present the submatrix method, a highly parallelizable method for the approximate calculation of inverse p-th roots of large sparse symmetric matrices which are required in different scientific applications. We follow the idea of Approximate Computing, allowing imprecision in the final result in order to be able to utilize the sparsity of the input matrix and to allow massively parallel execution. For an n x n matrix, the proposed algorithm allows to distribute the calculations over n nodes with only little communication overhead. The approximate result matrix exhibits the same sparsity pattern as the input matrix, allowing for efficient reuse of allocated data structures. We evaluate the algorithm with respect to the error that it introduces into calculated results, as well as its performance and scalability. We demonstrate that the error is relatively limited for well-conditioned matrices and that results are still valuable for error-resilient applications like preconditioning even for ill-conditioned matrices. We discuss the execution time and scaling of the algorithm on a theoretical level and present a distributed implementation of the algorithm using MPI and OpenMP. We demonstrate the scalability of this implementation by running it on a high-performance compute cluster comprised of 1024 CPU cores, showing a speedup of 665x compared to single-threaded execution

Fragment Approach to Constrained Density Functional Theory Calculations using Daubechies Wavelets

In a recent paper we presented a linear scaling Kohn-Sham density functional theory (DFT) code based on Daubechies wavelets, where a minimal set of localized support functions is optimized in situ and therefore adapted to the chemical properties of the molecular system. Thanks to the systematically controllable accuracy of the underlying basis set, this approach is able to provide an optimal contracted basis for a given system: accuracies for ground state energies and atomic forces are of the same quality as an uncontracted, cubic scaling approach. This basis set offers, by construction, a natural subset where the density matrix of the system can be projected. In this paper we demonstrate the flexibility of this minimal basis formalism in providing a basis set that can be reused as-is, i.e. without reoptimization, for charge-constrained DFT calculations within a fragment approach. Support functions, represented in the underlying wavelet grid, of the template fragments are roto-translated with high numerical precision to the required positions and used as projectors for the charge weight function. We demonstrate the interest of this approach to express highly precise and efficient calculations for preparing diabatic states and for the computational setup of systems in complex environments

Accurate and efficient linear scaling DFT calculations with universal applicability

Density Functional Theory calculations traditionally suffer from an inherent cubic scaling with respect to the size of the system, making big calculations extremely expensive. This cubic scaling can be avoided by the use of so-called linear scaling algorithms, which have been developed during the last few decades. In this way it becomes possible to perform ab-initio calculations for several tens of thousands of atoms or even more within a reasonable time frame. However, even though the use of linear scaling algorithms is physically well justified, their implementation often introduces some small errors. Consequently most implementations offering such a linear complexity either yield only a limited accuracy or, if one wants to go beyond this restriction, require a tedious fine tuning of many parameters. In our linear scaling approach within the BigDFT package, we were able to overcome this restriction. Using an ansatz based on localized support functions expressed in an underlying Daubechies wavelet basis -- which offers ideal properties for accurate linear scaling calculations -- we obtain an amazingly high accuracy and a universal applicability while still keeping the possibility of simulating large systems with only a moderate demand of computing resources

Metrics for measuring distances in configuration spaces

In order to characterize molecular structures we introduce configurational fingerprint vectors which are counterparts of quantities used experimentally to identify structures. The Euclidean distance between the configurational fingerprint vectors satisfies the properties of a metric and can therefore safely be used to measure dissimilarities between configurations in the high dimensional configuration space. We show that these metrics correlate well with the RMSD between two configurations if this RMSD is obtained from a global minimization over all translations, rotations and permutations of atomic indices. We introduce a Monte Carlo approach to obtain this global minimum of the RMSD between configurations

Daubechies Wavelets for Linear Scaling Density Functional Theory

We demonstrate that Daubechies wavelets can be used to construct a minimal set of optimized localized contracted basis functions in which the Kohn-Sham orbitals can be represented with an arbitrarily high, controllable precision. Ground state energies and the forces acting on the ions can be calculated in this basis with the same accuracy as if they were calculated directly in a Daubechies wavelets basis, provided that the amplitude of these contracted basis functions is sufficiently small on the surface of the localization region, which is guaranteed by the optimization procedure described in this work. This approach reduces the computational costs of DFT calculations, and can be combined with sparse matrix algebra to obtain linear scaling with respect to the number of electrons in the system. Calculations on systems of 10,000 atoms or more thus become feasible in a systematic basis set with moderate computational resources. Further computational savings can be achieved by exploiting the similarity of the contracted basis functions for closely related environments, e.g. in geometry optimizations or combined calculations of neutral and charged systems

Efficient Computation of Sparse Matrix Functions for Large-Scale Electronic Structure Calculations: The CheSS Library

We present CheSS, the “Chebyshev Sparse Solvers” library, which has been designed to solve typical problems arising in large-scale electronic structure calculations using localized basis sets. The library is based on a flexible and efficient expansion in terms of Chebyshev polynomials and presently features the calculation of the density matrix, the calculation of matrix powers for arbitrary powers, and the extraction of eigenvalues in a selected interval. CheSS is able to exploit the sparsity of the matrices and scales linearly with respect to the number of nonzero entries, making it well-suited for large-scale calculations. The approach is particularly adapted for setups leading to small spectral widths of the involved matrices and outperforms alternative methods in this regime. By coupling CheSS to the DFT code BigDFT, we show that such a favorable setup is indeed possible in practice. In addition, the approach based on Chebyshev polynomials can be massively parallelized, and CheSS exhibits excellent scaling up to thousands of cores even for relatively small matrix sizes.We gratefully acknowledge the support of the MaX (SM) and POP (MW) projects, which have received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 676598 and 676553, respectively. This work was also supported by the Energy oriented Centre of Excellence (EoCoE), grant agreement number 676629, funded within the Horizon2020 framework of the European Union, as well as by the Next-Generation Supercomputer project (the K computer project) and the FLAGSHIP2020 within the priority study5 (Development of new fundamental technologies for high-efficiency energy creation, conversion/storage and use) from the Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan. We (LG, DC, WD, TN) gratefully acknowledge the joint CEA-RIKEN collaboration action.Peer ReviewedPostprint (author's final draft

Lifetime effects and satellites in the photoelectron spectrum of tungsten metal

Tungsten (W) is an important and versatile transition metal and has a firm place at the heart of many technologies. A popular experimental technique for the characterization of tungsten and tungsten-based compounds is x-ray photoelectron spectroscopy (XPS), which enables the assessment of chemical states and electronic structure through the collection of core level and valence band spectra. However, in the case of tungsten metal, open questions remain regarding the origin, nature, and position of satellite features that are prominent in the photoelectron spectrum. These satellites are a fingerprint of the electronic structure of the material and have not been thoroughly investigated, at times leading to their misinterpretation. The present work combines high-resolution soft and hard x-ray photoelectron spectroscopy (SXPS and HAXPES) with reflected electron energy loss spectroscopy (REELS) and a multitiered ab initio theoretical approach, including density functional theory (DFT) and many-body perturbation theory (G0W0 and GW + C ), to disentangle the complex set of experimentally observed satellite features attributed to the generation of plasmons and interband transitions. This combined experiment-theory strategy is able to uncover previously undocumented satellite features, improving our understanding of their direct relationship to tungsten's electronic structure. Furthermore, it lays the groundwork for future studies into tungsten-based mixed-metal systems and holds promise for the reassessment of the photoelectron spectra of other transition and post-transition metals, where similar questions regarding satellite features remain.CK acknowledges the support from the Department of Chemistry, UCL. NKF acknowledges support from the Engineering and Physical Sciences Research Council (EP/L015277/1). AR acknowledges the support fromthe Analytical Chemistry Trust Fund for her CAMS-UK Fellowship. LER acknowledges support from an EPSRC Early Career Research Fellowship (EP/P033253/1). JL and JMK acknowledge funding from EPSRC under Grant No. EP/R002010/1 and from a Royal Society University Research Fellowship (URF/R/191004). This work used the ARCHER UK National Supercomputing Service via JL’s membership of the HEC Materials Chemistry Consortium of UK, which is funded by EPSRC (EP/L000202). JJGM and SM acknowledge the support from the FusionCAT project (001-P-001722) cofinanced by the European Union Regional Development Fund within the framework of the ERDF Operational Program of Catalonia 2014-2020 with a grant of 50% of total cost eligible, the access to computational resources at MareNostrum and the technical support provided by BSC (RES-QS-2020-3-0026). Part of this work was carried out using supercomputer resources provided under the EU-JA Broader Approach collaboration in the Computational Simulation Centre of International Fusion Energy Research Centre (IFERC-CSC)Peer Reviewed"Article signat per 13 autors/es: C. Kalha, L. E. Ratcliff, J. J. Gutiérrez Moreno, S. Mohr, M. Mantsinen, N. K. Fernando, P. K. Thakur, T.-L. Lee, H.-H. Tseng, T. S. Nunney, J. M. Kahk, J. Lischner, and A. Regoutz"Postprint (author's final draft

Towards electronic structure-based ab-initio molecular dynamics simulations with hundreds of millions of atoms

We push the boundaries of electronic structure-based ab-initio molecular dynamics (AIMD) beyond 100 million atoms. This scale is otherwise barely reachable with classical force-field methods or novel neural network and machine learning potentials. We achieve this breakthrough by combining innovations in linear-scaling AIMD, efficient and approximate sparse linear algebra, low and mixed-precision floating-point computation on GPUs, and a compensation scheme for the errors introduced by numerical approximations. The core of our work is the non-orthogonalized local submatrix method (NOLSM), which scales very favorably to massively parallel computing systems and translates large sparse matrix operations into highly parallel, dense matrix operations that are ideally suited to hardware accelerators. We demonstrate that the NOLSM method, which is at the center point of each AIMD step, is able to achieve a sustained performance of 324 PFLOP/s in mixed FP16/FP32 precision corresponding to an efficiency of 67.7% when running on 1536 NVIDIA A100 GPUs.The authors gratefully acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-centre.eu) for funding this project by providing computing time on the GCS Supercomputer JUWELS Booster at Jülich Supercomputing Centre (JSC). Additionally, we would like to thank for funding of this project by computing time provided by the Paderborn Center for Parallel Computing (PC), as well as the Federal Ministry of Education and Research (BMBF) and the state of North Rhine-Westphalia as part of the NHR Program. T.D.K. received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant Agreement No. 716142). T.D.K. and C.P. kindly acknowledge funding from Paderborn University’s research award for “GreenIT”. Finally, we thank Thomas Müller (JSC), Paul F. Baumeister (JSC), and Markus Hrywniak (NVIDIA) for valuable discussions.Peer Reviewed"Article signat per 11 autors/es: Robert Schade, Tobias Kenter, Hossam Elgabarty, Michael Lass, Ole Schütt, Alfio Lazzaro, Hans Pabst, Stephan Mohr, Jürg Hutter, Thomas D. Kühne, Christian Plessl"Postprint (published version

Relativistic central--field Green's functions for the RATIP package

From perturbation theory, Green's functions are known for providing a simple and convenient access to the (complete) spectrum of atoms and ions. Having these functions available, they may help carry out perturbation expansions to any order beyond the first one. For most realistic potentials, however, the Green's functions need to be calculated numerically since an analytic form is known only for free electrons or for their motion in a pure Coulomb field. Therefore, in order to facilitate the use of Green's functions also for atoms and ions other than the hydrogen--like ions, here we provide an extension to the Ratip program which supports the computation of relativistic (one--electron) Green's functions in an -- arbitrarily given -- central--field potential \rV(r). Different computational modes have been implemented to define these effective potentials and to generate the radial Green's functions for all bound--state energies $E < 0$. In addition, care has been taken to provide a user--friendly component of the Ratip package by utilizing features of the Fortran 90/95 standard such as data structures, allocatable arrays, or a module--oriented design.Comment: 20 pages, 1 figur