Flexible Multi-Dimensional FFTs for Plane Wave Density Functional Theory Codes

Multi-dimensional Fourier transforms are key mathematical building blocks that appear in a wide range of applications from materials science, physics, chemistry and even machine learning. Over the past years, a multitude of software packages targeting distributed multi-dimensional Fourier transforms have been developed. Most variants attempt to offer efficient implementations for single transforms applied on data mapped onto rectangular grids. However, not all scientific applications conform to this pattern, i.e. plane wave Density Functional Theory codes require multi-dimensional Fourier transforms applied on data represented as batches of spheres. Typically, the implementations for this use case are hand-coded and tailored for the requirements of each application. In this work, we present the Fastest Fourier Transform from Berkeley (FFTB) a distributed framework that offers flexible implementations for both regular/non-regular data grids and batched/non-batched transforms. We provide a flexible implementations with a user-friendly API that captures most of the use cases. Furthermore, we provide implementations for both CPU and GPU platforms, showing that our approach offers improved execution time and scalability on the HP Cray EX supercomputer. In addition, we outline the need for flexible implementations for different use cases of the software package.Comment: 17 pages, 9 figure

Diverse Manifestations of Electron-Phonon Coupling in a Kagome Superconductor

Recent angle-resolved photoemission spectroscopy (ARPES) experiments on the kagome metal CsV_{3}Sb_{5} revealed distinct multimodal dispersion kinks and nodeless superconducting gaps across multiple electron bands. The prominent photoemission kinks suggest a definitive coupling between electrons and certain collective modes, yet the precise nature of this interaction and its connection to superconductivity remain to be established. Here, employing the state-of-the-art ab initio many-body perturbation theory computation, we present direct evidence that electron-phonon (e-ph) coupling induces the multimodal photoemission kinks in CsV_{3}Sb_{5}, and profoundly, drives the nodeless s-wave superconductivity, showcasing the diverse manifestations of the e-ph coupling. Our calculations well capture the experimentally measured kinks and their fine structures, and reveal that vibrations from different atomic species dictate the multimodal behavior. Results from anisotropic GW-Eliashberg equations predict a phonon-mediated superconductivity with nodeless s-wave gaps, in excellent agreement with various ARPES and scanning tunneling spectroscopy measurements. Despite the universal origin of the e-ph coupling, the contributions of several characteristic phonon vibrations vary in different phenomena, highlighting a versatile role of e-ph coupling in shaping the low-energy excitations of kagome metals

Photoinduced Electron–Nuclear Dynamics of Fullerene and Its Monolayer Networks in Solvated Environments

The recently synthesized monolayer fullerene network in a quasi-hexagonal phase (qHP-C60) exhibits superior electron mobility and optoelectronic properties compared to molecular fullerene (C60), making it highly promising for a variety of applications. However, the microscopic carrier dynamics of qHP-C60 remain unclear, particularly in realistic environments, which are of significant importance for applications in optoelectronic devices. Unfortunately, traditional ab initio methods are prohibitive for capturing the real-time carrier dynamics of such large systems due to their high computational cost. In this work, we present the first real-time electron-nuclear dynamics study of qHP-C60 using velocity-gauge density functional tight binding, which enables us to perform several picoseconds of excited-state electron-nuclear dynamics simulations for nanoscale systems with periodic boundary conditions. When applied to C60, qHP-C60, and their solvated counterparts, we demonstrate that water/moisture significantly increases the electron-hole recombination time in C60 but has little impact on qHP-C60. Our excited-state electron-nuclear dynamics calculations show that qHP-C60 is extremely unique and enable exploration of time-resolved dynamics for understanding excited-state processes of large systems in complex, solvated environments

Static Subspace Approximation for Random Phase Approximation Correlation Energies: Applications to Materials for Catalysis and Electrochemistry

Modeling complex materials using high-fidelity, ab initio methods at low cost is a fundamental goal for quantum chemical software packages. The GW approximation and random phase approximation (RPA) provide a unified description of both electronic structure and total energies using the same physics in a many-body perturbative approach that can be more accurate than generalized-gradient density functional theory (DFT) methods. However, GW/RPA implementations have historically been limited to either specific materials classes or application toward small chemical systems. The static subspace approximation allows for reduced cost full-frequency GW/RPA calculations and has previously been benchmarked thoroughly for GW calculations. Here, we describe our approach to including partial occupations of electronic orbitals in full-frequency GW and RPA calculations for the study of electrocatalysts. We benchmarked RPA total energy calculations using the subspace approximation across a diverse test suite of materials for a variety of computational parameters. The benchmarking quantifies the impact of different extrapolation procedures for representing the static polarizability at infinite screened cutoff, and shows that using screened cutoffs above 20-25 Ryd result in diminishing accuracy returns for predicting RPA total energies. Additionally, for moderately sized electrocatalytic models, 2-3 times fewer computational resources are used to compute RPA total energies by representing the static polarizability with 20-30% of the static subspace basis, with an error of approximately 0.01 eV or better in RPA adsorption energy calculations. Finally, we show that for these electrochemical models RPA can shift DFT adsorption energy shifts by up to 0.5 eV and that GW can frequently shift DFT eigenvalues of surface and adsorbate states by approximately 0.5-1 eV

Static Subspace Approximation for Random Phase Approximation Correlation Energies: Implementation and Performance

Developing theoretical understanding of complex reactions and processes at interfaces requires using methods that go beyond semilocal density functional theory to accurately describe the interactions between solvent, reactants and substrates. Methods based on many-body perturbation theory, such as the random phase approximation (RPA), have previously been limited due to their computational complexity. However, this is now a surmountable barrier due to the advances in computational power available, in particular through modern GPU-based supercomputers. In this work, we describe the implementation of RPA calculations within BerkeleyGW and show its favorable computational performance on large complex systems relevant for catalysis and electrochemistry applications. Our implementation builds off of the static subspace approximation which, by employing a compressed representation of the frequency dependent polarizability, enables the evaluation of the RPA correlation energy with significant acceleration and systematically controllable accuracy. We find that the computational cost of calculating the RPA correlation energy scales only linearly with system size for systems containing up to 50 thousand bands, and is expected to scale quadratically thereafter. We also show excellent strong scaling results across several supercomputers, demonstrating the performance and portability of this implementation.Comment: 10 pages, 5 figure

Cost-Effective Methodology for Complex Tuning Searches in HPC: Navigating Interdependencies and Dimensionality

Tuning searches are pivotal in High-Performance Computing (HPC), addressing complex optimization challenges in computational applications. The complexity arises not only from finely tuning parameters within routines but also potential interdependencies among them, rendering traditional optimization methods inefficient. Instead of scrutinizing interdependencies among parameters and routines, practitioners often face the dilemma of conducting independent tuning searches for each routine, thereby overlooking interdependence, or pursuing a more resource-intensive joint search for all routines. This decision is driven by the consideration that some interdependence analysis and high-dimensional decomposition techniques in literature may be prohibitively expensive in HPC tuning searches. Our methodology adapts and refines these methods to ensure computational feasibility while maximizing performance gains in real-world scenarios. Our methodology leverages a cost-effective interdependence analysis to decide whether to merge several tuning searches into a joint search or conduct orthogonal searches. Tested on synthetic functions with varying levels of parameter interdependence, our methodology efficiently explores the search space. In comparison to Bayesian-optimization-based full independent or fully joint searches, our methodology suggested an optimized breakdown of independent and merged searches that led to final configurations up to 8% more accurate, reducing the search time by up to 95%. When applied to GPU-offloaded Real-Time Time-Dependent Density Functional Theory (RT-TDDFT), an application in computational materials science that challenges modern HPC autotuners, our methodology achieved an effective tuning search. Its adaptability and efficiency extend beyond RT-TDDFT, making it valuable for related applications in HPC

QRCODE: Massively parallelized real-time time-dependent density functional theory for periodic systems

We present a new software module, QRCODE (Quantum Research for Calculating Optically Driven Excitations), for massively parallelized real-time time-dependent density functional theory (RT-TDDFT) calculations of periodic systems in the open-source Qbox software package. Our approach utilizes a custom implementation of a fast Fourier transformation scheme that significantly reduces inter-node message passing interface (MPI) communication of the major computational kernel and shows impressive scaling up to 16,344 CPU cores. In addition to improving computational performance, QRCODE contains a suite of various time propagators for accurate RT-TDDFT calculations. As benchmark applications of QRCODE, we calculate the current density and optical absorption spectra of hexagonal boron nitride (h-BN) and photo-driven reaction dynamics of the ozone-oxygen reaction. We also calculate the second and higher harmonic generation of monolayer and multi-layer boron nitride structures as examples of large material systems. Our optimized implementation of RT-TDDFT in QRCODE enables large-scale calculations of real-time electron dynamics of chemical and material systems with enhanced computational performance and impressive scaling across several thousand CPU cores

Measurement of the cosmic ray spectrum above 4×10184{\times}10^{18} eV using inclined events detected with the Pierre Auger Observatory

A measurement of the cosmic-ray spectrum for energies exceeding $4{\times}10^{18}$ eV is presented, which is based on the analysis of showers with zenith angles greater than $60^{\circ}$ detected with the Pierre Auger Observatory between 1 January 2004 and 31 December 2013. The measured spectrum confirms a flux suppression at the highest energies. Above $5.3{\times}10^{18}$ eV, the "ankle", the flux can be described by a power law $E^{-\gamma}$ with index $\gamma=2.70 \pm 0.02 \,\text{(stat)} \pm 0.1\,\text{(sys)}$ followed by a smooth suppression region. For the energy ($E_\text{s}$) at which the spectral flux has fallen to one-half of its extrapolated value in the absence of suppression, we find $E_\text{s}=(5.12\pm0.25\,\text{(stat)}^{+1.0}_{-1.2}\,\text{(sys)}){\times}10^{19}$ eV.Comment: Replaced with published version. Added journal reference and DO

Standalone vertex finding in the ATLAS muon spectrometer

A dedicated reconstruction algorithm to find decay vertices in the ATLAS muon spectrometer is presented. The algorithm searches the region just upstream of or inside the muon spectrometer volume for multi-particle vertices that originate from the decay of particles with long decay paths. The performance of the algorithm is evaluated using both a sample of simulated Higgs boson events, in which the Higgs boson decays to long-lived neutral particles that in turn decay to bbar b final states, and pp collision data at √s = 7 TeV collected with the ATLAS detector at the LHC during 2011

Measurement of inclusive two-particle angular correlations in pp collisions with the ATLAS detector at the LHC

We present a measurement of two-particle angular correlations in proton- proton collisions at s√=900 GeV and 7 TeV. The collision events were collected during 2009 and 2010 with the ATLAS detector at the Large Hadron Collider using a single-arm minimum bias trigger. Correlations are measured for charged particles produced in the kinematic range of transverse momentum p T  > 100 MeV and pseudorapidity |η| < 2.5. A complex structure in pseudorapidity and azimuth is observed at both collision energies. Results are compared to pythia 8 and herwig++ as well as to the AMBT2B, DW and Perugia 2011 tunes of pythia 6. The data are not satisfactorily described by any of these models