TweTriS: Twenty trillion-atom simulation

Significant improvements are presented for the molecular dynamics code ls1 mardyn — a linked cell-based code for simulating a large number of small, rigid molecules with application areas in chemical engineering. The changes consist of a redesign of the SIMD vectorization via wrappers, MPI improvements and a software redesign to allow memory-efficient execution with the production trunk to increase portability and extensibility. Two novel, memory-efficient OpenMP schemes for the linked cell-based force calculation are presented, which are able to retain Newton’s third law optimization. Comparisons to well-optimized Verlet list-based codes, such as LAMMPS and GROMACS, demonstrate the viability of the linked cell-based approach. The present version of ls1 mardyn is used to run simulations on entire supercomputers, maximizing the number of sampled atoms. Compared to the preceding version of ls1 mardyn on the entire set of 9216 nodes of SuperMUC, Phase 1, 27% more atoms are simulated. Weak scaling performance is increased by up to 40% and strong scaling performance by up to more than 220%. On Hazel Hen, strong scaling efficiency of up to 81% and 189 billion molecule updates per second is attained, when scaling from 8 to 7168 nodes. Moreover, a total of 20 trillion atoms is simulated at up to 88% weak scaling efficiency running at up to 1.33 PFLOPS. This represents a fivefold increase in terms of the number of atoms simulated to date.BMBF, 01IH16008, Verbundprojekt: TaLPas - Task-basierte Lastverteilung und Auto-Tuning in der Partikelsimulatio

PetaFLOP Molecular Dynamics for Engineering Applications

Molecular dynamics (MD) simulations enable the investigation of multicomponent and multiphase processes relevant to engineering applications, such as droplet coalescence or bubble formation. These scenarios require the simulation of ensembles containing a large number of molecules. We present recent advances within the MD framework ls1 mardyn which is being developed with particular regard to this class of problems. We discuss several OpenMP schemes that deliver optimal performance at node-level. We have further introduced nonblocking communication and communication hiding for global collective operations. Together with revised data structures and vectorization, these improvements unleash PetaFLOP performance and enable multi-trillion atom simulations on the HLRS supercomputer Hazel Hen. We further present preliminary results achieved for droplet coalescence scenarios at a smaller scale.BMBF, 01IH16008, Verbundprojekt: TaLPas - Task-basierte Lastverteilung und Auto-Tuning in der Partikelsimulatio

Coarse grained force field for the molecular simulation of natural gases and condensates

AbstractThe atomistically-detailed molecular modelling of petroleum fluids is challenging, amongst other aspects, due to the very diverse multicomponent and asymmetric nature of the mixtures in question. Complicating matters further, the time scales for many important processes can be much larger than the current and foreseeable capacity of modern computers running fully-atomistic models. To overcome these limitations, a coarse grained (CG) model is proposed where some of the less-important degrees of freedom are safely integrated out, leaving as key parameters the average energy levels, the molecular conformations and the range of the Mie intermolecular potentials employed as the basis of the model. The parametrization is performed by using an analytical equation of state of the statistical associating fluid theory (SAFT) family to link the potential parameters to macroscopically observed thermophysical properties. The parameters found through this top-down approach are used directly in molecular dynamics simulations of multi-component multi-phase systems. The procedure is exemplified by calculating the phase envelope of the methane–decane binary and of two synthetic light condensate mixtures. A methodology based on the discrete expansion of a mixture is used to determine the bubble points of these latter mixtures, with an excellent agreement to experimental data. The model presented is entirely predictive and an abridged table of parameters for some fluids of interest is provided

GPU fast multipole method with lambda-dynamics features

A significant and computationally most demanding part of molecular dynamics simulations is the calculation of long-range electrostatic interactions. Such interactions can be evaluated directly by the naïve pairwise summation algorithm, which is a ubiquitous showcase example for the compute power of graphics processing units (GPUS). However, the pairwise summation has O(N^2) computational complexity for N interacting particles; thus, an approximation method with a better scaling is required. Today, the prevalent method for such approximation in the field is particle mesh Ewald (PME). PME takes advantage of fast Fourier transforms (FFTS) to approximate the solution efficiently. However, as the underlying FFTS require all-to-all communication between ranks, PME runs into a communication bottleneck. Such communication overhead is negligible only for a moderate parallelization. With increased parallelization, as needed for high-performance applications, the usage of PME becomes unprofitable. Another PME drawback is its inability to perform constant pH simulations efficiently. In such simulations, the protonation states of a protein are allowed to change dynamically during the simulation. The description of this process requires a separate evaluation of the energies for each protonation state. This can not be calculated efficiently with PME as the algorithm requires a repeated FFT for each state, which leads to a linear overhead with respect to the number of states. For a fast approximation of pairwise Coulombic interactions, which does not suffer from PME drawbacks, the Fast Multipole Method (FMM) has been implemented and fully parallelized with CUDA. To assure the optimal FMM performance for diverse MD systems multiple parallelization strategies have been developed. The algorithm has been efficiently incorporated into GROMACS and subsequently tested to determine the optimal FMM parameter set for MD simulations. Finally, the FMM has been incorporated into GROMACS to allow for out-of-the-box electrostatic calculations. The performance of the single-GPU FMM implementation, tested in GROMACS 2019, achieves about a third of highly optimized CUDA PME performance when simulating systems with uniform particle distributions. However, the FMM is expected to outperform PME at high parallelization because the FMM global communication overhead is minimal compared to that of PME. Further, the FMM has been enhanced to provide the energies of an arbitrary number of titratable sites as needed in the constant-pH method. The extension is not fully optimized yet, but the first results show the strength of the FMM for constant pH simulations. For a relatively large system with half a million particles and more than a hundred titratable sites, a straightforward approach to compute alternative energies requires the repetition of a simulation for each state of the sites. The FMM calculates all energy terms only a factor 1.5 slower than a single simulation step. Further improvements of the GPU implementation are expected to yield even more speedup compared to the actual implementation.2021-11-1

Algorithmische und Code-Optimierungen Molekulardynamiksimulationen für Verfahrenstechnik

The focus of this work lies on implementational improvements and, in particular, node-level performance optimization of the simulation software ls1-mardyn. Through data structure improvements, SIMD vectorization and, especially, OpenMP parallelization, the world’s first simulation of 2*1013 molecules at over 1 PFLOP/sec was enabled. To allow for long-range interactions, the Fast Multipole Method was introduced to ls1-mardyn. The algorithm was optimized for sequential, shared-memory, and distributed-memory execution on up to 32,768 MPI processes.Der Fokus dieser Arbeit liegt auf Code-Optimierungen und insbesondere Leistungsoptimierung auf Knoten-Ebene für die Simulationssoftware ls1-mardyn. Durch verbesserte Datenstrukturen, SIMD-Vektorisierung und vor allem OpenMP-Parallelisierung wurde die weltweit erste Petaflop-Simulation von 2*1013 Molekülen ermöglicht. Zur Simulation von langreichweitigen Wechselwirkungen wurde die Fast-Multipole-Methode in ls1-mardyn eingeführt. Sequenzielle, Shared- und Distributed-Memory-Optimierungen wurden angewandt und erlaubten eine Ausführung auf bis zu 32768 MPI-Prozessen

Various damage mechanisms in carbon and silicon materials under femtosecond x-ray irradiation

We review the results of our research on damage mechanisms in materials irradiated with femtosecond free-electron-laser (FEL) pulses. They were obtained using our hybrid approach, XTANT (X-ray-induced Thermal And Nonthermal Transitions). Various damage mechanisms are discussed with respect to the pulse fluence and material properties on examples of diamond, amorphous carbon, C60 crystal, and silicon. We indicate conditions: producing thermal melting of targets as a result of electron-ion energy exchange; nonthermal phase transitions due to modification of the interatomic potential; Coulomb explosion due to accumulated net charge in finite-size systems; spallation or ablation at higher fluences due to detachment of sample fragments; and warm dense matter formation. Transient optical coefficients are compared with experimental data whenever available, proving the validity of our modeling approach. Predicted diffraction patterns can be compared with the results of ongoing or future FEL experiments. Limitations of our model and possible future directions of development are outlined.Comment: This brief review is submitted for publicatio