Strong scaling of general-purpose molecular dynamics simulations on GPUs

We describe a highly optimized implementation of MPI domain decomposition in a GPU-enabled, general-purpose molecular dynamics code, HOOMD-blue (Anderson and Glotzer, arXiv:1308.5587). Our approach is inspired by a traditional CPU-based code, LAMMPS (Plimpton, J. Comp. Phys. 117, 1995), but is implemented within a code that was designed for execution on GPUs from the start (Anderson et al., J. Comp. Phys. 227, 2008). The software supports short-ranged pair force and bond force fields and achieves optimal GPU performance using an autotuning algorithm. We are able to demonstrate equivalent or superior scaling on up to 3,375 GPUs in Lennard-Jones and dissipative particle dynamics (DPD) simulations of up to 108 million particles. GPUDirect RDMA capabilities in recent GPU generations provide better performance in full double precision calculations. For a representative polymer physics application, HOOMD-blue 1.0 provides an effective GPU vs. CPU node speed-up of 12.5x.Comment: 30 pages, 14 figure

All-Atom Modeling of Protein Folding and Aggregation

Theoretical investigations of biorelevant processes in the life-science research require highly optimized simulation methods. Therefore, massively parallel Monte Carlo algorithms, namely MTM, were successfully developed and applied to the field of reversible protein folding allowing the thermodynamic characterization of proteins on an atomistic level. Further, the formation process of trans-membrane pores in the TatA system could be elucidated and the structure of the complex could be predicted

Classical and reactive molecular dynamics: Principles and applications in combustion and energy systems

Molecular dynamics (MD) has evolved into a ubiquitous, versatile and powerful computational method for fundamental research in science branches such as biology, chemistry, biomedicine and physics over the past 60 years. Powered by rapidly advanced supercomputing technologies in recent decades, MD has entered the engineering domain as a first-principle predictive method for material properties, physicochemical processes, and even as a design tool. Such developments have far-reaching consequences, and are covered for the first time in the present paper, with a focus on MD for combustion and energy systems encompassing topics like gas/liquid/solid fuel oxidation, pyrolysis, catalytic combustion, heterogeneous combustion, electrochemistry, nanoparticle synthesis, heat transfer, phase change, and fluid mechanics. First, the theoretical framework of the MD methodology is described systemically, covering both classical and reactive MD. The emphasis is on the development of the reactive force field (ReaxFF) MD, which enables chemical reactions to be simulated within the MD framework, utilizing quantum chemistry calculations and/or experimental data for the force field training. Second, details of the numerical methods, boundary conditions, post-processing and computational costs of MD simulations are provided. This is followed by a critical review of selected applications of classical and reactive MD methods in combustion and energy systems. It is demonstrated that the ReaxFF MD has been successfully deployed to gain fundamental insights into pyrolysis and/or oxidation of gas/liquid/solid fuels, revealing detailed energy changes and chemical pathways. Moreover, the complex physico-chemical dynamic processes in catalytic reactions, soot formation, and flame synthesis of nanoparticles are made plainly visible from an atomistic perspective. Flow, heat transfer and phase change phenomena are also scrutinized by MD simulations. Unprecedented details of nanoscale processes such as droplet collision, fuel droplet evaporation, and CO2 capture and storage under subcritical and supercritical conditions are examined at the atomic level. Finally, the outlook for atomistic simulations of combustion and energy systems is discussed in the context of emerging computing platforms, machine learning and multiscale modelling

Cell mechanics in flow: algorithms and applications

The computer simulations are pervasively used to improve the knowledge about biophysical phenomena and to quantify effects which are difficult to study experimentally. Generally, the numerical methods and models are desired to be as accurate as possible on the chosen length and time scales, but, at the same time, affordable in terms of computations. Until recently, the cell mechanics and blood flow phenomena on the sub-micron resolution could not be rigorously studied using computer simulations. However, within the last decade, advances in methods and hardware catalyzed the development of models for cells mechanics and blood flow modeling which, previously, were considered to be not feasible. In this context, a model should accurately describe a phenomenon, be computationally affordable, and be flexible to be applied to study different biophysical changes. This thesis focuses on the development of the new methods, models, and high-performance software implementation that expand the class of problems which can be studied numerically using particle-based methods. Microvascular networks have complex geometry, often without any symmetry, and to study them we need to tackle computational domains with several inlets and outlets. However, an absence of appropriate boundary conditions for particle- based methods hampers study of the blood flow in these domains. Another obstacle to model complex blood flow problems is the absence the highperformance software. This problem restricts the applicability of the of particlebased cell flow models to relatively small systems. Although there are several validated red blood cell models, to date, there are no models for suspended eukaryotic cells. The present thesis addresses these issues. We introduce new open boundary conditions for particle-based systems and apply them to study blood flow in a part of a microvascular network. We develop a software demonstrating outstanding performance on the largest supercomputers and used it to study blood flow in microfluidic devices. Finally, we present a new eukaryotic cell model which helps in quantifying the effect of sub-cellular components on the cell mechanics during deformations in microfluidic devices

Unstable periodic orbits in turbulent hydrodynamics

In this work we describe a novel parallel space-time algorithm for the computation of periodic solutions of the driven, incompressible Navier-Stokes equations in the turbulent regime. Efforts to apply the machinery of dynamical systems theory to fluid turbulence depend on the ability to accurately and reliably compute such unstable periodic orbits (UPOs). These UPOs can be used to construct the dynamical zeta function of the system, from which very accurate turbulent averages of observables can be extracted from first principles, thus circumventing the inherently statistical description of fluid turbulence. In order to identify these orbits we use a space-time variational principle, first introduced in 2004. This approach has not, to the best of our knowledge, been used before on dynamical systems of high dimension because of the formidable storage and computation required. In this thesis we describe the utilization of petascale high performance computation to the problem of applying this space-time algorithm to hydrodynamic turbulence. The lattice-Boltzmann method is used to simulate the Navier-Stokes equations, due to its locality, and is implemented in a fully-parallel software package using the Message Passing Interface. This implementation, called HYPO4D, was successfully deployed on a large variety of platforms both in the UK and the US with an extremely good scalability to tens of thousands of computing cores. Based on this fluid solver other routines were developed, for the systematic location of suitable candidate spacetime minima and their numerical relaxation, using the gradient descent and conjugate gradient algorithms. Following this methodology, several UPOs are identified in homogeneous turbulence driven by an Arnold-Beltrami-Childress force field in three spatial dimensions, at Reynolds numbers corresponding to weakly-turbulent flow. We characterize the transition to turbulence in the ABC flow and the periodic orbits computed, for a flow with Re = 371, after the transients have died down. The work concludes with a discussion of the potential for this approach to become a new paradigm in the study of driven dissipative dynamical systems

Temporal contrast-dependent modeling of laser-driven solids: studying femtosecond-nanometer interactions and probing

Establishing precise control over the unique beam parameters of laser-accelerated ions from relativistic ultra-short pulse laser-solid interactions has been a major goal for the past 20 years. While the spatio-temporal coupling of laser-pulse and target parameters create transient phenomena at femtosecond-nanometer scales that are decisive for the acceleration performance, these scales have also largely been inaccessible to experimental observation. Computer simulations of laser-driven plasmas provide valuable insight into the physics at play. Nevertheless, predictive capabilities are still lacking due to the massive computational cost to perform these in 3D at high resolution for extended simulation times. This thesis investigates the optimal acceleration of protons from ultra-thin foils following the interaction with an ultra-short ultra-high intensity laser pulse, including realistic contrast conditions up to a picosecond before the main pulse. Advanced ionization methods implemented into the highly scalable, open-source particle-in-cell code PIConGPU enabled this study. Supporting two experimental campaigns, the new methods led to a deeper understanding of the physics of Laser-Wake eld acceleration and Colloidal Crystal melting, respectively, for they now allowed to explain experimental observations with simulated ionization- and plasma dynamics. Subsequently, explorative 3D3V simulations of enhanced laser-ion acceleration were performed on the Swiss supercomputer Piz Daint. There, the inclusion of realistic laser contrast conditions altered the intra-pulse dynamics of the acceleration process significantly. Contrary to a perfect Gaussian pulse, a better spatio-temporal overlap of the protons with the electron sheath origin allowed for full exploitation of the accelerating potential, leading to higher maximum energies. Adapting well-known analytic models allowed to match the results qualitatively and, in chosen cases, quantitatively. However, despite complex 3D plasma dynamics not being reflected within the 1D models, the upper limit of ion acceleration performance within the TNSA scenario can be predicted remarkably well. Radiation signatures obtained from synthetic diagnostics of electrons, protons, and bremsstrahlung photons show that the target state at maximum laser intensity is encoded, previewing how experiments may gain insight into this previously unobservable time frame. Furthermore, as X-ray Free Electron Laser facilities have only recently begun to allow observations at femtosecond-nanometer scales, benchmarking the physics models for solid-density plasma simulations is now in reach. Finally, this thesis presents the first start-to-end simulations of optical-pump, X-ray-probe laser-solid interactions with the photon scattering code ParaTAXIS. The associated PIC simulations guided the planning and execution of an LCLS experiment, demonstrating the first observation of solid-density plasma distribution driven by near-relativistic short-pulse laser pulses at femtosecond-nanometer resolution.Die Erlangung präziser Kontrolle über die einzigartigen Strahlparameter von laserbeschleunigten Ionen aus relativistischen Ultrakurzpuls-Laser-Festkörper-Wechselwirkungen ist ein wesentliches Ziel der letzten 20 Jahre. Während die räumlich-zeitliche Kopplung von Laserpuls und Targetparametern transiente Phänomene auf Femtosekunden- und Nanometerskalen erzeugt, die für den Beschleunigungsprozess entscheidend sind, waren diese Skalen der experimentellen Beobachtung bisher weitgehend unzugänglich. Computersimulationen von lasergetriebenen Plasmen liefern dabei wertvolle Einblicke in die zugrunde liegende Physik. Dennoch mangelt es noch an Vorhersagemöglichkeiten aufgrund des massiven Rechenaufwands, um Parameterstudien in 3D mit hoher Auflösung für längere Simulationszeiten durchzuführen. In dieser Arbeit wird die optimale Beschleunigung von Protonen aus ultradünnen Folien nach der Wechselwirkung mit einem ultrakurzen Ultrahochintensitäts-Laserpuls unter Einbeziehung realistischer Kontrastbedingungen bis zu einer Pikosekunde vor dem Hauptpuls untersucht. Hierbei ermöglichen neu implementierte fortschrittliche Ionisierungsmethoden für den hoch skalierbaren, quelloffenen Partikel-in-Zelle-Code PIConGPU von nun an Studien dieser Art. Bei der Unterstützung zweier Experimentalkampagnen führten diese Methoden zu einem tieferen Verständnis der Laser-Wake eld-Beschleunigung bzw. des Schmelzens kolloidaler Kristalle, da nun experimentelle Beobachtungen mit simulierter Ionisations- und Plasmadynamik erklärt werden konnten. Im Anschluss werden explorative 3D3V Simulationen verbesserter Laser-Ionen-Beschleunigung vorgestellt, die auf dem Schweizer Supercomputer Piz Daint durchgeführt wurden. Dabei veränderte die Einbeziehung realistischer Laserkontrastbedingungen die Intrapulsdynamik des Beschleunigungsprozesses signifikant. Im Gegensatz zu einem perfekten Gauß-Puls erlaubte eine bessere räumlich-zeitliche Überlappung der Protonen mit dem Ursprung der Elektronenwolke die volle Ausnutzung des Beschleunigungspotentials, was zu höheren maximalen Energien führte. Die Adaptation bekannter analytischer Modelle erlaubte es, die Ergebnisse qualitativ und in ausgewählten Fällen auch quantitativ zu bestätigen. Trotz der in den 1D-Modellen nicht abgebildeten komplexen 3D-Plasmadynamik zeigt die Vorhersage erstaunlich gut das obere Limit der erreichbaren Ionen-Energien im TNSA Szenario. Strahlungssignaturen, die aus synthethischen Diagnostiken von Elektronen, Protonen und Bremsstrahlungsphotonen gewonnen wurden, zeigen, dass der Target-Zustand bei maximaler Laserintensität einkodiert ist, was einen Ausblick darauf gibt, wie Experimente Einblicke in dieses bisher unbeobachtbare Zeitfenster gewinnen können. Mit neuen Freie-Elektronen-Röntgenlasern sind Beobachtungen auf Femtosekunden-Nanometerskalen endlich zugänglich geworden. Damit liegt ein Benchmarking der physikalischen Modelle für Plasmasimulationen bei Festkörperdichte nun in Reichweite, aber Experimente sind immer noch selten, komplex, und schwer zu interpretieren. Zuletzt werden daher in dieser Arbeit die ersten Start-zu-End-Simulationen der Pump-Probe Wechselwirkungen von optischem sowie Röntgenlaser mit Festkörpern mittels des Photonenstreu-Codes ParaTAXIS vorgestellt. Darüber hinaus dienten die zugehörigen PIC-Simulationen als Grundlage für die Planung und Durchführung eines LCLS-Experiments zur erstmaligen Beobachtung einer durch nah-relativistische Kurzpuls-Laserpulse getriebenen Festkörper-Plasma-Dichte, dessen Auflösungsbereich gleichzeitig bis auf Femtosekunden und Nanometer vordrang

Temporal contrast-dependent modeling of laser-driven solids - studying femtosecond-nanometer interactions and probing

Establishing precise control over the unique beam parameters of laser-accelerated ions from relativistic ultra-short pulse laser-solid interactions has been a major goal for the past 20 years. While the spatio-temporal coupling of laser-pulse and target parameters create transient phenomena at femtosecond-nanometer scales that are decisive for the acceleration performance, these scales have also largely been inaccessible to experimental observation. Computer simulations of laser-driven plasmas provide valuable insight into the physics at play. Nevertheless, predictive capabilities are still lacking due to the massive computational cost to perform these in 3D at high resolution for extended simulation times. This thesis investigates the optimal acceleration of protons from ultra-thin foils following the interaction with an ultra-short ultra-high intensity laser pulse, including realistic contrast conditions up to a picosecond before the main pulse. Advanced ionization methods implemented into the highly scalable, open-source particle-in-cell code PIConGPU enabled this study. Supporting two experimental campaigns, the new methods led to a deeper understanding of the physics of Laser-Wakefield acceleration and Colloidal Crystal melting, respectively, for they now allowed to explain experimental observations with simulated ionization- and plasma dynamics. Subsequently, explorative 3D3V simulations of enhanced laser-ion acceleration were performed on the Swiss supercomputer Piz Daint. There, the inclusion of realistic laser contrast conditions altered the intra-pulse dynamics of the acceleration process significantly. Contrary to a perfect Gaussian pulse, a better spatio-temporal overlap of the protons with the electron sheath origin allowed for full exploitation of the accelerating potential, leading to higher maximum energies. Adapting well-known analytic models allowed to match the results qualitatively and, in chosen cases, quantitatively. However, despite complex 3D plasma dynamics not being reflected within the 1D models, the upper limit of ion acceleration performance within the TNSA scenario can be predicted remarkably well. Radiation signatures obtained from synthetic diagnostics of electrons, protons, and bremsstrahlung photons show that the target state at maximum laser intensity is encoded, previewing how experiments may gain insight into this previously unobservable time frame. Furthermore, as X-ray Free Electron Laser facilities have only recently begun to allow observations at femtosecond-nanometer scales, benchmarking the physics models for solid-density plasma simulations is now in reach. Finally, this thesis presents the first start-to-end simulations of optical-pump, X-ray-probe laser-solid interactions with the photon scattering code ParaTAXIS. The associated PIC simulations guided the planning and execution of an LCLS experiment, demonstrating the first observation of solid-density plasma distribution driven by near-relativistic short-pulse laser pulses at femtosecond-nanometer resolution