A massively parallel semi-Lagrangian solver for the six-dimensional Vlasov-Poisson equation

This paper presents an optimized and scalable semi-Lagrangian solver for the Vlasov-Poisson system in six-dimensional phase space. Grid-based solvers of the Vlasov equation are known to give accurate results. At the same time, these solvers are challenged by the curse of dimensionality resulting in very high memory requirements, and moreover, requiring highly efficient parallelization schemes. In this paper, we consider the 6d Vlasov-Poisson problem discretized by a split-step semi-Lagrangian scheme, using successive 1d interpolations on 1d stripes of the 6d domain. Two parallelization paradigms are compared, a remapping scheme and a classical domain decomposition approach applied to the full 6d problem. From numerical experiments, the latter approach is found to be superior in the massively parallel case in various respects. We address the challenge of artificial time step restrictions due to the decomposition of the domain by introducing a blocked one-sided communication scheme for the purely electrostatic case and a rotating mesh for the case with a constant magnetic field. In addition, we propose a pipelining scheme that enables to hide the costs for the halo communication between neighbor processes efficiently behind useful computation. Parallel scalability on up to 65k processes is demonstrated for benchmark problems on a supercomputer

ColDICE: a parallel Vlasov-Poisson solver using moving adaptive simplicial tessellation

Resolving numerically Vlasov-Poisson equations for initially cold systems can be reduced to following the evolution of a three-dimensional sheet evolving in six-dimensional phase-space. We describe a public parallel numerical algorithm consisting in representing the phase-space sheet with a conforming, self-adaptive simplicial tessellation of which the vertices follow the Lagrangian equations of motion. The algorithm is implemented both in six- and four-dimensional phase-space. Refinement of the tessellation mesh is performed using the bisection method and a local representation of the phase-space sheet at second order relying on additional tracers created when needed at runtime. In order to preserve in the best way the Hamiltonian nature of the system, refinement is anisotropic and constrained by measurements of local Poincar\'e invariants. Resolution of Poisson equation is performed using the fast Fourier method on a regular rectangular grid, similarly to particle in cells codes. To compute the density projected onto this grid, the intersection of the tessellation and the grid is calculated using the method of Franklin and Kankanhalli (1993) generalised to linear order. As preliminary tests of the code, we study in four dimensional phase-space the evolution of an initially small patch in a chaotic potential and the cosmological collapse of a fluctuation composed of two sinusoidal waves. We also perform a "warm" dark matter simulation in six-dimensional phase-space that we use to check the parallel scaling of the code.Comment: Code and illustration movies available at: http://www.vlasix.org/index.php?n=Main.ColDICE - Article submitted to Journal of Computational Physic

Fluid Simulations of Three-Dimensional Reconnection that Capture the Lower-Hybrid Drift Instability

Fluid models that approximate kinetic effects have received attention recently in the modelling of large scale plasmas such as planetary magnetospheres. In three-dimensional reconnection, both reconnection itself and current sheet instabilities need to be represented appropriately. We show that a heat flux closure based on pressure gradients enables a ten moment fluid model to capture key properties of the lower-hybrid drift instability (LHDI) within a reconnection simulation. Characteristics of the instability are examined with kinetic and fluid continuum models, and its role in the three-dimensional reconnection simulation is analysed. The saturation level of the electromagnetic LHDI is higher than expected which leads to strong kinking of the current sheet. Therefore, the magnitude of the initial perturbation has significant impact on the resulting turbulence.Comment: 20 pages, 9 figure

Energy sciences supercomputing 1990

This report contains papers on the following topics: meeting the computational challenge; lattice gauge theory: probing the standard model; supercomputing for the superconducting super collider; and overview of ongoing studies in climate model diagnosis and intercomparison; MHD simulation of the fueling of a tokamak fusion reactor through the injection of compact toroids; gyrokinetic particle simulation of tokamak plasmas; analyzing chaos: a visual essay in nonlinear dynamics; supercomputing and research in theoretical chemistry; monte carlo simulations of light nuclei; parallel processing; and scientists of the future: learning by doing

Matrix-free finite-element computations at extreme scale and for challenging applications

For numerical computations based on finite element methods (FEM), it is common practice to assemble the system matrix related to the discretized system and to pass this matrix to an iterative solver. However, the assembly step can be costly and the matrix might become locally dense, e.g., in the context of high-order, high-dimensional, or strongly coupled multicomponent FEM, leading to high costs when applying the matrix due to limited bandwidth on modern CPU- and GPU-based hardware. Matrix-free algorithms are a means of accelerating FEM computations on HPC systems, by applying the effect of the system matrix without assembling it. Despite convincing arguments for matrix-free computations as a means of improving performance, their usage still tends to be an exception at the time of writing of this thesis, not least because they have not yet proven their applicability in all areas of computational science, e.g., solid mechanics. In this thesis, we further develop a state-of-the-art matrix-free framework for high-order FEM computations with focus on the preconditioning and adopt it in novel application fields. In the context of high-order FEM, we develop means of improving cache efficiency by interleaving cell loops with vector updates, which we use to increase the throughput of preconditioned conjugate gradient methods and of block smoothers based on additive Schwarz methods; we also propose an algorithm for the fast application of hanging-node constraints in 3D for up to 137 refinement configurations. We develop efficient geometric and polynomial multigrid solvers with optimized transfer operators, whose performance is experimentally investigated in detail in the context of locally refined meshes, indicating the superiority of global-coarsening algorithms. We apply the developed solvers in the context of novel stage-parallel implicit Runge–Kutta methods and demonstrate the benefit of stage–parallel solvers in decreasing the time to solution at the scaling limit. Novel challenging application fields of matrix-free computations include high-dimensional computational plasma physics, solid-state-sintering simulations with a high and dynamically changing number of strongly coupled components, and coupled multiphysics problems with evaluation and integration at arbitrary points. In the context of these fields, we detail computational challenges, propose modified versions of the standard matrix-free algorithms for high-performance computing, and discuss preconditioning-related topics. The efficiency of the derived algorithms on the node level and at extreme scales is demonstrated experimentally on SuperMUC-NG, one of Germany’s leading supercomputers, with up to 150k processes and by solving systems of up to 5 × 1012 unknowns. Such problem sizes would not be conceivable for equivalent matrix-based algorithms. The major achievements of this thesis allow to run larger simulations faster and more efficiently, enabling progress and new possibilities for a range of application fields in computational science

Multiscale numerical simulations of the magnetized plasma sheath with massively parallel electrostatic particle-in-cell code

Understanding the physics of the plasma boundary and plasma-surface interactions is one of the key scientific challenges in fusion science and engineering. Large-scale integrated simulations and high-performance computing can provide valuable insights on the dynamic phenomena involved at the interface between the plasma and the material surface. Current state-of-the-art simulations of magnetically-confined fusion devices are typically performed using gyrokinetic approximations, aimed at resolving the physics of the core plasma, scrape-of-layer, and a portion of the divertor. However, the region of plasma near to the surface, called the plasma sheath, where the plasma ions accelerate from subsonic to supersonic conditions, is typically either not handled or treated with ad-hoc approximations. The characteristic scale of the near-surface plasma (sheath and presheath) is comparable to the Debye length, which is of the order of, or smaller, than the ion gyroradius. A detailed description of the kinetic processes occurring during the supersonic acceleration across the collisional and magnetic presheaths and requires a fully-kinetic model that is not present in any current fusion code, thus limiting a detailed evaluation of the energy-angle spectrum of the ions impacting on the surface of a tokamak. We have developed and verified a new massively-parallel Particle-in-Cell code, named hPIC, solving the multi-species Boltzmann-Poisson integro-differential set of equations. We give an overview of the model equations, of the architecture of the code, and summarize the verification tests, also presenting the scalability tests performed on the Blue Waters supercomputer at the University of Illinois. The model has been used for the numerical characterization of the plasma sheath and presheath in strong magnetic fields. Thanks to the new Particle-in-Cell, we have performed a systematic analysis of the structure of the magnetized plasma sheath, in order to determine the trends of the Ion Energy-Angle Distributions (IEAD) of the particles impacting on the wall after crossing the presheath and sheath regions. The model provides the dependance of the IEAD on the level of magnetization and magnetic inclination with respect to the surface. We have found that in regimes of intermediate-to-strong magnetization, the ion flow has a characteristic three-dimensional structure, which appears in all evidence within the magnetic presheath after the ions transition from sonic to supersonic. The model also suggests the disappearance of the electrostatic (Debye) sheath at high magnetic angles, with an interesting reduction of the ion flow down to subsonic conditions. Furthermore, detailed Particle-in-Cell simulations have been compared to simplified representations of the magnetized plasma sheath based on a set of fluid equations coupled to a Monte-Carlo particle-tracer for the reconstruction of the Ion Energy-Angle Distributions (IEAD) of the particles impacting on the wall, finding qualitative agreement and suggesting strategies of model reduction which could be used in Whole-Device Modeling. Finally, the model of the magnetic and collisional presheath has been validated against three-dimensional tomographic Laser-Induced Fluorescence measurements taken at the HELIX helicon facility at WVU. Our analysis highlights the role of neutral gas pressure, background neutral flow, and ambient electric field on the structure of the collisional and magnetic presheath, finding absolute quantitative agreement between our calculated data and experimental measurements. In particular, the work gives clear evidence of the three-dimensional structure of the magnetized plasma sheath, a unique feature not present in the classical thermal sheath in unmagnetized conditions

Temporal contrast-dependent modeling of laser-driven solids - studying femtosecond-nanometer interactions and probing

Establishing precise control over the unique beam parameters of laser-accelerated ions from relativistic ultra-short pulse laser-solid interactions has been a major goal for the past 20 years. While the spatio-temporal coupling of laser-pulse and target parameters create transient phenomena at femtosecond-nanometer scales that are decisive for the acceleration performance, these scales have also largely been inaccessible to experimental observation. Computer simulations of laser-driven plasmas provide valuable insight into the physics at play. Nevertheless, predictive capabilities are still lacking due to the massive computational cost to perform these in 3D at high resolution for extended simulation times. This thesis investigates the optimal acceleration of protons from ultra-thin foils following the interaction with an ultra-short ultra-high intensity laser pulse, including realistic contrast conditions up to a picosecond before the main pulse. Advanced ionization methods implemented into the highly scalable, open-source particle-in-cell code PIConGPU enabled this study. Supporting two experimental campaigns, the new methods led to a deeper understanding of the physics of Laser-Wakefield acceleration and Colloidal Crystal melting, respectively, for they now allowed to explain experimental observations with simulated ionization- and plasma dynamics. Subsequently, explorative 3D3V simulations of enhanced laser-ion acceleration were performed on the Swiss supercomputer Piz Daint. There, the inclusion of realistic laser contrast conditions altered the intra-pulse dynamics of the acceleration process significantly. Contrary to a perfect Gaussian pulse, a better spatio-temporal overlap of the protons with the electron sheath origin allowed for full exploitation of the accelerating potential, leading to higher maximum energies. Adapting well-known analytic models allowed to match the results qualitatively and, in chosen cases, quantitatively. However, despite complex 3D plasma dynamics not being reflected within the 1D models, the upper limit of ion acceleration performance within the TNSA scenario can be predicted remarkably well. Radiation signatures obtained from synthetic diagnostics of electrons, protons, and bremsstrahlung photons show that the target state at maximum laser intensity is encoded, previewing how experiments may gain insight into this previously unobservable time frame. Furthermore, as X-ray Free Electron Laser facilities have only recently begun to allow observations at femtosecond-nanometer scales, benchmarking the physics models for solid-density plasma simulations is now in reach. Finally, this thesis presents the first start-to-end simulations of optical-pump, X-ray-probe laser-solid interactions with the photon scattering code ParaTAXIS. The associated PIC simulations guided the planning and execution of an LCLS experiment, demonstrating the first observation of solid-density plasma distribution driven by near-relativistic short-pulse laser pulses at femtosecond-nanometer resolution

Temporal contrast-dependent modeling of laser-driven solids: studying femtosecond-nanometer interactions and probing

Establishing precise control over the unique beam parameters of laser-accelerated ions from relativistic ultra-short pulse laser-solid interactions has been a major goal for the past 20 years. While the spatio-temporal coupling of laser-pulse and target parameters create transient phenomena at femtosecond-nanometer scales that are decisive for the acceleration performance, these scales have also largely been inaccessible to experimental observation. Computer simulations of laser-driven plasmas provide valuable insight into the physics at play. Nevertheless, predictive capabilities are still lacking due to the massive computational cost to perform these in 3D at high resolution for extended simulation times. This thesis investigates the optimal acceleration of protons from ultra-thin foils following the interaction with an ultra-short ultra-high intensity laser pulse, including realistic contrast conditions up to a picosecond before the main pulse. Advanced ionization methods implemented into the highly scalable, open-source particle-in-cell code PIConGPU enabled this study. Supporting two experimental campaigns, the new methods led to a deeper understanding of the physics of Laser-Wake eld acceleration and Colloidal Crystal melting, respectively, for they now allowed to explain experimental observations with simulated ionization- and plasma dynamics. Subsequently, explorative 3D3V simulations of enhanced laser-ion acceleration were performed on the Swiss supercomputer Piz Daint. There, the inclusion of realistic laser contrast conditions altered the intra-pulse dynamics of the acceleration process significantly. Contrary to a perfect Gaussian pulse, a better spatio-temporal overlap of the protons with the electron sheath origin allowed for full exploitation of the accelerating potential, leading to higher maximum energies. Adapting well-known analytic models allowed to match the results qualitatively and, in chosen cases, quantitatively. However, despite complex 3D plasma dynamics not being reflected within the 1D models, the upper limit of ion acceleration performance within the TNSA scenario can be predicted remarkably well. Radiation signatures obtained from synthetic diagnostics of electrons, protons, and bremsstrahlung photons show that the target state at maximum laser intensity is encoded, previewing how experiments may gain insight into this previously unobservable time frame. Furthermore, as X-ray Free Electron Laser facilities have only recently begun to allow observations at femtosecond-nanometer scales, benchmarking the physics models for solid-density plasma simulations is now in reach. Finally, this thesis presents the first start-to-end simulations of optical-pump, X-ray-probe laser-solid interactions with the photon scattering code ParaTAXIS. The associated PIC simulations guided the planning and execution of an LCLS experiment, demonstrating the first observation of solid-density plasma distribution driven by near-relativistic short-pulse laser pulses at femtosecond-nanometer resolution.Die Erlangung präziser Kontrolle über die einzigartigen Strahlparameter von laserbeschleunigten Ionen aus relativistischen Ultrakurzpuls-Laser-Festkörper-Wechselwirkungen ist ein wesentliches Ziel der letzten 20 Jahre. Während die räumlich-zeitliche Kopplung von Laserpuls und Targetparametern transiente Phänomene auf Femtosekunden- und Nanometerskalen erzeugt, die für den Beschleunigungsprozess entscheidend sind, waren diese Skalen der experimentellen Beobachtung bisher weitgehend unzugänglich. Computersimulationen von lasergetriebenen Plasmen liefern dabei wertvolle Einblicke in die zugrunde liegende Physik. Dennoch mangelt es noch an Vorhersagemöglichkeiten aufgrund des massiven Rechenaufwands, um Parameterstudien in 3D mit hoher Auflösung für längere Simulationszeiten durchzuführen. In dieser Arbeit wird die optimale Beschleunigung von Protonen aus ultradünnen Folien nach der Wechselwirkung mit einem ultrakurzen Ultrahochintensitäts-Laserpuls unter Einbeziehung realistischer Kontrastbedingungen bis zu einer Pikosekunde vor dem Hauptpuls untersucht. Hierbei ermöglichen neu implementierte fortschrittliche Ionisierungsmethoden für den hoch skalierbaren, quelloffenen Partikel-in-Zelle-Code PIConGPU von nun an Studien dieser Art. Bei der Unterstützung zweier Experimentalkampagnen führten diese Methoden zu einem tieferen Verständnis der Laser-Wake eld-Beschleunigung bzw. des Schmelzens kolloidaler Kristalle, da nun experimentelle Beobachtungen mit simulierter Ionisations- und Plasmadynamik erklärt werden konnten. Im Anschluss werden explorative 3D3V Simulationen verbesserter Laser-Ionen-Beschleunigung vorgestellt, die auf dem Schweizer Supercomputer Piz Daint durchgeführt wurden. Dabei veränderte die Einbeziehung realistischer Laserkontrastbedingungen die Intrapulsdynamik des Beschleunigungsprozesses signifikant. Im Gegensatz zu einem perfekten Gauß-Puls erlaubte eine bessere räumlich-zeitliche Überlappung der Protonen mit dem Ursprung der Elektronenwolke die volle Ausnutzung des Beschleunigungspotentials, was zu höheren maximalen Energien führte. Die Adaptation bekannter analytischer Modelle erlaubte es, die Ergebnisse qualitativ und in ausgewählten Fällen auch quantitativ zu bestätigen. Trotz der in den 1D-Modellen nicht abgebildeten komplexen 3D-Plasmadynamik zeigt die Vorhersage erstaunlich gut das obere Limit der erreichbaren Ionen-Energien im TNSA Szenario. Strahlungssignaturen, die aus synthethischen Diagnostiken von Elektronen, Protonen und Bremsstrahlungsphotonen gewonnen wurden, zeigen, dass der Target-Zustand bei maximaler Laserintensität einkodiert ist, was einen Ausblick darauf gibt, wie Experimente Einblicke in dieses bisher unbeobachtbare Zeitfenster gewinnen können. Mit neuen Freie-Elektronen-Röntgenlasern sind Beobachtungen auf Femtosekunden-Nanometerskalen endlich zugänglich geworden. Damit liegt ein Benchmarking der physikalischen Modelle für Plasmasimulationen bei Festkörperdichte nun in Reichweite, aber Experimente sind immer noch selten, komplex, und schwer zu interpretieren. Zuletzt werden daher in dieser Arbeit die ersten Start-zu-End-Simulationen der Pump-Probe Wechselwirkungen von optischem sowie Röntgenlaser mit Festkörpern mittels des Photonenstreu-Codes ParaTAXIS vorgestellt. Darüber hinaus dienten die zugehörigen PIC-Simulationen als Grundlage für die Planung und Durchführung eines LCLS-Experiments zur erstmaligen Beobachtung einer durch nah-relativistische Kurzpuls-Laserpulse getriebenen Festkörper-Plasma-Dichte, dessen Auflösungsbereich gleichzeitig bis auf Femtosekunden und Nanometer vordrang

GENE-3D - ein globaler gyrokinetischer Turbulenzcode für Stellaratoren und gestörte Tokamaks

This thesis describes the development and application of GENE-3D, a global gyrokinetic turbulence HPC code for stellarators. The gyrokinetic equations as well as their implementation and the use of field-aligned coordinates in non-axisymmetric geometries are discussed. GENE-3D is benchmarked for validity and performance. Different geometries of Wendelstein 7-X are investigated for their influence on turbulent properties. Also the influence of the machine size on linear growth rates is studied.Diese Arbeit beschreibt die Entwicklung und Anwendung von GENE-3D, ein globaler gyrokinetischer Turbulenzcode für Stellaratoren. Die gyrokinetischen Gleichungen sowie deren Implementierung und das am Feld ausgerichtete Koordinatensystem werden für nicht-axisymmetrische Geometrien vorgestellt. GENE-3D wird auf Korrektheit getestet.Der Einfluß unterschiedlicher Wendelstein 7-X Geometrien auf den turbulenten Transport und der Einfluß der Maschinengröße auf die linearen Anwachsraten wird untersucht