Particle-in-Cell algorithms for emerging computer architectures

AbstractWe have designed Particle-in-Cell algorithms for emerging architectures. These algorithms share a common approach, using fine-grained tiles, but different implementations depending on the architecture. On the GPU, there were two different implementations, one with atomic operations and one with no data collisions, using CUDA C and Fortran. Speedups up to about 50 compared to a single core of the Intel i7 processor have been achieved. There was also an implementation for traditional multi-core processors using OpenMP which achieved high parallel efficiency. We believe that this approach should work for other emerging designs such as Intel Phi coprocessor from the Intel MIC architecture

Multi-GPU Acceleration of the iPIC3D Implicit Particle-in-Cell Code

iPIC3D is a widely used massively parallel Particle-in-Cell code for the simulation of space plasmas. However, its current implementation does not support execution on multiple GPUs. In this paper, we describe the porting of iPIC3D particle mover to GPUs and the optimization steps to increase the performance and parallel scaling on multiple GPUs. We analyze the strong scaling of the mover on two GPU clusters and evaluate its performance and acceleration. The optimized GPU version which uses pinned memory and asynchronous data prefetching outperform their corresponding CPU versions by 5-10x on two different systems equipped with NVIDIA K80 and V100 GPUs.Comment: Accepted for publication in ICCS 201

A 2D Electromagnetic PIC Code for Distributed Memory Parallel Computers

The two dimensional electrostatic plasma particle in cell (PIC) code described an [1] has been upgraded to a 2D electromagnetic PIC code running on the Caltech/JPL Mark IIIfp and the Intel iPSC/860 parallel MIMD computers. The code solves the complete time dependent Maxwell’s equations where the plasma responses, i.e., the charge and current density in the plasma, are evaluated by advancing in time the trajectories of ~ 10^6 particles in their self-consistent electromagnetic field. The field equations are solved in Fourier space. Parallelisation is achieved through domain decomposition in real and Fourier space. Results from a simulation showing a two-dimensional Alfèn wave filamentation instability are shown; these are the first simulations of this 2D Alfèn wave decay process

Implementation of 2D Domain Decomposition in the UCAN Gyrokinetic Particle-in-Cell Code and Resulting Performance of UCAN2

The massively parallel, nonlinear, three-dimensional (3D), toroidal, electrostatic, gyrokinetic, particle-in-cell (PIC), Cartesian geometry UCAN code, with particle ions and adiabatic electrons, has been successfully exercised to identify non-diffusive transport characteristics in present day tokamak discharges. The limitation in applying UCAN to larger scale discharges is the 1D domain decomposition in the toroidal (or z-) direction for massively parallel implementation using MPI which has restricted the calculations to a few hundred ion Larmor radii or gyroradii per plasma minor radius. To exceed these sizes, we have implemented 2D domain decomposition in UCAN with the addition of the y-direction to the processor mix. This has been facilitated by use of relevant components in the P2LIB library of field and particle management routines developed for UCLA's UPIC Framework of conventional PIC codes. The gyro-averaging specific to gyrokinetic codes is simplified by the use of replicated arrays for efficient charge accumulation and force deposition. The 2D domain-decomposed UCAN2 code reproduces the original 1D domain nonlinear results within round-off. Benchmarks of UCAN2 on the Cray XC30 Edison at NERSC demonstrate ideal scaling when problem size is increased along with processor number up to the largest power of 2 available, namely 131,072 processors. These particle weak scaling benchmarks also indicate that the 1 nanosecond per particle per time step and 1 TFlops barriers are easily broken by UCAN2 with 1 billion particles or more and 2000 or more processors.This work was supported in part in the USA by Grant No. DE-FG02-04ER54741 to the University of Alaska, Fairbanks, AK, from the Office of Fusion Energy Sciences, Office of Science, United States Department of Energy. It was also supported in part at Universidad Carlos III, Madrid, Spain, by Spanish National Project No. ENE2009-12213-C03-03. This research used resources of the National Energy Research Scientific Computing Center (NERSC), which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. It also took advantage of resources at the Barcelona Supercomputing Center (BSC), Centro Nacional de Supercomputación, Barcelona, Spain. One of us (Leboeuf) would particularly like to thank David Vicente from BSC and Zhengji Zhao from NERSC for their help in the porting, debugging, and optimization of UCAN2 on the mainframes at their respective centers

Nature of Transport across Sheared Zonal Flows in Electrostatic Ion-Temperature-Gradient Gyrokinetic Plasma Turbulence

4 pages, 4 figures.-- PACS nrs.: 52.35.Ra, 05.40.Fb, 52.55.Fa, 52.65.Tt.It is shown that the usual picture for the suppression of turbulent transport across a stable sheared flow based on a reduction of diffusive transport coefficients is, by itself, incomplete. By means of toroidal gyrokinetic simulations of electrostatic, collisionless ion-temperature-gradient turbulence, it is found that the nature of the transport is altered fundamentally, changing from diffusive to anticorrelated and subdiffusive. Additionally, whenever the flows are self-consistently driven by turbulence, the transport gains an additional non-Gaussian character. These results suggest that a description of transport across sheared flows using effective diffusivities is oversimplified.Research carried out at ORNL, managed by UT-Battelle LLC, for US DOE under Contract No. DE-AC05-00OR22725. Research funded by DOE Office of Science Grants No. DE-FG02-04ER54741 at University of Alaska and No. DE-FG02-04ER54740 at UCLA.Publicad

On the nature of radial transport across sheared zonal flows in electrostatic ion-temperature-gradient gyrokinetic tokamak plasma turbulence

11 pages, 12 figures.-- PACS nrs.: 52.35.Ra, 52.55.Fa, 05.40.Fb.It is argued that the usual understanding of the suppression of radial turbulent transport across a sheared zonal flow based on a reduction in effective transport coefficients is, by itself, incomplete. By means of toroidal gyrokinetic simulations of electrostatic, ion-temperature-gradient turbulence, it is found instead that the character of the radial transport is altered fundamentally by the presence of a sheared zonal flow, changing from diffusive to anticorrelated and subdiffusive. Furthermore, if the flows are self-consistently driven by the turbulence via the Reynolds stresses (in contrast to being induced externally), radial transport becomes non-Gaussian as well. These results warrant a reevaluation of the traditional description of radial transport across sheared flows in tokamaks via effective transport coefficients, suggesting that such description is oversimplified and poorly captures the underlying dynamics, which may in turn compromise its predictive capabilities.Research was carried out at Oak Ridge National Laboratory, managed by UT-Battelle LLC, for U.S. DOE under Contract No. DE-AC05-00OR22725. Research was funded by the DOE Office of Science Grant No. DE-FG02-04ER54741 at University of Alaska and Grant No. DEFG02-04ER54740 at UCLA. Simulations run, thanks to grants for use of supercomputing resources at the University of Alaska’s Arctic Region Supercomputing Center in Fairbanks, DOE’s National Energy Research Scientific Computing Center (NERSC) in Berkeley, and the Spanish National Supercomputing Network (RES) in Barcelona and Madrid.Publicad

One-to-one full scale simulations of laser wakefield acceleration using QuickPIC

We use the quasi-static particle-in-cell code QuickPIC to perform full-scale, one-to-one LWFA numerical experiments, with parameters that closely follow current experimental conditions. The propagation of state-of-the-art laser pulses in both preformed and uniform plasma channels is examined. We show that the presence of the channel is important whenever the laser self-modulations do not dominate the propagation. We examine the acceleration of an externally injected electron beam in the wake generated by 10 J laser pulses, showing that by using ten-centimeter-scale plasma channels it is possible to accelerate electrons to more than 4 GeV. A comparison between QuickPIC and 2D OSIRIS is provided. Good qualitative agreement between the two codes is found, but the 2D full PIC simulations fail to predict the correct laser and wakefield amplitudes.Comment: 5 pages, 5 figures, accepted for publication IEEE TPS, Special Issue - Laser & Plasma Accelerators - 8/200

A new field solver for modeling of relativistic particle-laser interactions using the particle-in-cell algorithm

A customized finite-difference field solver for the particle-in-cell (PIC) algorithm that provides higher fidelity for wave-particle interactions in intense electromagnetic waves is presented. In many problems of interest, particles with relativistic energies interact with intense electromagnetic fields that have phase velocities near the speed of light. Numerical errors can arise due to (1) dispersion errors in the phase velocity of the wave, (2) the staggering in time between the electric and magnetic fields and between particle velocity and position and (3) errors in the time derivative in the momentum advance. Errors of the first two kinds are analyzed in detail. It is shown that by using field solvers with different k-space operators in Faraday’s and Ampere’s law, the dispersion errors and magnetic field time-staggering errors in the particle pusher can be simultaneously removed for electromagnetic waves moving primarily in a specific direction. The new algorithm was implemented into Osiris by using customized higher-order finite-difference operators. Schemes using the proposed solver in combination with different particle pushers are compared through PIC simulation. It is shown that the use of the new algorithm, together with an analytic particle pusher (assuming constant fields over a time step), can lead to accurate modeling of the motion of a single electron in an intense laser field with normalized vector potentials, eA/mc2, exceeding 104 for typical cell sizes and time steps.info:eu-repo/semantics/publishedVersio

Modeling of laser wakefield acceleration in Lorentz boosted frame using EM-PIC code with spectral solver

WOS:000333403900007 (Nº de Acesso Web of Science)Simulating laser wakefield acceleration (LWFA) in a Lorentz boosted frame in which the plasma drifts towards the laser with nu(b) can speed up the simulation by factors of gamma(2)(b) = (1 nu(2)(b)/c(2))(-1). In these simulations the relativistic drifting plasma inevitably induces a high frequency numerical instability that contaminates the interesting physics. Various approaches have been proposed to mitigate this instability. One approach is to solve Maxwell equations in Fourier space (a spectral solver) as this has been shown to suppress the fastest growing modes of this instability in simple test problems using a simple low pass or "ring" or "shell" like filters in Fourier space. We describe the development of a fully parallelized, multi-dimensional, particle-in-cell code that uses a spectral solver to solve Maxwell's equations and that includes the ability to launch a laser using a moving antenna. This new EM-PIC code is called UPIC-EMMA and it is based on the components of the UCLA PIC framework (UPIC). We show that by using UPIC-EMMA, LWFA simulations in the boosted frames with arbitrary yb can be conducted without the presence of the numerical instability. We also compare the results of a few LWFA cases for several values of yb, including lab frame simulations using OSIRIS, an EM-PIC code with a finite-difference time domain (FDTD) Maxwell solver. These comparisons include cases in both linear and nonlinear regimes. We also investigate some issues associated with numerical dispersion in lab and boosted frame simulations and between FDTD and spectral solvers

Modeling of laser wakefield acceleration in Lorentz boosted frame using a Quasi-3D OSIRIS algorithm

Recently it was proposed in [A. F. Lifschitz, et. al., J. Comp. Phys. 228, 1803 (2009)] that laser wakefield acceleration could be modeled efficiently using a particle-in-cell code in cylindrical coordinates if the fields and currents were expanded into Fourier modes in the azimuthal angle, ?. We have implemented this algorithm into OSIRIS, including a new rigorous charge conserving deposition routine applicable for it [A. Davidson, et. al., J. Comp. Phys. 281, 1063 (2014)]. This algorithm can be interpreted as a PIC description in r - z and a gridless description in ? in which the expansion into ? modes is truncated at a desired level. This new quasi-3D algorithm greatly reduces the computational load by describing important three-dimensional (3D) geometrical effects with nearly two-dimensional calculations. In this paper, we propose to combine this algorithm with the Lorentz boosted frame method for simulations of Laser wakefield acceleration (LWFA). We show preliminary results, including an investigation of the unstable numerical Cerenkov instability modes for this geometry, and discuss directions for future work. These preliminary results indicate that combining the quasi-3D method and the Lorentz boosted frame method together may provide unprecedented speed ups for LWFA simulations.info:eu-repo/semantics/publishedVersio