A GPU-accelerated package for simulation of flow in nanoporous source rocks with many-body dissipative particle dynamics

Mesoscopic simulations of hydrocarbon flow in source shales are challenging, in part due to the heterogeneous shale pores with sizes ranging from a few nanometers to a few micrometers. Additionally, the sub-continuum fluid-fluid and fluid-solid interactions in nano- to micro-scale shale pores, which are physically and chemically sophisticated, must be captured. To address those challenges, we present a GPU-accelerated package for simulation of flow in nano- to micro-pore networks with a many-body dissipative particle dynamics (mDPD) mesoscale model. Based on a fully distributed parallel paradigm, the code offloads all intensive workloads on GPUs. Other advancements, such as smart particle packing and no-slip boundary condition in complex pore geometries, are also implemented for the construction and the simulation of the realistic shale pores from 3D nanometer-resolution stack images. Our code is validated for accuracy and compared against the CPU counterpart for speedup. In our benchmark tests, the code delivers nearly perfect strong scaling and weak scaling (with up to 512 million particles) on up to 512 K20X GPUs on Oak Ridge National Laboratory's (ORNL) Titan supercomputer. Moreover, a single-GPU benchmark on ORNL's SummitDev and IBM's AC922 suggests that the host-to-device NVLink can boost performance over PCIe by a remarkable 40\%. Lastly, we demonstrate, through a flow simulation in realistic shale pores, that the CPU counterpart requires 840 Power9 cores to rival the performance delivered by our package with four V100 GPUs on ORNL's Summit architecture. This simulation package enables quick-turnaround and high-throughput mesoscopic numerical simulations for investigating complex flow phenomena in nano- to micro-porous rocks with realistic pore geometries

Progress in particle-based multiscale and hybrid methods for flow applications

This work focuses on the review of particle-based multiscale and hybrid methods that have surfaced in the field of fluid mechanics over the last 20 years. We consider five established particle methods: molecular dynamics, direct simulation Monte Carlo, lattice Boltzmann method, dissipative particle dynamics and smoothed-particle hydrodynamics. A general description is given on each particle method in conjunction with multiscale and hybrid applications. An analysis on the length scale separation revealed that current multiscale methods only bridge across scales which are of the order of O(102)−O(103) and that further work on complex geometries and parallel code optimisation is needed to increase the separation. Similarities between methods are highlighted and combinations discussed. Advantages, disadvantages and applications of each particle method have been tabulated as a reference

Multiscale Universal Interface: A Concurrent Framework for Coupling Heterogeneous Solvers

Concurrently coupled numerical simulations using heterogeneous solvers are powerful tools for modeling multiscale phenomena. However, major modifications to existing codes are often required to enable such simulations, posing significant difficulties in practice. In this paper we present a C++ library, i.e. the Multiscale Universal Interface (MUI), which is capable of facilitating the coupling effort for a wide range of multiscale simulations. The library adopts a header-only form with minimal external dependency and hence can be easily dropped into existing codes. A data sampler concept is introduced, combined with a hybrid dynamic/static typing mechanism, to create an easily customizable framework for solver-independent data interpretation. The library integrates MPI MPMD support and an asynchronous communication protocol to handle inter-solver information exchange irrespective of the solvers' own MPI awareness. Template metaprogramming is heavily employed to simultaneously improve runtime performance and code flexibility. We validated the library by solving three different multiscale problems, which also serve to demonstrate the flexibility of the framework in handling heterogeneous models and solvers. In the first example, a Couette flow was simulated using two concurrently coupled Smoothed Particle Hydrodynamics (SPH) simulations of different spatial resolutions. In the second example, we coupled the deterministic SPH method with the stochastic Dissipative Particle Dynamics (DPD) method to study the effect of surface grafting on the hydrodynamics properties on the surface. In the third example, we consider conjugate heat transfer between a solid domain and a fluid domain by coupling the particle-based energy-conserving DPD (eDPD) method with the Finite Element Method (FEM).Comment: The library source code is freely available under the GPLv3 license at http://www.cfm.brown.edu/repo/release/MUI

Molecular Dynamics Simulation in Arbitrary Geometries for Nanoscale Fluid Mechanics

Simulations of nanoscale systems where fluid mechanics plays an important role are required to help design and understand nano-devices and biological systems. A simulation method which hybridises molecular dynamics (MD) and continuum computational fluid dynamics (CFD) is demonstrated to be able to accurately represent the relevant physical phenomena and be computationally tractable. An MD code has been written to perform MD simulations in systems where the geometry is described by a mesh of unstructured arbitrary polyhedral cells that have been spatially decomposed into irregular portions for parallel processing. The MD code that has been developed may be used for simulations on its own, or may serve as the MD component of a hybrid method. The code has been implemented using OpenFOAM, an open source C++ CFD toolbox (www.openfoam.org). Two key enabling components are described in detail. 1) Parallel generation of initial configurations of molecules in arbitrary geometries. 2) Calculation of intermolecular pair forces, including between molecules that lie on mesh portions assigned to different, and possibly non-neighbouring processors. To calculate intermolecular forces, the spatial relationship of mesh cells is calculated once at the start of the simulation and only the molecules contained in cells that have part of their surface closer than a cut-off distance are required to interact. Interprocessor force calculations are carried out by creating local copies of molecules from other processors in a layer around the processor in question. The process of creating these copied molecules is described in detail. A case study of flow in a realistic nanoscale mixing channel, where the geometry is drawn and meshed using engineering CAD tools, is simulated to demonstrate the capabilities of the code for complex simulations