A portable OpenCL-based unstructured edge-based Finite Element Navier-Stokes solver on graphics hardware

The rise of GPUs in modern high-performance systems increases the interest in porting portion of codes to such hardware. The current paper aims to explore the performance of a portable state-of-the-art FE solver on GPU accelerators. Performance evaluation is done by comparing with an existing highly-optimized OpenMP version of the solver. Code portability is ensured by writing the program using the OpenCL 1.1 specifications, while performance portability is sought through an optimization step performed at the beginning of the calculations to find out the optimal parameter set for the solver. The results show that the new implementation can be several times faster than the OpenMP version.Preprin

Research in Applied Mathematics, Fluid Mechanics and Computer Science

This report summarizes research conducted at the Institute for Computer Applications in Science and Engineering in applied mathematics, fluid mechanics, and computer science during the period October 1, 1998 through March 31, 1999

Methods for Multilevel Parallelism on GPU Clusters: Application to a Multigrid Accelerated Navier-Stokes Solver

Computational Fluid Dynamics (CFD) is an important field in high performance computing with numerous applications. Solving problems in thermal and fluid sciences demands enormous computing resources and has been one of the primary applications used on supercomputers and large clusters. Modern graphics processing units (GPUs) with many-core architectures have emerged as general-purpose parallel computing platforms that can accelerate simulation science applications substantially. While significant speedups have been obtained with single and multiple GPUs on a single workstation, large problems require more resources. Conventional clusters of central processing units (CPUs) are now being augmented with GPUs in each compute-node to tackle large problems. The present research investigates methods of taking advantage of the multilevel parallelism in multi-node, multi-GPU systems to develop scalable simulation science software. The primary application the research develops is a cluster-ready GPU-accelerated Navier-Stokes incompressible flow solver that includes advanced numerical methods, including a geometric multigrid pressure Poisson solver. The research investigates multiple implementations to explore computation / communication overlapping methods. The research explores methods for coarse-grain parallelism, including POSIX threads, MPI, and a hybrid OpenMP-MPI model. The application includes a number of usability features, including periodic VTK (Visualization Toolkit) output, a run-time configuration file, and flexible setup of obstacles to represent urban areas and complex terrain. Numerical features include a variety of time-stepping methods, buoyancy-drivenflow, adaptive time-stepping, various iterative pressure solvers, and a new parallel 3D geometric multigrid solver. At each step, the project examines performance and scalability measures using the Lincoln Tesla cluster at the National Center for Supercomputing Applications (NCSA) and the Longhorn cluster at the Texas Advanced Computing Center (TACC). The results demonstrate that multi-GPU clusters can substantially accelerate computational fluid dynamics simulations

Numerical Study of Flow and Heat Transfer in Rotating Microchannels

Investigation of fluid flow and heat transfer in rotating microchannels is important for centrifugal microfluidics, which has emerged as an advanced technique in biomedical applications and chemical separations. The centrifugal force and the Coriolis force, arising as a consequence of the microchannel rotation, change the flow pattern significantly from the symmetric profile of a non-rotating channel. A successful design of a centrifugal microfluidic device depends on effectively regulating these forces in rotating microchannels. Although a large number of experimental studies have been performed in order to demonstrate the applications of centrifugal microfluidics in various fields, a systematic study on the effect of rotation, channel aspect ratio, and wall boundary conditions on the fluid flow and heat transfer phenomena in rotating microchannels has not yet been conducted. During the present study, pressure-based finite volume solvers in both staggered and collocated grids were developed to solve steady and unsteady, incompressible Navier-Stokes equations. The serial solver in collocated grid was parallelized using a Message Passing Interface (MPI) library. In order to accelerate the convergence of the collocated finite volume solver, a non-linear multi-grid method was developed. The parallel performances of the single and multi-grid solvers were tested on a two-dimensional lid driven cavity flow. High fidelity benchmark solution to a lid driven cavity flow problem in a 1024 x 1024 grid was presented for a range of Reynolds numbers. Parallel multigrid speedup as high as three orders of magnitude was achieved for low Reynolds number flows. In addition, the optimal multigrid efficiency was validated. The fluid flow in a rotating microchannel was modeled as a steady, laminar in compressible flow with no slip and slip boundary conditions. For no slip boundary condition, critical values of parameters that determine the extent of the centrifugal force and the Coriolis force were identified. The critical aspect ratio (=width/height) that causes the optimal mixing of two liquids was found to be 1.0. For liquid slip boundary condition, the effect of rotation on liquid slip flow in rotating microchannels with hydrophobic and superhydrophobic surfaces was studied. New correlations for friction relation (fRe) as a function of slip length (λ) and rotational Reynolds number (Reω) were proposed. It was also found that, the liquid slip can increase or decrease the heat transfer depending on the secondary flow effect and the aspect ratio of the microchannel. The microscale effects, such as surface tension and contact angle boundary condition, were included in the modeled problem. A level set method was applied to incorporate these microscale effects, which will enable us to investigate the unsteady nature of the liquid meniscus during two-phase flow simulations

Chaotic multigrid methods for the solution of elliptic equations

Supercomputer power has been doubling approximately every 14 months for several decades, increasing the capabilities of scientific modelling at a similar rate. However, to utilize these machines effectively for applications such as computational fluid dynamics, improvements to strong scalability are required. Here, the particular focus is on semi-implicit, viscous-flow CFD, where the largest bottleneck to strong scalability is the parallel solution of the linear pressure-correction equation — an elliptic Poisson equation. State-of-the-art linear solvers, such as Krylov subspace or multigrid methods, provide excellent numerical performance for elliptic equations, but do not scale efficiently due to frequent synchronization between processes. Complete desynchronization is possible for basic, Jacobi-like solvers using the theory of ‘chaotic relaxations’. These non-deterministic, chaotic solvers scale superbly, as demonstrated herein, but lack the numerical performance to converge elliptic equations — even with the relatively lax convergence requirements of the example CFD application. However, these chaotic principles can also be applied to multigrid solvers. In this paper, a ‘chaotic-cycle’ algebraic multigrid method is described and implemented as an open-source library. It is tested on a model Poisson equation, and also within the context of CFD. Two CFD test cases are used: the canonical lid-driven cavity flow and the flow simulation of a ship (KVLCC2). The chaotic-cycle multigrid shows good scalability and numerical performance compared to classical V-, W- and F-cycles. On 2048 cores the chaotic-cycle multigrid solver performs up to faster than Flexible-GMRES and faster than classical V-cycle multigrid. Further improvements to chaotic-cycle multigrid can be made, relating to coarse-grid communications and desynchronized residual computations. It is expected that the chaotic-cycle multigrid could be applied to other scientific fields, wherever a scalable elliptic-equation solver is required