Towards aeraulic simulations at urban scale using the lattice Boltzmann method

International audienceThe lattice Boltzmann method (LBM) is an innovative approach in computational fluid dynamics (CFD). Due to the underlying lattice structure, the LBM is inherently parallel and therefore well suited for high performance computing. Its application to outdoor aeraulic studies is promising, e.g. applied on complex urban configurations, as an alternative approach to the commonplace Reynolds-averaged Navier-Stokes and large eddy simulation methods based on the Navier-Stokes equations. Emerging many-core devices, such as graphic processing units (GPUs), nowadays make possible to run very large scale simulations on rather inexpensive hardware. In this paper, we present simulation results obtained using our multi-GPU LBM solver. For validation purpose, we study the flow around a wall-mounted cube and show agreement with previously published experimental results. Furthermore, we discuss larger scale flow simulations involving nine cubes which demonstrate the practicability of CFD simulations in building external aeraulics

Development and performance of a HemeLB GPU code for human-scale blood flow simulation

In recent years, it has become increasingly common for high performance computers (HPC) to possess some level of heterogeneous architecture - typically in the form of GPU accelerators. In some machines these are isolated within a dedicated partition, whilst in others they are integral to all compute nodes - often with multiple GPUs per node - and provide the majority of a machine's compute performance. In light of this trend, it is becoming essential that codes deployed on HPC are updated to execute on accelerator hardware. In this paper we introduce a GPU implementation of the 3D blood flow simulation code HemeLB that has been developed using CUDA C++. We demonstrate how taking advantage of NVIDIA GPU hardware can achieve significant performance improvements compared to the equivalent CPU only code on which it has been built whilst retaining the excellent strong scaling characteristics that have been repeatedly demonstrated by the CPU version. With HPC positioned on the brink of the exascale era, we use HemeLB as a motivation to provide a discussion on some of the challenges that many users will face when deploying their own applications on upcoming exascale machines

Transport in complex systems : a lattice Boltzmann approach

Celem niniejszej pracy jest zbadanie możliwości efektywnego modelowania procesów transportu w złożonych systemach z zakresu dynamiki płynów za pomocą metody siatkowej Boltzmanna (LBM). Złożoność systemu została potraktowana wieloaspektowo i konkretne układy, które poddano analizie pokrywały szeroki zakres zagadnień fizycznych, m.in. przepływy wielofazowe, hemodynamikę oraz turbulencje. We wszystkich przypadkach szczególna uwaga została zwrócona na aspekty numeryczne — dokładność używanych modeli, jak również szybkość z jaką pozwalają one uzyskać zadowalające rozwiązanie. W ramach pracy rozwinięty został pakiet oprogramowania Sailfish, będący otwarta implementacja metody siatkowej Boltzmanna na procesory kart graficznych (GPU). Po analizie szybkości jego działania, walidacji oraz omówieniu założeń projektowych, pakiet ten został użyty do symulacji trzech typów przepływów. Pierwszym z nich były przepływy typu Brethertona/Taylora w dwu- i trójwymiarowych geometriach, do symulacji których zastosowano model energii swobodnej. Analiza otrzymanych wyników pokazała dobra zgodność z danymi dostępnymi w literaturze, zarówno eksperymentalnymi, jak i otrzymanymi za pomocą innych metod numerycznych. Drugim badanym problemem były przepływy krwi w realistycznych geometriach tętnic dostarczających krew do ludzkiego mózgu. Wyniki symulacji zostały dokładnie porównane z rozwiązaniem otrzymanym metoda objętości skończonych z wykorzystaniem pakietu OpenFOAM, przyspieszonego komercyjna biblioteka pozwalająca na wykonywanie obliczeń na GPU. Otrzymano dobra zgodność między badanymi metodami oraz pokazano, że metoda siatkowa Boltzmanna pozwala na wykonywanie symulacji do ok. 20 razy szybciej. Trzecim przeanalizowanym zagadnieniem były turbulentne przepływy w prostych geometriach. Po zwalidowaniu wszystkich zaimplementowanych modeli relaksacji na przypadku wiru Kidy, zbadano przepływy w pustym kanale oraz w obecności przeszkód. Do symulacji wykorzystano zarówno siatki zapewniające pełną rozdzielczość aż do skal Kolmogorova, jak i siatki o mniejszej rozdzielczości. Również w tym kontekście pokazano dobrą zgodność wyników otrzymanych metodą siatkową Boltzmanna z wynikami innych symulacji oraz badaniami eksperymentalnymi. Pokazano również, że implementacja LBM w pakiecie Sailfish zapewnia większą stabilność obliczeń niż ta opisana w literaturze dla tych samych przepływów i modeli relaksacji

A Simulation Suite for Lattice-Boltzmann based Real-Time CFD Applications Exploiting Multi-Level Parallelism on Modern Multi- and Many-Core Architectures

We present a software approach to hardware-oriented numerics which builds upon an augmented, previously published open-source set of libraries facilitating portable code development and optimisation on a wide range of modern computer architectures. In order to maximise eficiency, we exploit all levels of arallelism, including vectorisation within CPU cores, the Cell BE and GPUs, shared memory thread-level parallelism between cores, and parallelism between heterogeneous distributed memory resources in clusters. To evaluate and validate our approach, we implement a collection of modular building blocks for the easy and fast assembly and development of CFD applications based on the shallow water equations: We combine the Lattice-Boltzmann method with i-uid-structure interaction techniques in order to achieve real-time simulations targeting interactive virtual environments. Our results demonstrate that recent multi-core CPUs outperform the Cell BE, while GPUs are significantly faster than conventional multi-threaded SSE code. In addition, we verify good scalability properties of our application on small clusters

The TheLMA project: Multi-GPU Implementation of the Lattice Boltzmann Method

International audienceIn this paper, we describe the implementation of a multi-graphical processing unit (GPU) fluid flow solver based on the lattice Boltzmann method (LBM). The LBM is a novel approach in computational fluid dynamics, with numerous interesting features from a computational, numerical, and physical standpoint. Our program is based on CUDA and uses POSIX threads to manage multiple computation devices. Using recently released hardware, our solver may therefore run eight GPUs in parallel, which allows us to perform simulations at a rather large scale. Performance and scalability are excellent, the speedup over sequential implementations being at least of two orders of magnitude. In addition, we discuss tiling and communication issues for present and forthcoming implementations

A Simulation Suite for Lattice-Boltzmann based Real-Time CFD Applications Exploiting Multi-Level Parallelism on Modern Multi- and Many-Core Architectures

We present a software approach to hardware-oriented numerics which builds upon an augmented, previously published open-source set of libraries facilitating portable code development and optimisation on a wide range of modern computer architectures. In order to maximise eficiency, we exploit all levels of arallelism, including vectorisation within CPU cores, the Cell BE and GPUs, shared memory thread-level parallelism between cores, and parallelism between heterogeneous distributed memory resources in clusters. To evaluate and validate our approach, we implement a collection of modular building blocks for the easy and fast assembly and development of CFD applications based on the shallow water equations: We combine the Lattice-Boltzmann method with i-uid-structure interaction techniques in order to achieve real-time simulations targeting interactive virtual environments. Our results demonstrate that recent multi-core CPUs outperform the Cell BE, while GPUs are significantly faster than conventional multi-threaded SSE code. In addition, we verify good scalability properties of our application on small clusters

Accuracy and performance of the lattice Boltzmann method with 64-bit, 32-bit, and customized 16-bit number formats

Fluid dynamics simulations with the lattice Boltzmann method (LBM) are very memory-intensive. Alongside reduction in memory footprint, significant performance benefits can be achieved by using FP32 (single) precision compared to FP64 (double) precision, especially on GPUs. Here, we evaluate the possibility to use even FP16 and Posit16 (half) precision for storing fluid populations, while still carrying arithmetic operations in FP32. For this, we first show that the commonly occurring number range in the LBM is a lot smaller than the FP16 number range. Based on this observation, we develop novel 16-bit formats - based on a modified IEEE-754 and on a modified Posit standard - that are specifically tailored to the needs of the LBM. We then carry out an in-depth characterization of LBM accuracy for six different test systems with increasing complexity: Poiseuille flow, Taylor-Green vortices, Karman vortex streets, lid-driven cavity, a microcapsule in shear flow (utilizing the immersed-boundary method) and finally the impact of a raindrop (based on a Volume-of-Fluid approach). We find that the difference in accuracy between FP64 and FP32 is negligible in almost all cases, and that for a large number of cases even 16-bit is sufficient. Finally, we provide a detailed performance analysis of all precision levels on a large number of hardware microarchitectures and show that significant speedup is achieved with mixed FP32/16-bit.Comment: 30 pages, 20 figures, 4 tables, 2 code listing