A high-performance implementation of a multiphase lattice Boltzmann method based on the conservative Allen-Cahn model supporting high-density ratios and high Reynolds numbers is presented. Meta-programming techniques are used to generate optimized code for CPUs and GPUs automatically. The coupled model is specified in a high-level symbolic description and optimized through automatic transformations. The memory footprint of the resulting algorithm is reduced through the fusion of compute kernels. A roofline analysis demonstrates the excellent efficiency of the generated code on a single GPU. The resulting single GPU code has been integrated into the multiphysics framework waLBerla to run massively parallel simulations on large domains. Communication hiding and GPUDirect-enabled MPI yield near-perfect scaling behavior. Scaling experiments are conducted on the Piz Daint supercomputer with up to 2048 GPUs, simulating several hundred fully resolved bubbles. Further, validation of the implementation is shown in a physically relevant scenario—a three-dimensional rising air bubble in water.
9 November 2022 in Journal of Computational Physics
The scientific code generation package lbmpy supports the automated design and the efficient implementation of lattice Boltzmann method (LBM) through metaprogramming. It is based on a new, concise calculus for describing multiple relaxation-time LBMs, including techniques that enable the numerically advantageous subtraction of the constant background component from the populations. These techniques are generalized to a wide range of collision spaces and equilibrium distributions. The article contains an overview of lbmpy’s frontend and its code generation pipeline, which implements the new LBM calculus by means of symbolic formula manipulation tools and object-oriented programming. The generated codes have only a minimal number of arithmetic operations. Their automatic derivation rests on two novel chimera transforms that have been specifically developed for efficiently computing raw and central moments. Information contained in the symbolic representation of the methods is further exploited in a customized sequence of algebraic simplifications, further reducing computational cost. When combined, these algebraic transformations lead to concise and compact numerical kernels. Specifically, with these optimizations, the advanced central moment- and cumulant-based methods can be realized with only little additional cost as when compared with the simple Bhatnagar–Gross–Krook method. The effectiveness and flexibility of the new lbmpy code generation system is demonstrated in simulating Taylor–Green vortex decay and the automatic derivation of an LBM algorithm to solve the shallow water equations.
This study parametrically assessed the stability of the phase-field lattice Boltzmann model (PFLBM) before applying it to analyze the effect of annular piping geometry on the flow of Taylor bubbles. The impacts of both eccentricity and pipe diameter ratio were examined, providing an insight into the behavior of these bubbles as well as the applicability and shortcomings in current prediction methodologies. A recently developed PFLBM was implemented into the open-source simulation framework, waLBerla, for this analysis. The stability properties of the code were investigated in detail by assessing various lattice discretizations and relaxation kernels applied to the Rayleigh–Taylor benchmark problem and a Rayleigh–Taylor instability in a tubular geometry, with gravitational Reynolds numbers of up to 30 000 and 10 000, respectively. This paper makes three contributions relating to the stability and usage of the PFLBM as well as the flow of Taylor bubbles in annular pipes. First, the work numerically explored the stability properties of the velocity-based, PFLBM and concluded the impact of various collision models and lattice discretizations on simulation results. Second, it provided a flexible open-source code that the interested researcher can use interactively for practical flow problems as well as the analysis of numerical properties of various lattice Boltzmann algorithms. Finally, it quantified the effect of pipe eccentricity and diameter ratio on the propagation of a Taylor bubble inside a water-filled annular pipe, concluding that a previously defined closure model captured the diameter ratio for the cases examined. To extend this work, future studies aim to analytically investigate the stability properties parametrically observed in this study and apply the findings to simulate the interaction of multiple Taylor bubbles.
The primary goal of the EuroHPC JU project SCALABLE is to develop an industrial Lattice Boltzmann Method (LBM)-based computational fluid dynamics (CFD) solver capable of exploiting current and future extreme scale architectures, expanding current capabilities of existing industrial LBM solvers by at least two orders of magnitude in terms of processor cores and lattice cells, while preserving its accessibility from both the end-user and software developer's point of view. This is accomplished by transferring technology and knowledge between an academic code (waLBerla) and an industrial code (LaBS). This paper briefly introduces the characteristics and main features of both software packages involved in the process. We also highlight some of the performance achievements in scales of up to tens of thousand of cores presented on one academic and one industrial benchmark case.