The key common bottleneck in most stencil codes is data movement, and prior research has shown that improving data locality through optimisations that optimise across loops do particularly well. However, in many large PDE applications it is not possible to apply such optimisations through compilers because there are many options, execution paths and data per grid point, many dependent on run-time parameters, and the code is distributed across different compilation units. In this paper, we adapt the data locality improving optimisation called tiling for use in large OPS applications both in shared-memory and distributed-memory systems, relying on run-time analysis and delayed execution. We evaluate our approach on a number of applications, observing speedups of 2x on the Cloverleaf 2D/3D proxy applications, which contain 83(2D)/141(3D) loops, 3.5x on the linear solver TeaLeaf, and 1.7x on the compressible Navier-Stokes solver OpenSBLI. We demonstrate strong and weak scalability on up to 4608 cores of CINECA's Marconi supercomputer. We also evaluate our algorithms on Intel's Knights Landing, demonstrating maintained throughput as the problem size grows beyond 16GB, and we do scaling studies up to 8704 cores. The approach is generally applicable to any stencil DSL that provides per loopnest data access information.
8 July 2015 in IEEE Transactions on Parallel and Distributed Systems
Hydra is a full-scale industrial CFD application used for the design of turbomachinery at Rolls Royce plc., capable of performing complex simulations over highly detailed unstructured mesh geometries. Hydra presents major challenges in data organization and movement that need to be overcome for continued high performance on emerging platforms. We present research in achieving this goal through the OP2 domain-specific high-level framework, demonstrating the viability of such a high-level programming approach. OP2 targets the domain of unstructured mesh problems and enables execution on a range of back-end hardware platforms. We chart the conversion of Hydra to OP2, and map out the key difficulties encountered in the process. Specifically we show how different parallel implementations can be achieved with an active library framework, even for a highly complicated industrial application and how different optimizations targeting contrasting parallel architectures can be applied to the whole application, seamlessly, reducing developer effort and increasing code longevity. Performance results demonstrate that not only the same runtime performance as that of the hand-tuned original code could be achieved, but it can be significantly improved on conventional processor systems, and many-core systems. Our results provide evidence of how high-level frameworks such as OP2 enable portability across a wide range of contrasting platforms and their significant utility in achieving high performance without the intervention of the application programmer.
Pandemic management requires reliable and efficient dynamical simulation to predict and control disease spreading. The COVID-19 (SARS-CoV-2) pandemic is mitigated by several non-pharmaceutical interventions, but it is hard to predict which of these are the most effective for a given population. We developed the computationally effective and scalable, agent-based microsimulation frameworkPanSim, allowing us to test control measures in multiple infection waves caused by the spread of a new virus variant in a city-sized societal environment using a unified framework fitted to realistic data. We show that vaccination strategies prioritising occupational risk groups minimise the number of infections but allow higher mortality while prioritising vulnerable groups minimises mortality but implies an increased infection rate. We also found that intensive vaccination along with non-pharmaceutical interventions can substantially suppress the spread of the virus, while low levels of vaccination, premature reopening may easily revert the epidemic to an uncontrolled state. Our analysis highlights that while vaccination protects the elderly from COVID-19, a large percentage of children will contract the virus, and we also show the benefits and limitations of various quarantine and testing scenarios. The uniquely detailed spatio-temporal resolution ofPanSimallows the design and testing of complex, specifically targeted interventions with a large number of agents under dynamically changing conditions.
19 November 2018 in Geoscientific Model Development
In this paper, we present the VOLNA-OP2 tsunami model and implementation; a finite-volume non-linear shallow-water equation (NSWE) solver built on the OP2 domain-specific language (DSL) for unstructured mesh computations. VOLNA-OP2 is unique among tsunami solvers in its support for several high-performance computing platforms: central processing units (CPUs), the Intel Xeon Phi, and graphics processing units (GPUs). This is achieved in a way that the scientific code is kept separate from various parallel implementations, enabling easy maintainability. It has already been used in production for several years; here we discuss how it can be integrated into various workflows, such as a statistical emulator. The scalability of the code is demonstrated on three supercomputers, built with classical Xeon CPUs, the Intel Xeon Phi, and NVIDIA P100 GPUs. VOLNA-OP2 shows an ability to deliver productivity as well as performance and portability to its users across a number of platforms.
Wastewater-based epidemiology (WBE) has emerged as a promising approach for monitoring the COVID-19 pandemic, since the measurement process is cost-effective and is exposed to fewer potential errors compared to other indicators like hospitalization data or the number of detected cases. Consequently, WBE was gradually becoming a key tool for epidemic surveillance and often the most reliable data source, as the intensity of clinical testing for COVID-19 drastically decreased by the third year of the pandemic. Recent results suggests that the model-based fusion of wastewater measurements with clinical data and other indicators is essential in future epidemic surveillance. In this work, we developed a wastewater-based compartmental epidemic model with a two-phase vaccination dynamics and immune evasion. We proposed a multi-step optimization-based data assimilation method for epidemic state reconstruction, parameter estimation, and prediction. The computations make use of the measured viral load in wastewater, the available clinical data (hospital occupancy, delivered vaccine doses, and deaths), the stringency index of the official social distancing rules, and other measures. The current state assessment and the estimation of the current transmission rate and immunity loss allow a plausible prediction of the future progression of the pandemic. Qualitative and quantitative evaluations revealed that the contribution of wastewater data in our computational epidemiological framework makes predictions more reliable. Predictions suggest that at least half of the Hungarian population has lost immunity during the epidemic outbreak caused by the BA.1 and BA.2 subvariants of Omicron in the first half of 2022. We obtained a similar result for the outbreaks caused by the subvariant BA.5 in the second half of 2022. The proposed approach has been used to support COVID management in Hungary and could be customized for other countries as well.