Deep learning is a promising tool to determine the physical model that describes our universe. To handle the considerable computational cost of this problem, we present CosmoFlow: a highly scalable deep learning application built on top of the TensorFlow framework. CosmoFlow uses efficient implementations of 3D convolution and pooling primitives, together with improvements in threading for many element-wise operations, to improve training performance on Intel ® Xeon Phi™ processors. We also utilize the Cray PE Machine Learning Plugin for efficient scaling to multiple nodes. We demonstrate fully synchronous data-parallel training on 8192 nodes of Cori with 77% parallel efficiency, achieving 3.5 Pflop/s sustained performance. To our knowledge, this is the first large-scale science application of the TensorFlow framework at supercomputer scale with fully-synchronous training. These enhancements enable us to process large 3D dark matter distribution and predict the cosmological parameters Ω M , σ 8 and n s with unprecedented accuracy.
1 March 2019 in Future Generation Computer Systems
We analyse gather-scatter performance bottlenecks in molecular dynamics codes and the challenges that they pose for obtaining benefits from SIMD execution. This analysis informs a number of novel code-level and algorithmic improvements to Sandia's miniMD benchmark, which we demonstrate using three SIMD widths (128-, 256and 512bit). The applicability of these optimisations to wider SIMD is discussed, and we show that the conventional approach of exposing more parallelism through redundant computation is not necessarily best. In single precision, our optimised implementation is up to 5x faster than the original scalar code running on Intel®Xeon®processors with 256-bit SIMD, and adding a single Intel®Xeon Phi™coprocessor provides up to an additional 2x performance increase. These results demonstrate: (i) the importance of effective SIMD utilisation for molecular dynamics codes on current and future hardware; and (ii) the considerable performance increase afforded by the use of Intel®Xeon Phi™coprocessors for highly parallel workloads.
1 November 2013 in Journal of Parallel and Distributed Computing
SYCL™ is a heterogeneous programming framework built on top of modern C++. Data Parallel C++, recently introduced as part of Intel's oneAPI project, is an implementation of SYCL. Data Parallel C++ (DPC++) is being developed as an open-source project on top of Clang and LLVM. It combines C++, SYCL, and new extensions to improve programmer productivity when writing highly performant code for heterogeneous architectures. This technical presentation will describe several extensions that DPC++ has proposed and implemented on top of SYCL. While many of the extensions can help to improve application performance, all of them work to improve programmer productivity by integrating easily into existing C++ applications, and by simplifying common patterns found in SYCL and C++. DPC++ is a proving ground where the value of extensions can be demonstrated before being proposed for inclusion in future versions of the SYCL specification.
14 July 2021 in Computing in Science & Engineering
The phrase “performance portability” is commonly used, but may mean different things to different people. Developing a better appreciation of the needs of different software developers and a framework for talking about these needs improves our ability to define goals, design experiments, and make forward progress. This article discusses a methodology for quantifying, summarizing, visualizing, and understanding application performance portability, and programmer productivity.