There has been considerable recent activity applying deep convolutional neural nets (CNNs) to data from particle physics experiments. Current approaches on ATLAS/CMS have largely focussed on a subset of the calorimeter, and for identifying objects or particular particle types. We explore approaches that use the entire calorimeter, combined with track information, for directly conducting physics analyses: i.e. classifying events as known-physics background or new-physics signals. We use an existing RPV-Supersymmetry analysis as a case study and explore CNNs on multi-channel, high-resolution sparse images: applied on GPU and multi-node CPU architectures (including Knights Landing (KNL) Xeon Phi nodes) on the Cori supercomputer at NERSC. We compare statistical performance of our approaches with selections on high-level physics variables from the current physics analyses, and shallow classifiers trained on those variables. We also compare time-to-solution performance of CPU (scaling to multiple KNL nodes) and GPU implementations.
In recent years, a new scientific software design pattern has emerged, pairing a Python interface with high-performance kernels in lower-level languages. The rise of general-purpose GPUs necessitates the rewriting of many such kernels, which poses challenges in GPU programming and ensures future portability and flexibility. This paper documents our experience and observations during the process of porting TOAST, a cosmology software framework designed to take full advantage of a supercomputer, to work with GPUs. This exploration led us to compare two different porting strategies: utilizing the JAX Python library and employing OpenMP Target Offload compiler directives. JAX allows kernel code to be written in pure Python, whereas OpenMP Target Offload is a directive-based strategy that integrates seamlessly with our existing OpenMP-accelerated C++ kernels. Both frameworks are high-level, abstracting system architecture details while aiming for straightforward, portable, yet performant GPU code. Through the porting of a dozen kernels, we delve into the analysis of development cost, performance, and the viability of employing either of these frameworks for complex numerical Python applications.
Foundation models use large datasets to build an effective representation of data that can be deployed on diverse downstream tasks. Previous research developed the mniearn foundation model for jet physics, using unique properties of particle physics, and showed that it could significantly advance discovery potential across collider experiments. This paper introduces a major upgrade, resulting in the mniearned framework. This framework has three new elements: (1) updates to the model architecture and training, (2) using over 1 × 10 9 jets used for training, and (3) providing well-documented software for accessing all datasets and models. We demonstrate mniearned with three representative tasks: top-quark jet tagging with the community elphes-based benchmark dataset, b tagging with ATLAS full simulation, and anomaly detection with CMS experimental data. In each case, mniearned is the state of the art, further expanding the discovery potential of past, current, and future collider experiments.
1 October 2017 in Journal of Physics: Conference Series
In recent years there has been increasing use of HPC facilities for HEP experiments. This has initially focussed on less I/O intensive workloads such as generator-level or detector simulation. We now demonstrate the efficient running of I/O-heavy analysis workloads on HPC facilities at NERSC, for the ATLAS and ALICE LHC collaborations as well as astronomical image analysis for DESI and BOSS. To do this we exploit a new 900 TB NVRAM-based storage system recently installed at NERSC, termed a Burst Buffer. This is a novel approach to HPC storage that builds ondemand filesystems on all-SSD hardware that is placed on the high-speed network of the new Cori supercomputer. We describe the hardware and software involved in this system, and give an overview of its capabilities, before focusing in detail on how the ATLAS, ALICE and astronomical workflows were adapted to work on this system. We describe these modifications and the resulting performance results, including comparisons to other filesystems. We demonstrate that we can meet the challenging I/O requirements of HEP experiments and scale to many thousands of cores accessing a single shared storage system.
26 April 2023 in Computing and Software for Big Science
Most of the fundamental, emergent, and phenomenological parameters of particle and nuclear physics are determined through parametric template fits. Simulations are used to populate histograms which are then matched to data. This approach is inherently lossy, since histograms are binned and low-dimensional. Deep learning has enabled unbinned and high-dimensional parameter estimation through neural likelihood(-ratio) estimation. We compare two approaches for neural simulation-based inference (NSBI): one based on discriminative learning (classification) and one based on generative modeling. These two approaches are directly evaluated on the same datasets, with a similar level of hyperparameter optimization in both cases. In addition to a Gaussian dataset, we study NSBI using a Higgs boson dataset from the FAIR Universe Challenge. We find that both the direct likelihood and likelihood ratio estimation are able to effectively extract parameters with reasonable uncertainties. For the numerical examples and within the set of hyperparameters studied, we found that the likelihood ratio method is more accurate and/or precise. Both methods have a significant spread from the network training and would require ensembling or other mitigation strategies in practice.