Scientific applications continue to grow and produce extremely large amounts of data, which require efficient compression algorithms for long-term storage. Compression errors in scientific applications can have a deleterious impact on downstream processing. Thus, it is crucial to preserve all the “known” Quantities of Interest (QoI) during compression. To address this issue, most existing approaches guarantee the reconstruction error of the original data or primary data (PD), but cannot directly control the problem of preserving the QoI. In this work, we propose a physics-informed compression technique that is composed of two parts: (i) reduction of the PD with bounded errors and (ii) preservation of the QoI. In the first step, we combine tensor decompositions, autoencoders, product quantizers, and error-bounded lossy compressors to bound the reconstruction error at high levels of compression. In the second step, we use constraint satisfaction post-processing followed by quantization to preserve the QoI. To illustrate the challenges of reducing the reconstruction errors of the PD and QoI, we focus on simulation data generated by a large-scale fusion code, XGC, which can produce tens of petabytes in a single day. The results show that our approach can achieve a high compression amount while accurately preserving the QoI within scientifically acceptable bounds.

Neural network-based autoencoders have been successfully used in many image and video compression applications. Unfortunately, these approaches do not provide error guarantees on an instance by instance basis, making their use limited in scientific and other applications. In this paper, we propose a new Guaranteed Autoencoder (GAE) that provides error guarantees on all instances. Our approach is based on a key observation that neural networks with piecewise linear activation units (PLUs) can be represented as instance-specific linear operators. This implies that a global model can be converted into a set of local models. The generated linear operators then enable us to find the direct and linear relationship between the compressed data and reconstructed data. We are then able to represent the residual of the reconstruction of each instance in the SVD basis of the corresponding linear operator and then augment the reconstruction until an error bound is met. Additionally, we show that the above approach based on the use of linear operators can be applied in the presence of both fully connected and convolutional layers. Thus, our approach is reasonably general and should find a large number of applications. We demonstrate our approach using both large-scale scientific data and public-domain image datasets. Experimental results show that the GAE can successfully satisfy user-provided error bounds at an instance level while achieving high compression. Additionally, the computational overhead of our approach is small.

The Data and AI Systems Section of the Computer Science and Mathematics Division at Oak Ridge National Laboratory conducts research on the data and artificial intelligence foundations of scientific discovery, shortening and automating the path from instrument and simulation to insight at the scale of leadership computing facilities. This report collects 27 project highlights from fiscal year 2026, organized by the section's five groups, namely Data Engineering, Learning Systems, Performance Engineering, Visualization, and Workflow Systems, each introduced by its group leader. Every highlight is written by the people who did the work and names someone who can be contacted about it directly. The highlights cover AI methods co-designed with the platforms that run them, from spiking and neuromorphic models to energy-efficient large language models. They also cover data reduction and campaign management that make scientific datasets AI-ready, uncertainty-aware and AI-assisted visualization at leadership scale, and workflows that couple experiment, simulation, and AI into closed loops needing little human intervention. Reported applications span fusion energy, materials science, electron microscopy, cosmology, plant phenotyping, health analytics, and national security. Much of our current agenda runs through the Department of Energy's Genesis Mission, where we contribute to the American Science Cloud and to the Transformational AI Models Consortium.

Recent electron microscopy workflows, particularly 4D Scanning Transmission Electron Microscopy (4D STEM), generate massive data volumes, sometimes exceeding tens of terabytes per dataset. This growth is outpacing improvements in storage, network, and I/O bandwidths. Although lossless compression offers a viable solution, it is often limited by modest compression ratios. Lossy compression offers an alternative; however, compression errors can propagate to downstream analyses. The goal is therefore to deploy a compression method that preserves essential data features while defining a suitable metric. In this paper, we present a moment-preserving compression workflow combining multigrid adaptive reduction (MGARD) with a constraint satisfaction technique. MGARD provides mathematically guaranteed error bounds on raw data, while a constraint satisfaction method rectifies decompressed data to strictly preserve selected statistical moments in diffraction space. This allows moment-derived observables, such as Center-of-Mass (COM) measurements, to be recovered consistently from the corrected tensor. Furthermore, we propose a frequency-domain reliability criterion in raw diffraction space to evaluate the acceptable accuracy limits. Experiments on 4D STEM datasets demonstrate that this moment-preserving adjustment substantially reduces errors in targeted downstream quantities, providing a practical path toward compressed 4D STEM data compatible with downstream microscopy analysis.

Neural networks and traditional compression techniques can reproduce inputs with high accuracy. However, downstream quantities derived from the reproduced inputs might be distorted because the relationship between reconstruction errors on raw data and errors on downstream quantities is not straightforward. Downstream quantities are crucial in many applications. For example, they play key roles in scientific simulations designed to analyze physics phenomena. These quantities of interest (QoI) are usually obtained via complex post-analysis and are nonlinear on raw data. In this paper, we present a post-processing algorithm for nonlinear constraint satisfaction that can be applied to outputs of neural networks or any other algorithms. We assume that the nonlinear QoI can be approximated by QoI neural networks that have multiple fully connected layers with piecewise linear activation units (PLUs). In this work, the last layer features of the neural networks become nonlinear QoI. Given the true features and network weights, we seek to transform the output of compressive autoencoders such that the reconstructions approximately satisfy the nonlinear QoI. To this end, we introduce a nonlinear-to-linear transformation wherein the entire QoI network is represented as instance-specific linear operators. The use of PLUs (or Leaky ReLUs) is critical to this transformation. The conversion of the entire set of weights and PLU nonlinearities into instance-specific linear operators permits us to recast nonlinear constraint satisfaction as the satisfaction of linear constraints imposed on autoencoder outputs. This transformation also comes at a cost: we have to record the “bitstream” path taken by each instance through the QoI network, specifically, the set of chosen PLU linear segments. Increasing the cardinality of PLU segments usually adversely affects the compression ratio since more bits have to be stored. The results on a standard image dataset (Fashion) and a scientific data application (XGC) clearly demonstrate the ability of the nonlinear-by-linear approach to perform nonlinear constraint satisfaction while compressing the primary data.