The continual expansion of high-performance computing (HPC) brings with it an increasing need for efficiency. Heavy investment in energy, hardware, and software infrastructure to support peta- and exascale computing requires the optimization of existing systems and, wherever possible, the discernment and adoption of best-practices towards these goals. Such is the case for runtime prediction. When a job is submitted to an HPC system, an estimate of its runtime is provided by the user in the form of “requested wallclock”. Error in this user-provided estimate can lead to jobs being prematurely killed by the scheduler, increased wait time on the queue, and decreased system utilization. More than fifteen years of research has been directed at mitigating these effects by using data-driven runtime predictions. Codified here is a set of commonalities and insights emerging from this body of work, which we present as recommendations and best practices. These practices are combined into a methodological approach described and evaluated on an 11-million-job dataset from the National Renewable Energy Laboratory’s petascale HPC system, Eagle. This dataset and the accompanying codebase have been released to the public domain for the benefit of the wider HPC research community.
At the core of the predictive analytics applied to High Performance Computing (HPC), the most prominent tasks are the prediction of job runtimes and the prediction of job queue times, both of which have the potential for informing HPC users during their every-day decision making. Accurate runtime predictions can help users better choose so-called wallclock times at job submission, decreasing the odds of their jobs waiting in queues longer than necessary. The accurate and timely queue time predictions offered for the available partitions can inform the favorable selection of partitions for running jobs. This potential is well understood as we see in the abundance of research studies that propose solutions for these tasks, including the work published in the last several years. These tasks are seemingly receptive to the Machine Learning (ML) solutions, considering that there is no shortage of training data where HPC centers over time run millions and millions of jobs. However, we study the existing research literature, as well as look for examples in the toolchains supported on the exemplar HPC facilities, and, surprisingly, do not find any practical solutions that are ready to be adopted. We interpret this as a manifestation of the shortage of UX/UI efforts that support HPC analytics and also as a sign that the research has not come to the consensus on solving these tasks. In this study, we aim to shed new light on the long-running task of job queue time prediction by exploring the utility of runtime predictions in improving prediction accuracy and, actually, predicting these two metrics together, in tandem. In other words, we show how runtime predictions become valuable input in the queue time modeling. We challenge the existing approaches to feature engineering for the queue time prediction and describe promising results we obtained for a large dataset of HPC jobs from a supercomputer at the National Renewable Energy Laboratory.
14 September 2023 in IEEE Computer Graphics and Applications
Uncertainty is an important aspect to data understanding. Without awareness of the variability, error, or reliability of a dataset, the ability to make decisions on that data is limited. However, practices around uncertainty visualization remain domain-specific, rooted in convention, and in many instances, absent entirely. Part of the reason for this may be a lack of established guidelines for navigating difficult choices of when uncertainty should be added, how to visualize uncertainty, and how to evaluate its effectiveness. Unsurprisingly, the inclusion of uncertainty into visualizations is a major challenge to visualization. As work concerned with uncertainty visualization grows, it has become clear that simple visual additions of uncertainty information to traditional visualization methods do not appropriately convey the meaning of the uncertainty, pose many perceptual challenges, and, in the worst case, can lead a viewer to a completely wrong understanding of the data. These challenges are the driving motivator for this special issue.
Mature bacterial biofilms have elaborate three-dimensional architectures that endow these structures with their durability and resistance to environmental perturbations. We used agent-based modeling to explore whether local cellular interactions were sufficient to give rise to global structural features of biofilms. Specifically, we asked whether chemorepulsion from a self-produced quorum-sensing molecule, autoinducer-2 (AI-2), was sufficient to recapitulate biofilm growth and cellular organization observed for biofilms of the human pathogen Helicobacter pylori . To carry out this modeling, we modified an existing platform, Individual-based Dynamics of Microbial Communities Simulator (iDynoMiCS), to incorporate three-dimensional chemotaxis, planktonic cells that could join or leave the biofilm structure, and cellular production of AI-2. We simulated biofilm growth of previously characterized H. pylori strains with varying AI-2 production and sensing capacities. Using biologically plausible parameters, we were able to recapitulate both the variation in biofilm mass and cellular distributions observed with these strains. Specifically, the strains that were competent to chemotax away from AI-2 produced smaller and more heterogeneously spaced biofilms, whereas the AI-2 chemotaxis defective strains produced larger and more homogeneously spaced biofilms. The model also provided new insights into the cellular demographics contributing to the biofilm patterning of each strain. Our analysis supports the idea that cellular interactions at small spatial and temporal scales are sufficient to give rise to larger scale emergent properties of biofilms. Importance: Most bacteria exist in aggregated, three-dimensional structures called biofilms. Biofilms are resistant to antimicrobials and can pose societal problems, for example when they grow in plumbing systems or on medical implants. Understanding the processes that promote the growth and disassembly of biofilms could lead to better strategies to manage these structures. We had previously shown that Helicobacter pylori bacteria are repulsed by high concentrations of a self-produced molecule, autoinducer-2 (AI-2) and that H. pylori mutants deficient in AI-2 sensing form larger and more homogeneously spaced biofilms. Here we used computer simulations of biofilm formation to show that local H. pylori behavior of repulsion from high AI-2 could explain the overall architecture of H. pylori biofilms. Our findings demonstrate that it is possible to change global biofilm organization by manipulating local cell behaviors, which suggests that simple strategies targeting cells at local scales could be useful for controlling biofilms in industrial and medical settings.
Wind plants operate in stochastic environments characterized by complex turbulent flow dynamics and high-dimensional random variables. A key step in uncertainty quantification studies is sensitivity analysis and dimension reduction that can facilitate the development of surrogate models to be used for forward and inverse propagation or optimization under uncertainty. Prior work has shown active subspaces are an effective tool for identifying important directions in the space of stochastic inputs; however, they have only been applied to single-fidelity wind plant models. In this study, we investigate the efficacy of a multi-fidelity active subspace method for analyzing the uncertainty in wind plant power output. The multi-fidelity active subspace estimator offers the promise of increased accuracy in identifying active subspaces as compared to a single-fidelity estimator for the same computational cost, or a reduction in cost for the same accuracy. This makes the study of uncertainty in larger wind plants and with higher fidelity physics tractable. The multi-fidelity active subspace method is applied to gridded and existing wind plant layouts with single and multiple inflow conditions and its performance for surrogate modeling and uncertainty propagation is compared against a single-fidelity active subspace method. This multi-fidelity approach yields substantial computational speedups of 2x - 3.4x across the test cases along with acceptable accuracy in surrogate modeling and computing statistical moments.