At the core of the predictive analytics applied to High Performance Computing (HPC), the most prominent tasks are the prediction of job runtimes and the prediction of job queue times, both of which have the potential for informing HPC users during their every-day decision making. Accurate runtime predictions can help users better choose so-called wallclock times at job submission, decreasing the odds of their jobs waiting in queues longer than necessary. The accurate and timely queue time predictions offered for the available partitions can inform the favorable selection of partitions for running jobs. This potential is well understood as we see in the abundance of research studies that propose solutions for these tasks, including the work published in the last several years. These tasks are seemingly receptive to the Machine Learning (ML) solutions, considering that there is no shortage of training data where HPC centers over time run millions and millions of jobs. However, we study the existing research literature, as well as look for examples in the toolchains supported on the exemplar HPC facilities, and, surprisingly, do not find any practical solutions that are ready to be adopted. We interpret this as a manifestation of the shortage of UX/UI efforts that support HPC analytics and also as a sign that the research has not come to the consensus on solving these tasks. In this study, we aim to shed new light on the long-running task of job queue time prediction by exploring the utility of runtime predictions in improving prediction accuracy and, actually, predicting these two metrics together, in tandem. In other words, we show how runtime predictions become valuable input in the queue time modeling. We challenge the existing approaches to feature engineering for the queue time prediction and describe promising results we obtained for a large dataset of HPC jobs from a supercomputer at the National Renewable Energy Laboratory.
High-Performance Computing (HPC) facilities generate vast quantities of operational data, yet turning this data into actionable insight remains hindered by fragmented, site-specific analytics pipelines. Results from one site cannot easily be compared with another, and promising methods stay siloed because there is no shared language for describing the data, the experiments, or the outcomes. We present HPC ODA Commons, an open-source, CLI-first toolkit that establishes community-governed contracts in the form of versioned schemas, canonical artifacts, and benchmark recipes for HPC Operational Data Analytics (ODA). The toolkit ingests heterogeneous site-specific data into a standardized format, evaluates predictive models through reproducible recipe-driven benchmarks, and can aggregate results into leaderboards for cross-site comparison, all without requiring data to leave the local cluster. Version 0.1 targets job runtime prediction, shipping three models: a mean-prediction baseline, an XGBoost model with rolling-window evaluation, and a TF-IDF + kNN text-similarity model. We describe the architecture, demonstrate results on two supercomputer datasets totaling 13 million jobs, and show how the toolkit’s structured, declarative design makes it well-suited to operation by AI coding agents. HPC ODA Commons is available as open-source software under a BSD-3-Clause license.