Serverless computing has grown massively in popularity over the last few years, and has provided developers with a way to deploy function-sized code units without having to take care of the actual servers or deal with logging, monitoring, and scaling of their code. High-performance computing (HPC) clusters can profit from improved serverless resource sharing capabilities compared to reservation-based systems such as Slurm. However, before running self-hosted serverless platforms in HPC becomes a viable option, serverless platforms must be able to deliver a decent level of performance. Other researchers have already pointed out that there is a distinct lack of studies in the area of comparative benchmarks on serverless platforms, especially for open-source self-hosted platforms. This study takes a step towards filling this gap by systematically benchmarking two promising self-hosted Kubernetes-based serverless platforms in comparison. While the resulting benchmarks signal potential, they demonstrate that many opportunities for performance improvements in serverless computing are being left on the table.

DECICE is a Horizon Europe project that is developing an AI-enabled open and portable management framework for automatic and adaptive optimization and deployment of applications in computing continuum encompassing from IoT sensors on the Edge to large-scale Cloud / HPC computing infrastructures. In this paper, we describe the DECICE framework and architecture. Furthermore, we highlight use-cases for framework evaluation: intelligent traffic intersection, magnetic resonance imaging, and emergency response.

Recent developments indicate a shift toward web services that employ ever larger AI models, e.g., Large Language Models (LLMs), requiring powerful hardware for inference. High-Performance Computing (HPC) systems are commonly equipped with such hardware for the purpose of large scale computation tasks. However, HPC infrastructure is inherently unsuitable for hosting real-time web services due to network, security and scheduling constraints. While various efforts exist to integrate external scheduling solutions, these often require compromises in terms of security or usability for existing HPC users. In this paper, we present SAIA, a Slurm-native platform consisting of a scheduler and a proxy. The scheduler interacts with Slurm to ensure the availability and scalability of services, while the proxy provides external access, which is secured via confined SSH commands. We have demonstrated SAIA’s applicability by deploying a large-scale LLM web service that has served over 50,000 users.

Recent developments indicate a shift toward web services that employ ever-larger AI models, e.g., large language models (LLMs), requiring powerful and expensive hardware for inference. High-performance computing (HPC) systems are commonly equipped with such hardware for the purpose of large-scale computation tasks. However, HPC infrastructure is inherently unsuitable for hosting real-time web services due to network, security, and scheduling constraints. While various efforts exist to integrate external scheduling solutions, these often require compromises in terms of security or usability for existing HPC users. In this paper, we present SAIA, a Slurm-native platform consisting of a scheduler and a proxy. The scheduler interacts with Slurm to ensure the availability and scalability of services, while the proxy provides external access, which is secured via confined SSH commands. We have demonstrated SAIA’s applicability by deploying a large-scale LLM web service that has served over 100 000 users.

With the rise of LLMs, GPU acceleration has become essential for both training and serving AI models. This requires HPC systems to be highly flexible with assigning multi-GPU nodes while also maintaining high security standards. Existing approaches involve utilizing nodes with batch and service schedulers, e.g., Slurm and Kubernetes, by dynamically moving nodes between the schedulers either through negotiation between the systems or via an external system. However, such a multi-use approach also increases the attack surface as more scheduling components operate with root permission. Moreover, it becomes increasingly difficult to recover from a security incident as attackers might have infected parts of either scheduling system. In this work, we present Ephemeral Kubernetes as a way to dynamically deploy and remove Kubernetes clusters in Warewulf managed environments such that nodes can be booted to be either part of a Slurm or Kubernetes cluster while being wiped at shutdown.

Kubernetes has emerged as the industry standard for container orchestration in cloud environments, with its scheduler dynamically placing container instances across cluster nodes based on predefined rules and algorithms. Various efforts have been made to extend and improve upon the Kubernetes scheduler. However, as the majority of Kubernetes clusters operate on homogeneous hardware, most scheduling algorithms are also only developed for homogeneous systems. Heterogeneous infrastructures, which include IoT devices or specialized hardware, have become more widespread and require specialized tuning to optimize workload assignment, for which researchers and developers working on scheduling systems require access to heterogeneous hardware for development and testing; such data may not be available. While simulations such as CloudSim or K8sSim can provide insights, the level of detail they can offer to validate new schedulers is limited, as they are only simulations. To address this, we introduce Q8S, a tool for emulating heterogeneous Kubernetes clusters including x86_64 and ARM64 architectures on OpenStack using QEMU. Emulations created through Q8S provide a higher level of detail than simulations and can be used to train machine learning scheduling algorithms. By providing an environment capable of executing real workloads, Q8S enables researchers and developers to test and refine their scheduling algorithms, ultimately leading to more efficient and effective heterogeneous cluster management. We release our implementation of Q8S as open source.