Supercomputing centers are seeing increasing demand for user-defined software stacks (UDSS), instead of or in addition to the stack provided by the center. These UDSS support user needs such as complex dependencies or build requirements, externally required configurations, portability, and consistency. The challenge for centers is to provide these services in a usable manner while minimizing the risks: security, support burden, missing functionality, and performance. We present Charliecloud, which uses the Linux user and mount namespaces to run industry-standard Docker containers with no privileged operations or daemons on center resources. Our simple approach avoids most security risks while maintaining access to the performance and functionality already on offer, doing so in less than 500 lines of code. Charliecloud promises to bring an industry-standard UDSS user workflow to existing, minimally altered HPC resources.
Advances in computational capabilities have transformed national security. For the US nuclear deterrent these advances have allowed wiser options for a production complex that is a fraction of its cold war-era size, enabled more effective military solutions, and have been a safeguard against consequential mistakes. An ability to faithfully simulate increasingly complex physical systems in human learning times has been the key to this progress. Experience spanning almost 80 years shows that qualitative leaps in simulation capabilities have allowed new contributions to national security. Through detailed quantitative analysis of representative simulations, we show that reliance on technologies developed for popular market applications will likely stall progress. We argue that if the United States is to continue benefiting from advances in computing, investments in deeper codesign of hardware and software addressing levels of branching and sparsity not found in Machine Learning (ML) or most other major market applications will be needed.
29 November 2023 in IEEE Transactions on Services Computing
Testing is one of the most important steps in software development–it ensures the quality of software. Continuous Integration (CI) is a widely used testing standard that can report software quality to the developer in a timely manner during development progress. Performance, especially scalability, is another key factor for High Performance Computing (HPC) applications. There are many existing profiling and performance tools for HPC applications, but none of these are integrated into CI tools. In this work, we propose BeeSwarm, an HPC container based parallel scaling performance system that can be easily applied to the current CI test environments. BeeSwarm is mainly designed for HPC application developers who need to monitor how their applications can scale on different compute resources. We demonstrate BeeSwarm using three different HPC applications: CoMD, LULESH and NWChem. We utilize GitHub Actions and provision resources from Google Compute Engine. Our results show that BeeSwarm can be used for scalability and performance testing of a variety of HPC applications, allowing developers to monitor application performance over time.