Before the emergence of commercial cloud computing, interests in parallel algorithm analysis have been mostly academic. When computing and communication resources are charged by hours, cost effective parallel processing would become a required skill. This paper reports a resource planning study using a method derived from classical program time complexity analysis, we call Timing Models. Unlike existing qualitative performance analysis methods, a Timing Model uses application instrumented capacity measures to capture the quantitative dependencies between a computer program (sequential or parallel) and its processing environments. For applications planning to use commercial clouds, this tool is ideally suited for choosing the most cost-effective configuration. The contribution of the proposed tool is its ability to explore multiple dimensions of a program quantitatively to gain non-trivial insights. This paper uses a simple matrix multiplication application to illustrate the modeling, program instrumentation and performance prediction processes. Since cloud vender do offer HPC hardware resources, we use Amazon EC2 as the target processing environments. The computing and communication models are not only useful in choosing the processing platform but also for understanding the resource usage bills. Comparisons between predicted and actual resource usages show that poor processing granularity wastes resources. Prediction errors are minimized near the optimal number of processors.
General purpose GPU (GPGPU) computing has produced the fastest running supercomputers in the world. For continued sustainable progress, GPU computing at scale also need to address two open issues: a) how increase applications mean time between failures (MTBF) as we increase supercomputer's component counts, and b) how to minimize unnecessary energy consumption. Since energy consumption is defined by the number of components used, we consider a sustainable high performance computing (HPC) application can allow better performance and reliability at the same time when adding computing or communication components. This paper reports a two-tier semantic statistical multiplexing framework for sustainable HPC at scale. The idea is to leverage the powers of statistic multiplexing to tame the nagging HPC scalability challenges. We include the theoretical model, sustainability analysis and computational experiments with automatic system level multiple CPU/GPU failure containment. Our results show that assuming three times slowdown of the statistical multiplexing layer, for an application using 1024 processors with 35% checkpoint overhead, the two-tier framework will produce sustained time and energy savings for MTBF less than 6 hours. With 5% checkpoint overhead, 1.5 hour MTBF would be the break even point. These results suggest the practical feasibility for the proposed two-tier framework.
1 November 2012 in Journal of Parallel and Distributed Computing
This article reports two techniques for building a high performance lossless service-oriented architecture (SOA). It employs a lossless ESB (enterprise service bus) and a lossless high performance transaction processing (database) cluster.
The ldquomain-streamrdquo inter-process communication models (share-memory and message-passing) require the programmers responsible for the construction of a very complex state machine for parallel processing. This has resulted multiple difficulties including programming, performance tuning, debugging, job scheduling and fault tolerance. The most troubling is the degree of difficulties. It increases exponentially as the multiprocessor grows in size. Inspired by the successes of packet switching protocols, this paper reports our preliminary findings in using decoupling technologies for parallel applications with high reliability, high performance and programmability.
Recent GPU developments have attracted much interest in the HPC community. Since each GPU interface requires a dedicated host processor, the unused high performance non-GPU processors are simply wasted. GPUs are energy intensive and are more likely to fail than CPUs, we are interested in using all processors to a) boosting application performance, and b) defending GPU failures. This paper reports parallel computation experiments using a natural semantic multiplexing substrate; we call Deeply Decoupled Parallel Processing (D2P2). The idea is to apply statistic multiplexing on application's semantic network with application-defined data tuples. Tuple space parallel processing is a natural choice for applying statistic multiplexing on application semantic networks. We report up to 53% performance gain for CPU:GPU capability ratio of 1:5. For faster GPUs, CPUs are better used to prevent application halt when GPU fails. The D2P2 substrate allows fault tolerant parallel processing using heterogeneous processors.