3 January 2020 in IEEE Access
20 January 2021 in IEEE Access
Dedicated infrastructures are commonly used for urgent computations. However, using dedicated resources is not always affordable due to budget constraints. As a result, utilizing shared infrastructures becomes an alternative solution for urgent computations. Since the infrastructures are meant to serve many users, the urgent jobs may arrive when regular jobs are using the necessary resources. In such a case, it is necessary to preempt the regular jobs so that urgent jobs can be executed immediately. Most conventional methods for job scheduling have focused on reducing the response times and waiting times of all jobs. However, these methods can delay urgent jobs and hinder them from being completed within a stipulated deadline. Furthermore, in heterogeneous systems with coprocessors, preemption becomes more difficult because coprocessors rely on several system software functionalities provided by the host processor. In this paper, we propose a parallel job scheduling method to effectively use shared heterogeneous systems for urgent computations. Our method employs an in-memory process swapping mechanism to preempt jobs running on the coprocessor devices. The results of our simulations show that our method can achieve a significant reduction in the response time and slowdown of regular jobs without substantial delays of urgent jobs.
13 March 2025 in Genetics
Genome-wide association studies (GWAS) are computationally intensive, requiring significant time and resources with computational complexity scaling at least linearly with sample size. Here, we present an accurate and resource-efficient pipeline for GWAS that mitigates the impact of sample size on computational demands. Our approach involves (1) randomly partitioning the cohort into equally sized sub-cohorts, (2) conducting independent GWAS within each sub-cohort, and (3) integrating the results using a novel meta-analysis technique that accounts for population structure and other confounders between sub-cohorts. Importantly, we demonstrate through simulations and real-data examples in humans that our approach effectively manages analyzing related individuals, a critical factor in real datasets, while controlling for inflated effect sizes, a phenomenon known as winner's curse. We show that our method achieves the same discovery levels as standard approaches but with significantly reduced computational costs. Additionally, it is well-suited for incremental GWAS as new samples are added over time. Our implementation within a bioinformatics workflow management system enhances reproducibility and scalability.
5 October 2023 in CCF Transactions on High Performance Computing
NEC SX-Aurora TSUBASA (SX-AT) is the latest vector supercomputer, consisting of host processors called Vector Hosts (VHs) and vector processors called Vector Engines (VEs). The goal of this work is to simultaneously use both VHs and VEs to increase the resource utilization and improve the system throughput by co-executing more workloads. One difficulty is that performance interferences among VH and VE workloads could occur because they share some computing resources and potentially compete to use the same resource at the same time, so-called resource conflicts. To achieve efficient workload co-execution, first, this paper experimentally investigates the performance interference between a VH and a VE, when each of the two processors executes a different workload. It is empirically shown that the frequency of system calls from the VE workload could be a good indicator to predict if the co-execution could cause severe performance interference, even though monitoring system calls requires a huge runtime overhead and it is impractical to simply use it for decision making of co-execution. Then, this paper proposes a workload co-execution strategy based on a practical approach to identifying a pair of VE and VH workloads that could cause severe performance interferences. Our evaluation results clearly demonstrate that the system call frequency can be used to predict if the workload can affect the performance of another co-executing workload, and VH’s CPU load can be a good approximation of the system call frequency. The proposed approach based on the CPU loads could accurately identify a pair of workloads causing frequent resource conflicts, and thus reduce the risk of severe performance interferences between co-executing workloads on an SX-AT system, resulting in shorter makespan without significantly increasing the turn-around time.
19 September 2024 in bioRxiv
Genome-wide association studies (GWAS) aim to identify genetic variants that are associated with a trait or disease. The scale of genomic datasets has increased to millions of genetic variants and hundreds of thousands of individuals, opening the possibilities for GWAS discoveries. However, large-scale GWAS analyses are prone to high false positive rates because of the multiple testing problem. Permutation testing is the gold standard for maintaining false positive rates, yet it is impractical for large-scale GWAS because it demands vast computational resources. This paper presents GWA-X, a software package that can exploit GPU potential to accelerate permutation testing in GWAS. Unlike previous methods, GWA-X employs a novel whole-genome regression method and a permutation testing strategy to batch the computations of many genetic markers. It achieved a two-order magnitude speed-up compared with the existing CPU-based and GPU-based permutation methods. In addition, it provides an extensible package for GWAS permutation testing on GPUs.