Qiang Wang

Dr. Qiang Wang


Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 August 2025
Verified
Conference Paper
Open access
YWYuxin Wang
Yuxin Wang
YCYuhan Chen
Yuhan Chen
ZLZeyu Li
Zeyu Li
XKXueze Kang
Xueze Kang
YFYuchu Fang
Yuchu Fang
XCXiaowen Chu
Xiaowen Chu

24 June 2020 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
QWQiang Wang
Qiang Wang
XCXiaowen Chu
Xiaowen Chu
Contemporary graphics processing units (GPUs) support dynamic voltage and frequency scaling to balance computational performance and energy consumption. However, accurate and straightforward performance estimation for a given GPU kernel under different frequency settings is still lacking for real hardware, which is essential to determine the best frequency configuration for energy saving. In this article, we reveal a fine-grained analytical model to estimate the execution time of GPU kernels with both core and memory frequency scaling. Compared to the cycle-level simulators, which are too slow to apply on real hardware, our model only needs simple and one-off micro-benchmarks to extract a set of hardware parameters and kernel performance counters without any source code analysis. Our experimental results show that the proposed performance model can capture the kernel performance scaling behaviors under different frequency settings and achieve decent accuracy (average errors of 3.85, 8.6, 8.82, and 8.83 percent on a set of 20 GPU kernels with four modern Nvidia GPUs).

8 June 2022 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
QWQiang Wang
Qiang Wang
XMXinxin Mei
Xinxin Mei
HLHai Liu
Hai Liu
YLYiu-Wing Leung
Yiu-Wing Leung
ZLZongpeng Li
Zongpeng Li
XCXiaowen Chu
Xiaowen Chu
Energy conservation of large data centers for high performance computing workloads, such as deep learning with Big Data, is of critical significance, where cutting down a few percent of electricity translates into million-dollar savings. This work studies energy conservation on emerging CPU-GPU hybrid clusters through dynamic voltage and frequency scaling (DVFS). We aim at minimizing the total energy consumption of processing a batch of offline tasks or a sequence of real-time tasks under deadline constraints. We derive a fast and accurate analytical model to compute the appropriate voltage/frequency setting for each task, and assign multiple tasks to the cluster with heuristic scheduling algorithms. In particular, our model stresses the nonlinear relationship between task execution time and processor speed for GPU-accelerated applications, for more accurately capturing real-world GPU energy consumption. In performance evaluation driven by real-world power measurement traces, our scheduling algorithm shows comparable energy savings to the theoretical upper bound. With a GPU scaling interval where analytically at most 36% of energy can be saved, we record 33-35% of energy savings. Our results are applicable to energy management on modern heterogeneous clusters.

30 March 2025
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
XYXiangrui Yu
Xiangrui Yu
PDPeijie Dong
Peijie Dong
ZLZeyu Li
Zeyu Li
GGGu Gong
Gu Gong
XCXiaowen Chu
Xiaowen Chu

22 April 2024
Verified
Conference Paper
Open access
SSShaohuai Shi
Shaohuai Shi
XPXinglin Pan
Xinglin Pan
QWQiang Wang
Qiang Wang
CLChengjian Liu
Chengjian Liu
XRXiaozhe Ren
Xiaozhe Ren
XCXiaowen Chu
Xiaowen Chu
In recent years, large-scale models can be easily scaled to trillions of parameters with sparsely activated mixture-of-experts (MoE), which significantly improves the model quality while only requiring a sub-linear increase in computational costs. However, MoE layers require the input data to be dynamically routed to a particular GPU for computing during distributed training. The highly dynamic property of data routing and high communication costs in MoE make the training system low scaling efficiency on GPU clusters. In this work, we propose an extensible and efficient MoE training system, ScheMoE, which is equipped with several features. 1) ScheMoE provides a generic scheduling framework that allows the communication and computation tasks in training MoE models to be scheduled in an optimal way. 2) ScheMoE integrates our proposed novel all-to-all collective which better utilizes intra- and inter-connect bandwidths. 3) ScheMoE supports easy extensions of customized all-to-all collectives and data compression approaches while enjoying our scheduling algorithm. Extensive experiments are conducted on a 32-GPU cluster and the results show that ScheMoE outperforms existing state-of-the-art MoE systems, Tutel and Faster-MoE, by 9%-30%.

9 January 2018 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
CLChengjian Liu
Chengjian Liu
QWQiang Wang
Qiang Wang
XCXiaowen Chu
Xiaowen Chu
YLYiu-Wing Leung
Yiu-Wing Leung
Recently, erasure coding has been extensively deployed in large-scale storage systems to replace data replication. With the increase in disk I/O throughput and network bandwidth, the performance of erasure coding becomes a major bottleneck of erasure-coded storage systems. In this paper, we propose a graphics processing unit (GPU)-based implementation of erasure coding named G-CRS, which employs the Cauchy Reed-Solomon (CRS) code, to overcome the aforementioned bottleneck. To maximize the coding performance of G-CRS, we designed and implemented a set of optimization strategies, such as a compact structure to store the bitmatrix in GPU constant memory, efficient data access through shared memory, and decoding parallelism, to fully utilize the GPU resources. In addition, we derived a simple yet accurate performance model to demonstrate the maximum coding performance of G-CRS on GPU. We evaluated the performance of G-CRS through extensive experiments on modern GPU architectures such as Maxwell and Pascal, and compared with other state-of-the-art coding libraries. The evaluation results revealed that the throughput of G-CRS was 10 times faster than most of the other coding libraries. Moreover, G-CRS outperformed PErasure (a recently developed, well optimized CRS coding library on the GPU) by up to 3 times in the same architecture.