Sa Wang

Sa Wang

Auto-generated profile, based on publicly available information

Sa Wang

Institute of Computing Technology, Chinese Academy of Science, Beijing, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 June 2025 in ACM Transactions on Architecture and Code Optimization
Verified
Journal Article
Open access
CHCunchen Hu
Cunchen Hu
HHHeyang Huang
Heyang Huang
LXLiangliang Xu
Liangliang Xu
XCXusheng Chen
Xusheng Chen
CWChenxi Wang
Chenxi Wang
YSYizhou Shan
Yizhou Shan
Transformer-based large language model (LLM) inference serving is now the backbone of many cloud services. LLM inference consists of a prefill phase and a decode phase. However, existing LLM deployment practices often overlook the distinct characteristics of these phases, leading to significant interference. To mitigate interference, our insight is to carefully schedule and group inference requests based on their characteristics. We realize this idea in ShuffleInfer through three pillars. First, it partitions prompts into fixed-size chunks so that the accelerator always runs close to its computation-saturated limit. Second, it disaggregates prefill and decode instances so each can run independently. Finally, it uses a smart two-level scheduling algorithm augmented with predicted resource usage to avoid decode scheduling hotspots. Results show that ShuffleInfer improves time-to-first-token (TTFT), job completion time (JCT), and inference efficiency in terms of performance per dollar by a large margin, e.g., it uses 38% less resources all the while lowering average TTFT and average JCT by 97% and 47%, respectively.

22 April 2024
Verified
Conference Paper
Open access
GLGuodong Liu
Guodong Liu
YMYoushan Miao
Youshan Miao
ZLZhiqi Lin
Zhiqi Lin
XSXiaoxiang Shi
Xiaoxiang Shi
SMSaeed Maleki
Saeed Maleki
SWSa Wang
Sa Wang
Many parallel mechanisms, including data parallelism, tensor parallelism, and pipeline parallelism, have been proposed and combined together to support training increasingly large deep neural networks (DNN) on massive GPU devices. Given a DNN model and GPU cluster, finding the optimal configuration by combining these parallelism mechanisms is an NP-hard problem. Widely adopted mathematical programming approaches have been proposed to search in a configuration subspace, but they are still too costly when scaling to large models over numerous devices. Aceso is a scalable parallel-mechanism auto-configuring system that operates iteratively. For a given parallel configuration, Aceso identifies a performance bottleneck and then, by summarizing all possible configuration adjustments with their resource consumption changes, infers their performance impacts to the bottleneck and selects one that mitigates the bottleneck. This process repeats for many iterations until a desired final configuration is found. Unlike mathematical programming approaches that examine the configurations subspace to find the optimal solution, Aceso searches in the configuration space in a stochastic approach by repeatedly identifying and alleviating bottlenecks. Aceso significantly reduces configuration searching cost by taking the approach of resolving one bottleneck at a time. This allows Aceso to find configurations that would be usually missed in subspace search approaches. We implemented and tested Aceso on representative DNN models. Evaluations show that it can scale to 1K-layer models. Compared to state-of-the-art systems, Aceso achieves up to 1.33× throughput improvement with less than 5% of the searching cost.

23 June 2024
Verified
Conference Paper
Open access
YXYinan Xu
Yinan Xu
SWSa Wang
Sa Wang
DTDan Tang
Dan Tang
NSNinghui Sun
Ninghui Sun
YBYungang Bao
Yungang Bao
Coverage metrics have been widely adopted to quantify the completeness of hardware verification. Recently, coverage-guided fuzzing has emerged as a popular method for automatically creating test inputs toward higher verification coverage reach. However, we observe that its effectiveness on CPUs is hindered by limited sources of seed corpus and efficiency of mutations. To broaden the fuzzing horizons, this paper proposes the PathFuzz framework incorporating an efficient input format for fuzzing CPUs, the footprint memory, with seed corpus from real-world large-scale programs. Experiments demonstrate that using PathFuzz reaches over 95% verification coverage with four long-standing bugs newly identified in two well-known open-source CPU designs.

31 July 2023 in Journal of Computer Science and Technology
Verified
Journal Article
Closed access
PDF file
YXYi-Nan Xu
Yi-Nan Xu
ZYZi-Hao Yu
Zi-Hao Yu
KWKai-Fan Wang
Kai-Fan Wang
HWHua-Qiang Wang
Hua-Qiang Wang
JLJia-Wei Lin
Jia-Wei Lin
YBYun-Gang Bao
Yun-Gang Bao

22 June 2023
Verified
Conference Paper
Open access
CHCunchen Hu
Cunchen Hu
CWChenxi Wang
Chenxi Wang
SWSa Wang
Sa Wang
NSNinghui Sun
Ninghui Sun
YBYungang Bao
Yungang Bao
YSYizhou Shan
Yizhou Shan

20 November 2024
Verified
Conference Paper
Open access
ZHZiwei Huang
Ziwei Huang
MXMengyao Xie
Mengyao Xie
STShibo Tang
Shibo Tang
ZCZihao Chang
Zihao Chang
ZYZhicheng Yao
Zhicheng Yao
SWSa Wang
Sa Wang
Identifying and managing performance interference in clouds has long been a critical and challenging task for cloud providers. They keep seeking useful performance indicators from underlying systems to monitor cloud applications accurately. However, state-of-the-art indicators are either sensitive to limited applications and resource contention or are unrobust to the continually changing production environments. There still lacks a practical and efficient indicator for production environments. This paper proposes INS, a cloud runtime system that can effectively detect the performance fluctuation of online cloud applications and reallocate resources to curb performance interference. It proposes INSPath as the new performance indicator to describe the degree of performance degradation and pinpoint the resource bottlenecks. Our evaluation of nine widely-used applications demonstrates that INS can detect the SLO violations of applications and identify the resource bottleneck accurately. Meanwhile, INS outperforms state-of-the-art PARTIES with more responsive and effective resource tuning and fewer SLO violations.