Yanghua Peng

Yanghua Peng

Auto-generated profile, based on publicly available information

Yanghua Peng

ByteDance Inc., Seattle, USA

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 March 2025
Verified
Conference Paper
Open access
GSGuangming Sheng
Guangming Sheng
CZChi Zhang
Chi Zhang
ZYZilingfeng Ye
Zilingfeng Ye
XWXibin Wu
Xibin Wu
WZWang Zhang
Wang Zhang
CWChuan Wu
Chuan Wu

19 January 2021 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
YPYanghua Peng
Yanghua Peng
YBYixin Bao
Yixin Bao
YCYangrui Chen
Yangrui Chen
CWChuan Wu
Chuan Wu
CMChen Meng
Chen Meng
WLWei Lin
Wei Lin
Efficient resource scheduling is essential for maximal utilization of expensive deep learning (DL) clusters. Existing cluster schedulers either are agnostic to machine learning (ML) workload characteristics, or use scheduling heuristics based on operators' understanding of particular ML framework and workload, which are less efficient or not general enough. In this article, we show that DL techniques can be adopted to design a generic and efficient scheduler. Specifically, we propose DL2, a DL-driven scheduler for DL clusters, targeting global training job expedition by dynamically resizing resources allocated to jobs. DL2 advocates a joint supervised learning and reinforcement learning approach: a neural network is warmed up via offline supervised learning based on job traces produced by the existing cluster scheduler; then the neural network is plugged into the live DL cluster, fine-tuned by reinforcement learning carried out throughout the training progress of the DL jobs, and used for deciding job resource allocation in an online fashion. We implement DL2 on Kubernetes and enable dynamic resource scaling in DL jobs on MXNet. Extensive evaluation shows that DL2 outperforms fairness scheduler (i.e., DRF) by 44.1 percent and expert heuristic scheduler (i.e., Optimus) by 17.5 percent in terms of average job completion time.

20 February 2024
Verified
Conference Paper
Open access
JZJuntao Zhao
Juntao Zhao
BWBorui Wan
Borui Wan
CWChuan Wu
Chuan Wu
YPYanghua Peng
Yanghua Peng
HLHaibin Lin
Haibin Lin
The immense sizes of Large-scale language models (LLMs) have led to high resource demand and cost for running the models. Though the models are largely served using uniform high-caliber GPUs nowadays, utilizing a heterogeneous cluster with a mix of available high- and low-capacity GPUs can potentially substantially reduce the serving cost. This paper proposes LLM-PQ, a system that advocates adaptive model quantization and phase-aware partition to improve LLM serving efficiency on heterogeneous GPU clusters. Extensive experiments on production inference workloads demonstrate throughput improvement in inference, showing great advantages over state-of-the-art works.

8 September 2022 in IEEE/ACM Transactions on Networking
Verified
Journal Article
Closed access
YBYixin Bao
Yixin Bao
YPYanghua Peng
Yanghua Peng
CWChuan Wu
Chuan Wu
Nowadays, most leading IT companies host a variety of distributed machine learning (ML) workloads in ML clusters to support AI-driven services, such as speech recognition, machine translation, and image processing. While multiple jobs are executed concurrently in a shared cluster to improve resource utilization, interference among co-located ML jobs can lead to significant performance downgrade. Existing cluster schedulers, such as YARN and Mesos, are interference-agnostic in their job placement, leading to suboptimal resource efficiency and usage. Some literature has studied interference-aware job placement policy, but relies on detailed workload profiling and interference modeling, which is not a general solution. In this work, we present, a deep learning-driven ML cluster scheduler that places heterogeneous training jobs (either with parameter server architecture or all-reduce architecture) in a manner that minimizes interference and maximizes performance (i.e., training completion time minimization). The design of is based on a carefully designed deep reinforcement learning (DRL) framework enhanced with reward modeling. The DRL integrates a dynamic sequence-to-sequence model with the state-of-the-art techniques to stabilize training and improve convergence, including actor-critic algorithm, job-aware action space exploration, multi-head attention, and experience replay. In view of a common lack of reward samples corresponding to different placement decisions, we build an auxiliary sequence-to-sequence reward prediction model, which is trained with historical samples and used for producing reward for unseen placement. Experiments using real ML workloads in a Kubernetes cluster of $6$ GPU servers show that outperforms representative schedulers by $16\%$ - $42\%$ in terms of average job completion time.

12 October 2025
Verified
Conference Paper
Open access
BWBorui Wan
Borui Wan
GLGaohong Liu
Gaohong Liu
ZSZuquan Song
Zuquan Song
JWJun Wang
Jun Wang
YZYun Zhang
Yun Zhang
LXLiang Xiang
Liang Xiang

22 April 2024
Verified
Conference Paper
Open access
HHHanpeng Hu
Hanpeng Hu
JSJunwei Su
Junwei Su
JZJuntao Zhao
Juntao Zhao
YPYanghua Peng
Yanghua Peng
YZYibo Zhu
Yibo Zhu
CWChuan Wu
Chuan Wu
Deep Neural Networks (DNNs) have shown excellent performance in a wide range of machine learning applications. Knowing the latency of running a DNN model or tensor program on a specific device is useful in various tasks, such as DNN graph- or tensor-level optimization and device selection. Considering the large space of DNN models and devices that impedes direct profiling of all combinations, recent efforts focus on building a predictor to model the performance of DNN models on different devices. However, none of the existing attempts have achieved a cost model that can accurately predict the performance of various tensor programs while supporting both training and inference accelerators. We propose CDMPP, an efficient tensor program latency prediction framework for both cross-model and cross-device prediction. We design an informative but efficient representation of tensor programs, called compact ASTs, and a pre-order-based positional encoding method, to capture the internal structure of tensor programs. We develop a domain-adaption-inspired method to learn domain-invariant representations and devise a KMeans-based sampling algorithm, for the predictor to learn from different domains (i.e., different DNN operators and devices). Our extensive experiments on a diverse range of DNN models and devices demonstrate that CDMPP significantly outperforms state-of-the-art baselines with 14.03% and 10.85% prediction error for cross-model and cross-device prediction, respectively, and one order of magnitude higher training efficiency. The implementation and the expanded dataset are available at https://github.com/joapolarbear/cdmpp.