Guangming Sheng

Guangming Sheng

Auto-generated profile, based on publicly available information

Guangming Sheng

Computer Science, The University of Hong Kong, Hong Kong, Hong Kong

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 March 2025
Verified
Conference Paper
Open access
GSGuangming Sheng
Guangming Sheng
CZChi Zhang
Chi Zhang
ZYZilingfeng Ye
Zilingfeng Ye
XWXibin Wu
Xibin Wu
WZWang Zhang
Wang Zhang
CWChuan Wu
Chuan Wu

12 October 2025
Verified
Conference Paper
Open access
BWBorui Wan
Borui Wan
GLGaohong Liu
Gaohong Liu
ZSZuquan Song
Zuquan Song
JWJun Wang
Jun Wang
YZYun Zhang
Yun Zhang
LXLiang Xiang
Liang Xiang

24 August 2024
Verified
Conference Paper
Open access
GSGuangming Sheng
Guangming Sheng
JSJunwei Su
Junwei Su
CHChao Huang
Chao Huang
CWChuan Wu
Chuan Wu
Memory-based Temporal Graph Neural Networks (MTGNNs) are a class of temporal graph neural networks that utilize a node memory module to capture and retain long-term temporal dependencies, leading to superior performance compared to memory-less counterparts. However, the iterative reading and updating process of the memory module in MTGNNs to obtain up-to-date information needs to follow the temporal dependencies. This introduces significant overhead and limits training throughput. Existing optimizations for static GNNs are not directly applicable to MTGNNs due to differences in training paradigm, model architecture, and the absence of a memory module. Moreover, these optimizations do not effectively address the challenges posed by temporal dependencies, making them ineffective for MTGNN training. In this paper, we propose MSPipe, a general and efficient framework for memory-based TGNNs that maximizes training throughput while maintaining model accuracy. Our design specifically addresses the unique challenges associated with fetching and updating node memory states in MTGNNs by integrating staleness into the memory module. However, simply introducing a predefined staleness bound in the memory module to break temporal dependencies may lead to suboptimal performance and lack of generalizability across different models and datasets. To overcome this, we introduce an online pipeline scheduling algorithm in MSPipe that strategically breaks temporal dependencies with minimal staleness and delays memory fetching to obtain fresher memory states. This is achieved without stalling the MTGNN training stage or causing resource contention. Additionally, we design a staleness mitigation mechanism to enhance training convergence and model accuracy. Furthermore, we provide convergence analysis and demonstrate that MSPipe maintains the same convergence rate as vanilla sampling-based GNN training. Experimental results show that MSPipe achieves up to 2.45× speed-up without sacrificing accuracy, making it a promising solution for efficient MTGNN training. The implementation of our paper can be found at the following link: https://github.com/PeterSH6/MSPipe.

18 July 2024 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
YZYuchen Zhong
Yuchen Zhong
GSGuangming Sheng
Guangming Sheng
JLJuncheng Liu
Juncheng Liu
JYJinhui Yuan
Jinhui Yuan
CWChuan Wu
Chuan Wu
As the size of deep learning models gets larger and larger, training takes longer time and more resources, making fault tolerance more and more critical. Existing state-of-the-art methods like CheckFreq and Elastic Horovod need to back up a copy of the model state (i.e., parameters and optimizer states) in memory, which is costly for large models and leads to non-trivial overhead. This paper presents SWIFT, a novel recovery design for distributed deep neural network training that significantly reduces the failure recovery overhead without affecting training throughput and model accuracy. Instead of making an additional copy of the model state, SWIFT resolves the inconsistencies of the model state caused by the failure and exploits the replicas of the model state in data parallelism for failure recovery. We propose a logging-based approach when replicas are unavailable, which records intermediate data and replays the computation to recover the lost state upon a failure. The re-computation is distributed across multiple machines to accelerate failure recovery further. We also log intermediate data selectively, exploring the trade-off between recovery time and intermediate data storage overhead. Evaluations show that SWIFT significantly reduces the failure recovery time and achieves similar or better training throughput during failure-free execution compared to state-of-the-art methods without degrading final model accuracy. SWIFT can also achieve up to 1.16x speedup in total training time compared to state-of-the-art methods.

26 April 2026
Verified
Conference Paper
Open access
GSGuangming Sheng
Guangming Sheng
YTYuxuan Tong
Yuxuan Tong
BWBorui Wan
Borui Wan
WZWang Zhang
Wang Zhang
CJChaobo Jia
Chaobo Jia
CWChuan Wu
Chuan Wu
Reinforcement learning (RL) post-training for Large Language Models (LLMs) is now scaling to large clusters and running for extended durations to enhance model reasoning performance. However, the scalability of existing RL frameworks is limited, as extreme long-tail skewness in RL trajectory generation causes severe GPU underutilization. Current asynchronous RL systems attempt to mitigate this, but they rely on global weight synchronization between the actor and all rollouts, which creates a rigid model update schedule. This global synchronization is ill-suited for the highly skewed and evolving distribution of trajectory generation latency in RL training, crippling training efficiency. Our key insight is that efficient scaling requires breaking this lockstep through trajectory-level asynchrony, which generates and consumes each trajectory independently. We propose Laminar, a scalable and robust RL post-training system built on a fully decoupled architecture. First, we replace global updates with a tier of relay workers acting as a distributed parameter service. This enables asynchronous and fine-grained weight synchronization, allowing rollouts to pull the latest weight anytime without stalling the actor's training loop. Second, a dynamic repack mechanism consolidates long-tail trajectories onto a few dedicated rollouts, maximizing generation throughput. The fully decoupled design also isolates failures, ensuring robustness for long-running jobs. Our evaluation on a 1024–GPU cluster shows that Laminar achieves up to 5.48X training throughput speedup over state-of-the-art systems, while reducing model convergence time.

14 November 2026
Verified
Conference Paper
Open access
PLPengbo Li
Pengbo Li
FZFeiyuan Zhang
Feiyuan Zhang
GSGuangming Sheng
Guangming Sheng
GHGuangxin He
Guangxin He
DCDi Chai
Di Chai
KCKai Chen
Kai Chen