Zheng Wei

Zheng Wei

Auto-generated profile, based on publicly available information

Zheng Wei

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 November 2025
Verified
Conference Paper
Open access
WHWenjing Huang
Wenjing Huang
JYJinwu Yang
Jinwu Yang
SYShengquan Yin
Shengquan Yin
HLHaoxu Li
Haoxu Li
YGYida Gu
Yida Gu
DTDingwen Tao
Dingwen Tao


13 July 2026
Verified
Conference Paper
Open access
MLMan Liu
Man Liu
XLXingchen Liu
Xingchen Liu
XTXingjian Tian
Xingjian Tian
BLBing Lu
Bing Lu
SLShengkai Lyu
Shengkai Lyu
DTDingwen Tao
Dingwen Tao
Handling communication overhead in large-scale tensor-parallel training remains a critical challenge due to the dense, near-zero distributions of intermediate tensors, which exacerbate errors under frequent communication and introduce significant computational overhead during compression. To this end, we propose TACO (Tensor-parallel Adaptive COmmunication compression), a robust FP8-based framework for compressing TP intermediate tensors. First, we employ a data-driven reshaping strategy combined with an Adaptive Scale–Hadamard Transform to enable high-fidelity FP8 quantization, while its Dual-Scale Quantization mechanism ensures numerical stability throughout training. Second, we design a highly fused compression operator to reduce memory traffic and kernel launch overhead, allowing efficient overlap with communication. Finally, we integrate TACO with existing state-of-the-art methods for Data and Pipeline Parallelism to develop a compression-enabled 3D-parallel training framework. Detailed experiments on GPT models and Qwen model demonstrate up to 1.87 × end-to-end throughput improvement while maintaining near-lossless accuracy, validating the effectiveness and efficiency of TACO in large-scale training.

11 August 2026
Verified
Conference Paper
Open access
ZLZedong Liu
Zedong Liu
XMXinyang Ma
Xinyang Ma
DLDejun Luo
Dejun Luo
HZHairui Zhao
Hairui Zhao
BLBing Lu
Bing Lu
GTGuangming Tan
Guangming Tan
LLMs are widely adopted in production, pushing inference systems to their limits. Disaggregated LLM serving (e.g., PD separation and KV state disaggregation) improves scalability and cost efficiency, but it also turns KV into an explicit payload crossing network and storage boundaries, making KV a dominant end-to-end bottle-neck. Existing KV compression are typically statically configured at runtime, despite production service context varies over time in workload mix, bandwidth, and SLO/quality budgets. As a result, a fixed choice can be suboptimal or even increase latency. We present KVServe, the first service-aware and adaptive KV communication compression framework for disaggregated LLM serving: KVServe (1) unifies KV compression into a modular strategy space with new components and cross-method recomposition; (2) introduces Bayesian Profiling Engine that efficiently searches this space and distills a 3D Pareto candidate set, reducing 50x offline search overhead; and (3) deploys a Service-Aware Online Controller that combines an analytical latency model with a lightweight bandit to select profiles under constraints and correct offline-to-online mismatch. Integrated into vLLM and evaluated across datasets, models, GPUs and networks, KVServe achieves up to 9.13x JCT speedup in PD-separated serving and up to 32.8x TTFT reduction in KV-disaggregated serving.

1 July 2026 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
ZWZheng Wei
Zheng Wei
JXJing Xing
Jing Xing
YGYida Gu
Yida Gu
WHWenjing Huang
Wenjing Huang
YMYili Ma
Yili Ma
DTDingwen Tao
Dingwen Tao

20 July 2025
Verified
Conference Paper
Open access
ZWZheng Wei
Zheng Wei
JXJing Xing
Jing Xing
YGYida Gu
Yida Gu
WHWenjing Huang
Wenjing Huang
DDDong Dai
Dong Dai
DTDingwen Tao
Dingwen Tao