Xiangrui Yu

Xiangrui Yu

Auto-generated profile, based on publicly available information

Xiangrui Yu

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Research Focus & Metrics

Representative Publications

View All Publications
30 March 2025
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
XYXiangrui Yu
Xiangrui Yu
PDPeijie Dong
Peijie Dong
ZLZeyu Li
Zeyu Li
GGGu Gong
Gu Gong
XCXiaowen Chu
Xiaowen Chu

22 March 2026
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
XYXiangrui Yu
Xiangrui Yu
XPXinglin Pan
Xinglin Pan
ZLZeyu Li
Zeyu Li
WLWeile Luo
Weile Luo
XCXiaowen Chu
Xiaowen Chu
Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This '' load-compressed, compute-decompressed '' design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21× kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22× over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.

5 July 2026
Verified
Conference Paper
Open access
XYXiangrui Yu
Xiangrui Yu
RFRuibo Fan
Ruibo Fan
ZLZeyu Li
Zeyu Li
WLWeile Luo
Weile Luo
GGGu Gong
Gu Gong
XCXiaowen Chu
Xiaowen Chu

28 January 2026
Verified
Conference Paper
Open access
WLWeile Luo
Weile Luo
YCYuhan Chen
Yuhan Chen
XYXiangrui Yu
Xiangrui Yu
QWQiang Wang
Qiang Wang
RFRuibo Fan
Ruibo Fan
XCXiaowen Chu
Xiaowen Chu