Ruibo Fan

Ruibo Fan

Auto-generated profile, based on publicly available information

Ruibo Fan

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 April 2024
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
WWWei Wang
Wei Wang
XCXiaowen Chu
Xiaowen Chu
Sparse Matrix-Matrix Multiplication (SpMM) is a building-block operation in scientific computing and machine learning applications. Recent advancements in hardware, notably Tensor Cores (TCs), have created promising opportunities for accelerating SpMM. However, harnessing these hardware accelerators to speed up general SpMM necessitates considerable effort. In this paper, we undertake a comprehensive analysis of the state-of-the-art techniques for accelerating TC-based SpMM and identify crucial performance gaps. Drawing upon these insights, we propose DTC-SpMM, a novel approach with systematic optimizations tailored for accelerating general SpMM on TCs. DTC-SpMM encapsulates diverse aspects, including efficient compression formats, reordering methods, and runtime pipeline optimizations. Our extensive experiments on modern GPUs with a diverse range of benchmark matrices demonstrate remarkable performance improvements in SpMM acceleration by TCs in conjunction with our proposed optimizations. The case study also shows that DTC-SpMM speeds up end-to-end GNN training by up to 1.91× against popular GNN frameworks.

30 March 2025
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
XYXiangrui Yu
Xiangrui Yu
PDPeijie Dong
Peijie Dong
ZLZeyu Li
Zeyu Li
GGGu Gong
Gu Gong
XCXiaowen Chu
Xiaowen Chu

11 August 2026
Verified
Conference Paper
Open access
WLWenxiang Lin
Wenxiang Lin
XPXinglin Pan
Xinglin Pan
RFRuibo Fan
Ruibo Fan
SSShaohuai Shi
Shaohuai Shi
XCXiaowen Chu
Xiaowen Chu
Communication has emerged as a critical bottleneck in the distributed training of large language models (LLMs). While numerous approaches have been proposed to reduce communication overhead, the potential of lossless compression has remained largely underexplored since compression and decompression typically consume larger overheads than the benefits of reduced communication traffic. We observe that the communication data, including activations, gradients and parameters, during training often follows a near-Gaussian distribution, which is a key feature for data compression. Thus, we introduce ZipCCL, a lossless compressed communication library of collectives for LLM training. ZipCCL is equipped with our novel techniques: (1) theoretically grounded exponent coding that exploits the Gaussian distribution of LLM tensors to accelerate compression without expensive online statistics, (2) GPU-optimized compression and decompression kernels that carefully design memory access patterns and pipeline using communication-aware data layout, and (3) adaptive communication strategies that dynamically switch collective operations based on workload patterns and system characteristics. Evaluated on a 64-GPU cluster using both mixture-of-experts and dense transformer models, ZipCCL reduces communication time by up to 1.35X and achieves end-to-end training speedups of up to 1.18X without any impact on model quality.

5 October 2026 in ACM Transactions on Computer Systems
Verified
Journal Article
Open access
WLWeile Luo
Weile Luo
RFRuibo Fan
Ruibo Fan
ZLZeyu Li
Zeyu Li
DDDayou Du
Dayou Du
HLHongyuan Liu
Hongyuan Liu
XCXiaowen Chu
Xiaowen Chu
This study presents a comprehensive multi-level analysis of the NVIDIA Hopper GPU architecture, focusing on its performance characteristics and novel features. We benchmark Hopper’s memory subsystem, highlighting improvements in the L2 partitioned cache and global memory access compared to Ampere and Ada Lovelace. The evaluation of Hopper’s fourth-generation tensor cores reveals the benefits of FP8 precision and asynchronous wgmma instructions for matrix operations. Additionally, we investigate the performance of DPX instructions for dynamic programming, distributed shared memory (DSM) for inter-SM communication, and the Tensor Memory Accelerator (TMA) for asynchronous data movement. Through multi-level evaluation, we discover that the Hopper architecture demonstrates significant acceleration potential in real-world applications. For instance, the asynchronous programming model supported by TMA achieves a 1.5× speedup in matrix multiplication, FP8 delivers nearly double the performance of FP16, and DPX instructions accelerate a computational biology algorithm by at least 4.75×. Our findings provide actionable insights for optimizing compute-intensive workloads, from AI training to bioinformatics, on Hopper GPUs.

28 January 2026
Verified
Conference Paper
Open access
WLWeile Luo
Weile Luo
YCYuhan Chen
Yuhan Chen
XYXiangrui Yu
Xiangrui Yu
QWQiang Wang
Qiang Wang
RFRuibo Fan
Ruibo Fan
XCXiaowen Chu
Xiaowen Chu

22 March 2026
Verified
Conference Paper
Open access
RFRuibo Fan
Ruibo Fan
XYXiangrui Yu
Xiangrui Yu
XPXinglin Pan
Xinglin Pan
ZLZeyu Li
Zeyu Li
WLWeile Luo
Weile Luo
XCXiaowen Chu
Xiaowen Chu
Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This '' load-compressed, compute-decompressed '' design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21× kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22× over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.