Zhenyu Pan

Zhenyu Pan

Auto-generated profile, based on publicly available information

Zhenyu Pan

University of Rochester, Rochester, USA

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 June 2023
Verified
Conference Paper
Open access
AGAnqi Guo
Anqi Guo
YHYuchen Hao
Yuchen Hao
CWChunshu Wu
Chunshu Wu
PHPouya Haghi
Pouya Haghi
ZPZhenyu Pan
Zhenyu Pan
TGTong Geng
Tong Geng
Deep Learning Recommendation Models (DLRMs) are important applications in various domains and have evolved into one of the largest and most important machine learning applications. With their trillions of parameters necessarily exceeding the high bandwidth memory (HBM) capacity of GPUs, ever more massive DLRMs require large-scale multi-node systems for distributed training and inference. However, these all suffer from the all-to-all communication bottleneck, which limits scalability. SmartNICs couple computation and communication capabilities to provide powerful network-facing heterogeneous devices that reduce communication overhead. There has not, however, been a distributed system design that fully leverages SmartNIC resources to address scalability of DLRMs. We propose a software-hardware co-design of a heterogeneous SmartNIC system that overcomes the communication bottleneck of distributed DLRMs, mitigates the pressure on memory bandwidth, and improves computation efficiency. We provide a set of SmartNIC designs of cache systems (including local cache and remote cache) and SmartNIC computation kernels that reduce data movement, relieve memory lookup intensity, and improve the GPU's computation efficiency. In addition, we propose a graph algorithm that improves the data locality of queries within batches and optimizes the overall system performance with higher data reuse. Our evaluation shows that the system achieves 2.1× latency speedup for inference and 1.6× throughput speedup for training.

6 June 2014 in Pattern Analysis and Applications
Verified
Journal Article
Closed access
PDF file
HJHui Jiang
Hui Jiang
ZPZhenyu Pan
Zhenyu Pan
PHPingzhao Hu
Pingzhao Hu