Peng Xie

Peng Xie

Auto-generated profile, based on publicly available information

Peng Xie

Information Sciences Institute, University of Southern California, Arlington, VA, USA

Research Focus & Metrics

Representative Publications

View All Publications
22 April 2024
Verified
Conference Paper
Open access
CIConnor Imes
Connor Imes
ARAndrew Rittenbach
Andrew Rittenbach
PXPeng Xie
Peng Xie
DKDong In D. Kang
Dong In D. Kang
JWJohn Paul Walters
John Paul Walters
SCStephen P. Crago
Stephen P. Crago
Deep learning is commonly used to make personalized recommendations to users for a wide variety of activities. However, deep learning recommendation model (DLRM) training is increasingly dominated by all-to-all and many-to-many communication patterns. While there are a wide variety of algorithms to efficiently overlap communication and computation for many collective operations, these patterns are strictly limited by network bottlenecks. We propose co-designing DLRM model training with the recently proposed Opera network, which is designed to avoid multiple network hops using time-varying source-to-destination circuits. Using measurements from state-of-the-art NVIDIA A100 GPUs, we simulate DLRM model training on networks ranging from 16 to 1024 nodes and demonstrate up to 1.79× improvement using Opera compared with equivalent fat-tree networks. We identify important parameters affecting training time and demonstrate that careful co-design is needed to optimize training latency.