Kiwan Maeng

Kiwan Maeng

Auto-generated profile, based on publicly available information

Kiwan Maeng

Pennsylvania State University, University Park, PA, USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 June 2023
Verified
Conference Paper
Open access
RJRishabh Jain
Rishabh Jain
SCScott Cheng
Scott Cheng
VKVishwas Kalagi
Vishwas Kalagi
VSVrushabh Sanghavi
Vrushabh Sanghavi
SKSamvit Kaul
Samvit Kaul
CDChita R. Das
Chita R. Das
Deep Learning Recommendation Models (DLRMs) are very popular in personalized recommendation systems and are a major contributor to the data-center AI cycles. Due to the high computational and memory bandwidth needs of DLRMs, specifically the embedding stage in DLRM inferences, both CPUs and GPUs are used for hosting such workloads. This is primarily because of the heavy irregular memory accesses in the embedding stage of computation that leads to significant stalls in the CPU pipeline. As the model and parameter sizes keep increasing with newer recommendation models, the computational dominance of the embedding stage also grows, thereby, bringing into question the suitability of CPUs for inference. In this paper, we first quantify the cause of irregular accesses and their impact on caches and observe that off-chip memory access is the main contributor to high latency. Therefore, we exploit two well-known techniques: (1) Software prefetching, to hide the memory access latency suffered by the demand loads and (2) Overlapping computation and memory accesses, to reduce CPU stalls via hyperthreading to minimize the overall execution time. We evaluate our work on a single-core and 24-core configuration with the latest recommendation models and recently released production traces. Our integrated techniques speed up the inference by up to 1.59x, and on average by 1.4x.

17 April 2024
Verified
Conference Paper
Open access
MLMaximilian Lam
Maximilian Lam
JJJeff Johnson
Jeff Johnson
WXWenjie Xiong
Wenjie Xiong
KMKiwan Maeng
Kiwan Maeng
UGUdit Gupta
Udit Gupta
ESEdward Suh
Edward Suh
On-device machine learning (ML) inference can enable the use of private user data on user devices without revealing them to remote servers. However, a pure on-device solution to private ML inference is impractical for many applications that rely on embedding tables that are too large to be stored on-device. In particular, recommendation models typically use multiple embedding tables each on the order of 1--10 GBs of data, making them impractical to store on-device. To overcome this barrier, we propose the use of private information retrieval (PIR) to efficiently and privately retrieve embeddings from servers without sharing any private information. As off-the-shelf PIR algorithms are usually too computationally intensive to directly use for latency-sensitive inference tasks, we 1) propose novel GPU-based acceleration of PIR, and 2) co-design PIR with the downstream ML application to obtain further speedup. Our GPU acceleration strategy improves system throughput by more than 20× over an optimized CPU PIR implementation, and our PIR-ML co-design provides an over 5× additional throughput improvement at fixed model quality. Together, for various on-device ML applications such as recommendation and language modeling, our system on a single V100 GPU can serve up to 100,000 queries per second---a > 100× throughput improvement over a CPU-based baseline---while maintaining model accuracy.


27 April 2024
Verified
Conference Paper
Open access
JLJuntaek Lim
Juntaek Lim
YKYoungeun Kwon
Youngeun Kwon
RHRanggi Hwang
Ranggi Hwang
KMKiwan Maeng
Kiwan Maeng
ESEdward Suh
Edward Suh
MRMinsoo Rhu
Minsoo Rhu
Differential privacy (DP) is widely being employed in the industry as a practical standard for privacy protection. While private training of computer vision or natural language processing applications has been studied extensively, the computational challenges of training of recommender systems (RecSys) with DP have not been explored. In this work, we first present our detailed characterization of private RecSys training using DP-SGD, root-causing its several performance bottlenecks. Specifically, we identify DP-SGD's noise sampling and noisy gradient update stage to suffer from a severe compute and memory bandwidth limitation, respectively, causing significant performance overhead in training private RecSys. Based on these findings, we propose LazyDP, an algorithm-software co-design that addresses the compute and memory challenges of training RecSys with DP-SGD. Compared to a state-of-the-art DP-SGD training system, we demonstrate that LazyDP provides an average 119× training throughput improvement while also ensuring mathematically equivalent, differentially private RecSys models to be trained.


23 October 2025 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
JLJinyu Liu
Jinyu Liu
KMKiwan Maeng
Kiwan Maeng
Secure multi-party computation (MPC) allows multiple parties to collaboratively run machine learning (ML) training and inference without each party revealing its secret data or model weights. Prior works characterized popular MPC-based ML libraries, such as Meta's CrypTen, to reveal their system overheads and built optimizations guided by the observations. However, we found potential concerns in this process. Through a careful inspection of the CrypTen library, we discovered several inefficient implementations that could overshadow fundamental MPC-related overheads. Furthermore, we observed that the characteristics can vary significantly depending on several factors, such as the model type, batch size, sequence length, and network conditions, many of which prior works do not vary during their evaluation. Our results indicate that focusing solely on a narrow experimental setup and/or relying on characterization without a deep understanding can misguide researchers, and call for a more mature framework and standardized evaluation methodology.