Kwanhee Kyung

Kwanhee Kyung

Auto-generated profile, based on publicly available information

Kwanhee Kyung

Seoul National University, Seoul, South Korea

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 April 2024
Verified
Conference Paper
Open access
JPJaehyun Park
Jaehyun Park
JCJaewan Choi
Jaewan Choi
KKKwanhee Kyung
Kwanhee Kyung
MKMichael Jaemin Kim
Michael Jaemin Kim
YKYongsuk Kwon
Yongsuk Kwon
JAJung Ho Ahn
Jung Ho Ahn

15 August 2023 in IEEE Computer Architecture Letters
Verified
Journal Article
Open access
JCJaewan Choi
Jaewan Choi
JPJaehyun Park
Jaehyun Park
KKKwanhee Kyung
Kwanhee Kyung
NKNam Sung Kim
Nam Sung Kim
JAJung Ho Ahn
Jung Ho Ahn
Transformer-based generative models, such as GPT, summarize an input sequence by generating key/value (KV) matrices through attention and generate the corresponding output sequence by utilizing these matrices once per token of the sequence. Both input and output sequences tend to get longer, which improves the understanding of contexts and conversation quality. These models are also typically batched for inference to improve the serving throughput. All these trends enable the models' weights to be reused effectively, increasing the relative importance of sequence generation, especially in processing KV matrices through attention. We identify that the conventional computing platforms (e.g., GPUs) are not efficient at handling this attention part for inference because each request generates different KV matrices, it has a low operation per byte ratio regardless of the batch size, and the aggregate size of the KV matrices can even surpass that of the entire model weights. This motivates us to propose AttAcc, which exploits the fact that the KV matrices are written once during summarization but used many times (proportional to the output sequence length), each multiplied by the embedding vector corresponding to an output token. The volume of data entering/leaving AttAcc could be more than orders of magnitude smaller than what should be read internally for attention. We design AttAcc with multiple processing-in-memory devices, each multiplying the embedding vector with the portion of the KV matrices within the devices, saving external (inter-device) bandwidth and energy consumption. AttAcc improves the performance and energy efficiency for DRAM access of serving GPT-3 by 3.19 and 3.22 times over the DGX A100 640 GB with an input sequence length of 2048 and an output sequence length of 128.

30 May 2024
Verified
Conference Paper
Open access
SYSungmin Yun
Sungmin Yun
HNHwayong Nam
Hwayong Nam
KKKwanhee Kyung
Kwanhee Kyung
JPJaehyun Park
Jaehyun Park
BKByeongho Kim
Byeongho Kim
JAJung Ho Ahn
Jung Ho Ahn
An embedding layer is one of the most critical building blocks of deep neural networks, especially for recommender systems and graph neural networks. The embedding layer dominates a large portion of the total execution time due to its large memory requirements and little data reuse in operations. To accelerate the embedding layers, dual in-line memory module (DIMM) based near-data processing architectures have been proposed. They amplify bandwidth by adding a processing unit to the DIMM’s buffer. However, prior architectures have less capacity scalability due to the limited number of memory channels. Crucially, they are limited in performance improvement due to the load imbalance problem and the limitations of DIMM-based memory systems with a multi-drop bus structure between the processing units and the host. In this paper, we propose CLAY, a CXL-based scalable near-data processing architecture that accelerates general embedding layers in DNN. Breaking away from conventional memory channel structures, CLAY interconnects the DRAM modules to reduce the data transfer overhead among DRAM modules. Furthermore, we devise a dedicated memory address mapping to mitigate load imbalance in CLAY and a packet duplication scheme that enables full utilization of CLAY by reducing the required instruction transmission bandwidth. We propose a method of scaling CLAY and a software stack to use CLAY. Compared to the state-of-the-art NDP architectures of FeaNMP and G-NMP, CLAY achieves an end-to-end speedup of up to 1.87 × and 2.77 × for recommender systems and graph neural networks, respectively.

30 March 2025
Verified
Conference Paper
Open access
WDWanju Doh
Wanju Doh
YMYaebin Moon
Yaebin Moon
SKSeoyoung Ko
Seoyoung Ko
SCSeunghwan Chung
Seunghwan Chung
KKKwanhee Kyung
Kwanhee Kyung
JAJung Ho Ahn
Jung Ho Ahn

24 July 2025 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
KKKwanhee Kyung
Kwanhee Kyung
SYSungmin Yun
Sungmin Yun
JAJung Ho Ahn
Jung Ho Ahn
Large Language Models (LLMs) applying Mixture-of-Experts (MoE) scale to trillions of parameters but require vast memory, motivating a line of research to offload expert weights from fast-but-small DRAM (HBM) to denser Flash SSDs. While SSDs provide cost-effective capacity, their read energy per bit is substantially higher than that of DRAM. This paper quantitatively analyzes the energy implications of offloading MoE expert weights to SSDs during the critical decode stage of LLM inference. Our analysis, comparing SSD, CPU memory (DDR), and HBM storage scenarios for models like DeepSeek-R1, reveals that offloading MoE weights to current SSDs drastically increases per-token-generation energy consumption (e.g., by up to $\sim 12\times$ compared to the HBM baseline), dominating the total inference energy budget. Although techniques like prefetching effectively hide access latency, they cannot mitigate this fundamental energy penalty. We further explore future technological scaling, finding that the inherent sparsity of MoE models could potentially make SSDs energy-viable if Flash read energy improves significantly, roughly by an order of magnitude.

13 January 2023 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
YMYaebin Moon
Yaebin Moon
WDWanju Doh
Wanju Doh
KKKwanhee Kyung
Kwanhee Kyung
ELEojin Lee
Eojin Lee
JAJung Ho Ahn
Jung Ho Ahn
Tiered memory using DRAM as upper-tier (fast memory) and emerging slower-but-larger byte-addressable memory as lower-tier (slow memory) is a promising approach to expanding main-memory capacity. Based on the observation that there are many cold pages in data-center applications, proactive demotion schemes demote cold pages to slow memory even when free space in fast memory is not deficient. Prior works on proactive demotion lower the requirement of expensive fast-memory capacity by reducing applications' resident set size in fast memory. Also, some of the prior works mitigate the massive performance drop due to insufficient fast-memory capacity when there is a spike in demand for hot data. However, there is room for further improvement to save a larger fast-memory capacity with further aggressive demotion, which can fully reap the aforementioned advantages of proactive demotion. In this paper, we propose a new proactive demotion scheme, ADT, which performs a ggressive d emotion and promotion for t iered memory. Using the memory access locality within the unit in which applications and memory allocators allocate memory, ADT extends the unit of demotion/promotion from the page adopted by prior works to make its demotion more aggressive. By performing demotion and promotion by the extended unit, ADT reduces 29% of fast-memory usage with only a 2.3% performance drop. Also, it achieves 2.28× speedup compared to the default Linux kernel when the system's memory usage is larger than fast-memory capacity, which outperforms state-of-the-art schemes for tiered memory management.