YeonGon Cho

YeonGon Cho

Auto-generated profile, based on publicly available information

YeonGon Cho

Samsung Advanced Institute of Technology

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 July 2021 in IEEE Micro
Verified
Journal Article
Closed access
LKLiu Ke
Liu Ke
XZXuan Zhang
Xuan Zhang
JSJinin So
Jinin So
JLJong-Geon Lee
Jong-Geon Lee
SKShin-Haeng Kang
Shin-Haeng Kang
HLHsien-Hsin Sean Lee
Hsien-Hsin Sean Lee
Near-memory processing (NMP) is a prospective paradigm enabling memory-centric computing. By moving the compute capability next to the main memory (DRAM modules), it can fundamentally address the CPU-memory bandwidth bottleneck and thus effectively improve the performance of memory-constrained workloads. Using the personalized recommendation system as a driving example, we developed a scalable, practical DIMM-based NMP solution tailor-designed for accelerating the inference serving. Our solution is demonstrated on a versatile FPGA-enabled NMP platform called AxDIMM that allows rapid prototyping and evaluation of NMP's performance potential on real hardware under a realistic system setting using industry-representative recommendation framework. We experimentally validated the performance of a two-ranked AxDIMM prototype which achieves up to 1.89x speedup in latency and 31.6% memory energy saving for embedding operations. For end-to-end recommendation inference serving, AxDIMM improves the throughput up to 1.5x and latency-bounded throughput up to 1.77x, respectively.

25 March 2024 in IEEE Micro
Verified
Journal Article
Closed access
BKByeongho Kim
Byeongho Kim
SCSanghoon Cha
Sanghoon Cha
SPSangsoo Park
Sangsoo Park
JLJieun Lee
Jieun Lee
SLSukhan Lee
Sukhan Lee
KSKyomin Sohn
Kyomin Sohn
Large Language Model (LLM) changes our lives, while it requires unprecedented computing resources, especially it requires large memory capacity and high bandwidth to process weights. However, while the logic process was developing, the speed of development of the memory process could not keep up, causing problems that resulted in the performance of LLM being hindered by memory. Samsung have introduced breakthrough Processing-in-Memory/Processing-near-Memory (PIM/PNM) solutions that enhance the main memory bandwidth. With the HBM-PIM-based GPU-cluster system and LPDDR5-PIM-based system, the performance of transformer-based LLMs improved by up to 1.9× and 2.7×, respectively. The CXL-based PNM solution serves memory-centric computing systems by implementing logic inside the CXL memory controller. This results in a performance gain of over 4.4× with an energy reduction of about 53% with PNM. Furthermore, we provide PIM/PNM software stacks, including an AI compiler targeting the acceleration of AI models.

13 October 2024
Verified
Conference Paper
Open access
SKSowoong Kim
Sowoong Kim
ESEunyeong Sim
Eunyeong Sim
YSYoungsam Shin
Youngsam Shin
YCYeonGon Cho
YeonGon Cho
WBWoongki Baek
Woongki Baek
Generative artificial intelligence is widely used for various tasks such as language translation and art creation. Generative inference employs the key and value tensors to encode relational information among the tokens in the input and output sequences. Most of the existing generative inference frameworks use KV caching (KVC), which caches the key and value tensors (i.e., the KV cache) to avoid recomputing the key and value tensors for each of the processed tokens. Despite the widespread use of KVC, in-depth characterization of KVC with tensor offloading, which enables generative inference with a single GPU, remains yet to be explored. To bridge this gap, this work presents an in-depth characterization study of KVC, which demonstrates that KVC makes a sub-optimal trade-off between computations and communications in the context of generative inference with tensor offloading. Guided by the characterization results, we propose a novel caching technique called activation sequence caching (ASC) for high-throughput and memory-efficient generative inference with a single GPU. ASC is designed to significantly reduce the memory usage and communication overheads of generative inference at the cost of the increased computational complexity. We provide an analytical analysis of the three caching techniques (i.e., zero caching, KVC, and ASC) for generative inference in terms of the memory usage, communication overheads, and computational complexity. Our quantitative evaluation demonstrates the effectiveness of ASC in that ASC significantly (e.g., 78.3% higher throughput) outperforms a state-of-the-art generative inference framework (i.e., FlexGen) that implements KVC across various model sizes, input and output sequence lengths, and GPUs.

24 November 2023 in IEEE Access
Verified
Journal Article
Open access
MHMyeonggyun Han
Myeonggyun Han
EPEunseong Park
Eunseong Park
YSYoungsam Shin
Youngsam Shin
DODeok-Jae Oh
Deok-Jae Oh
YCYeongon Cho
Yeongon Cho
WBWoongki Baek
Woongki Baek
With the rapid growth in memory demands, the slowdown of DRAM scaling, and the DRAM price fluctuations, DRAM has become one of the critical resources in cloud computing systems and datacenters. The compressed memory swap (CMS) is a promising technique that improves the effective memory capacity of the underlying computer system by compressing and storing a subset of pages in memory instead of the disk swap. While prior works have extensively investigated resource management techniques for workload consolidation, they lack the capability of dynamically allocating cores, memory, andCMSto the consolidated applications in a controlled and efficient manner. To bridge this gap, this work presents the in-depth characterization of the impact of cores, memory, and CMS on the QoS and throughput of the consolidated latency-critical (LC) and batch applications. Guided by the characterization results, we propose COSMOS, a software-based runtime system for coordinated management of cores, memory, and CMS for QoS-aware and efficient workload consolidation for memory-intensive applications. COSMOS dynamically collects the runtime data from the consolidated applications and the underlying system and allocates the resources to the consolidated applications in a way that achieves high throughput with strong QoS guarantees. Our quantitative evaluation based on a real system and widely-used memory-intensive benchmarks demonstrates the effectiveness of COSMOS in that it robustly satisfies the QoS and achieves high throughput across all the evaluatedworkload mixes and scenarios and significantly reduces the number of explored system states.

24 January 2025 in IEEE Transactions on Dependable and Secure Computing
Verified
Journal Article
Closed access
WXWenjie Xiong
Wenjie Xiong
LKLiu Ke
Liu Ke
MOMaxim Ostapenko
Maxim Ostapenko
YTYongmin Tai
Yongmin Tai
YCYeongon Cho
Yeongon Cho
GSG. Edward Suh
G. Edward Suh
Trusted Executing Environments (TEEs) in hardware designs protect program execution from other untrusted software programs in the processor as well as untrusted off-chip hardware components. Meanwhile, Near-Memory Processing (NMP) has shown performance and energy benefits on memory-intensive workloads. Recently, novel memory encryption schemes have been proposed to allow TEEs to leverage the benefit of NMP without trusting the NMP components. In this paper, we present a system design of confidential computing with NMP that can be directly used in Intel SGX, a TEE platform available in commercial processors today. We develop the full software stack and evaluate the results on commercial processors with the emulated AxDIMM, an FPGA-based NMP platform. In our case study on personalized Deep Learning Recommendation Model (DLRM) inference, the proposed confidential computing in NMP achieves up to 1.51× latency reduction and up to 2.57× throughout improvement.