Hyunmin Choi

Hyunmin Choi

Auto-generated profile, based on publicly available information

Hyunmin Choi

School of Computing, Korea Advanced Institute of Science and Technology, Daejeon, South Korea

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 April 2024
Verified
Conference Paper
Open access
GHGuseul Heo
Guseul Heo
SLSangyeop Lee
Sangyeop Lee
JCJaehong Cho
Jaehong Cho
HCHyunmin Choi
Hyunmin Choi
SLSanghyeon Lee
Sanghyeon Lee
JPJongse Park
Jongse Park
Modern transformer-based Large Language Models (LLMs) are constructed with a series of decoder blocks. Each block comprises three key components: (1) QKV generation, (2) multi-head attention, and (3) feed-forward networks. In batched processing, QKV generation and feed-forward networks involve compute-intensive matrix-matrix multiplications (GEMM), while multi-head attention requires bandwidth-heavy matrix-vector multiplications (GEMV). Machine learning accelerators like TPUs or NPUs are proficient in handling GEMM but are less efficient for GEMV computations. Conversely, Processing-in-Memory (PIM) technology is tailored for efficient GEMV computation, while it lacks the computational power to handle GEMM effectively. Inspired by this insight, we propose NeuPIMs, a heterogeneous acceleration system that jointly exploits a conventional GEMM-focused NPU and GEMV-optimized PIM devices. The main challenge in efficiently integrating NPU and PIM lies in enabling concurrent operations on both platforms, each addressing a specific kernel type. First, existing PIMs typically operate in a "blocked" mode, allowing only either NPU or PIM to be active at any given time. Second, the inherent dependencies between GEMM and GEMV in LLMs restrict their parallel processing. To tackle these challenges, NeuPIMs is equipped with dual row buffers in each bank, facilitating the simultaneous management of memory read/write operations and PIM commands. Further, NeuPIMs employs a runtime sub-batch interleaving technique to maximize concurrent execution, leveraging batch parallelism to allow two independent sub-batches to be pipelined within a single NeuPIMs device. Our evaluation demonstrates that compared to GPU-only, NPU-only, and a naïve NPU+PIM integrated acceleration approaches, NeuPIMs achieves 3×, 2.4× and 1.6× throughput improvement, respectively.

3 November 2025 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
JCJaehong Cho
Jaehong Cho
HCHyunmin Choi
Hyunmin Choi
JPJongse Park
Jongse Park
This paper introduces LLMServingSim2.0, a system simulator designed for exploring heterogeneous hardware in large-scale LLM serving systems. LLMServingSim2.0 addresses two key limitations of its predecessor: (1) integrating hardware models into system-level simulators is non-trivial due to the lack of a clear abstraction, and (2) existing simulators support only a narrow subset of serving techniques, leaving no infrastructure that captures the breadth of approaches in modern LLM serving. To overcome these issues, LLMServingSim2.0 adopts trace-driven performance modeling, accompanied by an operator-level latency profiler, enabling the integration of new accelerators with a single command. It further embeds up-to-date serving techniques while exposing flexible interfaces for request routing, cache management, and scheduling policies. In a TPU case study, our profiler requires 18.5× fewer LoC and outperforms the predecessor's hardware-simulator integration, demonstrating LLMServingSim2.0's low-effort hardware extensibility. Our experiments further show that LLMServingSim2.0 reproduces GPU-based LLM serving with 1.9% error, while maintaining practical simulation time, making it a comprehensive platform for both hardware developers and LLM service providers.

6 May 2026 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
JHJinghan Huang
Jinghan Huang
HZHongkun Zeng
Hongkun Zeng
MMMike Montano
Mike Montano
JCJaehong Cho
Jaehong Cho
HCHyunmin Choi
Hyunmin Choi
NKNam Sung Kim
Nam Sung Kim
The cost of large language model (LLM) inference is heavily tied to the immense memory capacity required to store model parameters—a challenge exacerbated by current systems that replicate these parameters across multiple servers to scale. To tackle this challenge, we may consider LLM inference systems exploiting a CXL memory pool to store and share model parameters across multiple servers, while keeping server-specific activations and KV-caches within each server’s local memory. Motivated by the lack of existing simulators for such systems, we first develop a simulator that extends LLMServingSim and integrates it with NS-3 and Mess to accurately model the CXL switch and the memory modules within the CXL pool, respectively. Second, we validate this simulator against a system connecting an industry CXL memory pool to two servers running two instances of the same LLM; across separate evaluations for OPT-1.6B and OPT-6.7B, the simulator demonstrates a $\approx$3% error in inference latency. Lastly, using our simulator, we uncover a significant sensitivity between inference latency and the interconnect bandwidth of the CXL memory pool, demonstrating that adopting an emerging optical interconnect over the traditional electrical counterpart reduces inference latency by 67.9% and 64.4% for OPT-1.6B and OPT-6.7B, respectively.