Soyeon Kim

Soyeon Kim

Auto-generated profile, based on publicly available information

Soyeon Kim

School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 May 2023 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SKSangyeob Kim
Sangyeob Kim
SKSangjin Kim
Sangjin Kim
SUSoyeon Um
Soyeon Um
SKSoyeon Kim
Soyeon Kim
KKKwantae Kim
Kwantae Kim
HYHoi-Jun Yoo
Hoi-Jun Yoo
A highly energy-efficient neuromorphic computing-in-memory (Neuro-CIM) processor is proposed for ultralow-power deep learning applications. Neuro-CIM can support spiking neural network (SNN) to eliminate the power and area overhead of previous CIM processor. The sign extended bits gating reduces the bitline (BL) voltage switching rate due to negative small-magnitude weights allowing 38% power reduction at 8-b weight condition and 25% at 4-b weight condition. In addition, Neuro-CIM replaces high-precision analog-to-digital converter (ADC) with 1-b comparator by exploiting the characteristic of the SNN, and thus, power and area efficiencies are significantly enhanced. Furthermore, the early stopping scheme terminates unnecessary neuronal operations, reducing power consumption by 31%. In addition, the analog and digital networks are integrated for high reconfigurability and energy efficiency. The analog network with voltage folding circuit enables accurate analog-domain aggregation by increasing the dynamic range without compromising the voltage resolution. The digital network-in-memory supports input–output channel extension for high reconfigurability and input data reuse scheme for reducing input memory (IMEM) access. Neuro-CIM is fabricated in 28-nm CMOS technology and occupies the 2.9-mm $^{2}$ die area. It achieves the state-of-the-art energy consumption per classification of 0.72 $\mu$ J and 92.1% accuracy for CIFAR-10 with 4-b input and 4-b weight and 372.2 $\mu$ J and 65.8% accuracy for ImageNet with 6-b input and 8-b weight at 200 MHz, and 1.1-V conditions. Moreover, the proposed CIM processor achieves 310.37 tera operations per second/watt (TOPS/W) and 90.7% accuracy with 4-b input and 1-b weight for Canadian Institute for Advanced Research, 10 classes (CIFAR-10) classification.

22 April 2021 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SKSanghoon Kang
Sanghoon Kang
DHDonghyeon Han
Donghyeon Han
JLJuhyoung Lee
Juhyoung Lee
DIDongseok Im
Dongseok Im
SKSangyeob Kim
Sangyeob Kim
HYHoi-Jun Yoo
Hoi-Jun Yoo
This article presents generative adversarial network processing unit (GANPU), an energy-efficient multiple deep neural network (DNN) training processor for GANs. It enables on-device training of GANs on performance- and battery-limited mobile devices, without sending user-specific data to servers, fully evading privacy concerns. Training GANs require a massive amount of computation, and therefore, it is difficult to accelerate in a resource-constrained platform. Besides, networks and layers in GANs show dramatically changing operational characteristics, making it difficult to optimize the processor's core and bandwidth allocation. For higher throughput and energy efficiency, this article proposed three key features. An adaptive spatiotemporal workload multiplexing is proposed to maintain high utilization in accelerating multiple DNNs in a single GAN model. To take advantage of ReLU sparsity during both inference and training, dual-sparsity exploitation architecture is proposed to skip redundant computations due to input and output feature zeros. Moreover, an exponent-only ReLU speculation (EORS) algorithm is proposed along with its lightweight processing element (PE) architecture, to estimate the location of output feature zeros during the inference with minimal hardware overhead. Fabricated in a 65-nm process, the GANPU achieved the energy efficiency of 75.68 TFLOPS/W for 16-bit floating-point computation, which is 4.85 × higher than the state of the art. As a result, GANPU enables on-device training of GANs with high energy efficiency.

15 November 2023 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SKSangyeob Kim
Sangyeob Kim
SKSoyeon Kim
Soyeon Kim
SHSeongyon Hong
Seongyon Hong
SKSangjin Kim
Sangjin Kim
DHDonghyeon Han
Donghyeon Han
HYHoi-Jun Yoo
Hoi-Jun Yoo
In this article, we propose a complementary deep-neural-network (C-DNN) processor by combining convolutional neural network (CNN) and spiking neural network (SNN) to take advantage of them. The C-DNN processor can support both complementary inference and training with heterogeneous CNN and SNN core architecture. In addition, the C-DNN processor is the first DNN accelerator application-specific integrated circuit (ASIC) that can support CNN–SNN workload division by using their magnitude–energy tradeoff. The C-DNN processor integrates the CNN–SNN workload allocator and attention module to find a more energy-efficient network domain for each workload in DNN. They enable the C-DNN processor to operate at the energy optimal point. Moreover, the SNN processing element (PE) array with distributed L1 cache can reduce the redundant memory access for SNN processing, resulting in a 42.2%–49.1% reduction. For high energy-efficient DNN training, the C-DNN processor integrates the global counter and local delta-weight (LDW) unit to eliminate power-consuming counters for a forward delta-weight generation. Furthermore, the forward delta-weight-based sparsity generation (FDWSG) is proposed to reduce the number of operations for training by 31%–79% The C-DNN processor achieves an energy efficiency of 85.8 and 79.9 TOPS/W for inference with CIFAR-10 and CIFAR-100, respectively (VGG-16). Moreover, the C-DNN processor achieves ImageNet classification with state-of-the-art energy efficiency of 24.5 TOPS/W (ResNet-50). For training, the C-DNN processor achieves the state-of-the-art energy efficiency of 84.5 and 17.2 TOPS/W for CIFAR-10 and ImageNet, respectively. Furthermore, it achieves 77.1% accuracy for ImageNet training with ResNet-50.

9 May 2023 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SKSangyeob Kim
Sangyeob Kim
SKSangjin Kim
Sangjin Kim
SUSoyeon Um
Soyeon Um
SKSoyeon Kim
Soyeon Kim
JLJuhyoung Lee
Juhyoung Lee
HYHoi-Jun Yoo
Hoi-Jun Yoo
In this article, an energy-efficient spike domain deep-neural-network processor (SNPU) is proposed. Recently, many sparsity-aware processors were proposed to increase energy efficiency. However, because the activation of the real-world network was not high compared to the ideal condition, they were unable to completely employ integrated zero skipping logic. In addition, they employed weight sparsity by pruning, but their zero skipping logic was designed to perform best only under specific sparsity condition. We adopt two-step encoding (neural and computing encoding) and an omni-sparsity handling unit to solve these issues. The input neural encoding eliminates pseudo-random number generators (PRNGs) for the previous encoding, and the output neural encoding reduces redundant memory access. As a result, weight can be reused by a delayed factor, while input spike encoding consumes 99.4% less power compared to the Poisson encoder. Also, SNPU uses computing encoding with multi-level encoding and spiketrain decomposition. The multi-level encoding reduces the time-window size by merging several spikes, and SNPU uses a single shift-and-accumulation (SAC) rather than several accumulations, reducing 43.5% of operations compared to single-level encoding. In addition, the spike-train decomposition is proposed to decrease the number of SACs by computation reuse, resulting in a reduction of 71.8% operations compared to previous spiking neural-network (SNN) processing, which cannot reuse computation result. As a result, the computing encoding reduces the operations by 83.3% compared to SNPU without multi-level encoding and spike-train decomposition. Moreover, SAC units in SNN PE process multi-level spikes while consuming 80.3% less power than accumulator arrays that do not use the two-step encoding. Also, the neuron link and spike-train allocator handle variable time-window sizes, and they can increase PE utilization of SNPU by 23.1%. To sum it up, SNPU shows the state-of-the-art energy efficiency of 13.7 TOPS/W for object detection and 31.5 TOPS/W for ImageNet classification. Moreover, SNPU shows the state-of-the-art power consumption of 63.2 $\mu$ W for face recognition.

15 October 2021 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Closed access
SKSanghoon Kang
Sanghoon Kang
GPGwangtae Park
Gwangtae Park
SKSangjin Kim
Sangjin Kim
SKSoyeon Kim
Soyeon Kim
DHDonghyeon Han
Donghyeon Han
HYHoi-Jun Yoo
Hoi-Jun Yoo
This paper presents a detailed overview of sparsity exploitation in deep neural network (DNN) accelerators. Despite the algorithmic advancements which drove DNNs to become the standard of artificial intelligence (AI), DNNs’ computational and memory overhead limits the deployment of off-the-shelf models on edge devices. Numerous optimizations have been widely studied to efficiently run DNN models on performance and energy-limited mobile devices, from both software’s and hardware’s perspective. Sparsity exploitation one of the mainstream optimization techniques, whose objective is to achieve higher efficiency and speed through avoiding redundant multiply-and-accumulate (MAC) operations cause by zero operands. This paper overviews previous contributions on sparsity exploitation from both the software and hardware side of views, with the newly proposed taxonomy to categorize and analyze the works. On the software side, different sparsification algorithms are explained, including pruning and output speculation. From the hardware’s perspective, advancements in architectures to efficiently handle sparse DNN computation are elaborated. The proposed taxonomy will help classify previous accelerators easily, by which sparsity they exploit and how. In addition, related works on sparse processing-in-memory (PIM) architectures and similarity exploitation are briefly introduced.

15 July 2021 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SKSoyeon Kim
Soyeon Kim
SKSanghoon Kang
Sanghoon Kang
DHDonghyeon Han
Donghyeon Han
SKSangjin Kim
Sangjin Kim
SKSangyeob Kim
Sangyeob Kim
HYHoi-Jun Yoo
Hoi-Jun Yoo
Generative adversarial networks (GANs) consist of multiple deep neural networks cooperating and competing with each other. Due to their complex architectures and large feature map sizes, training GANs requires a huge amount of computations. Moreover, instance normalization (IN) layers in GANs dramatically increase the external memory access (EMA). However, retraining GANs with user-specific data is critical on mobile devices because the pre-trained model outputs distorted images under user-specific conditions. This article proposes a GAN training accelerator to enable energy-efficient domain-specific optimization of GAN with user’s local data. Selective layer retraining (SELRET) picks out layers that are effective in enhancing the quality of the retrained model. Without image quality degradation, the SELRET reduces the required computation by 69%. Moreover, reordering layers for instance normalization (ROLIN) is proposed to reduce the EMA of intermediate data. Through the implementation of the proposed architecture, which splits and reorders the IN layers, 38.7% and 32.2% of overall EMA reduction are achieved in the forward propagation (FP) stage and the error propagation (EP) stage, respectively. The proposed processor is fabricated in a 65-nm CMOS process, showing 0.38-TFLOPS/W energy efficiency. The chip can retrain a face modification GAN with a custom dataset of 256 $\times $ 256 images over 100 epochs under 30 s while only consuming 274 mW. Compared to the previous FPGA implementation, this work improved the retraining performance and energy efficiency by 2 $\times $ and 39 $\times $ , respectively. As a result, the proposed accelerator enables GAN’s domain-specific optimization on a mobile platform.