Junkyu Lee

Junkyu Lee

Auto-generated profile, based on publicly available information

Junkyu Lee

Institute for Analytics and Data Science, University of Essex, Colchester, U.K.

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 July 2023 in ACM Computing Surveys
Verified
Journal Article
Open access
JLJunKyu Lee
JunKyu Lee
LMLev Mukhanov
Lev Mukhanov
AMAmir Sabbagh Molahosseini
Amir Sabbagh Molahosseini
UMUmar Minhas
Umar Minhas
YHYang Hua
Yang Hua
HVHans Vandierendonck
Hans Vandierendonck
Convolutional neural networks (CNNs) are used in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, and face recognition, among others. However, deep CNNs demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of CNNs, whereas the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient CNN techniques in terms of model-, arithmetic-, and implementation-level techniques, and identifies the research gaps for resource-efficient CNN techniques across the three different level techniques. Our survey clarifies the influence from higher- to lower-level techniques based on our resource efficiency metric definition and discusses the future trend for resource-efficient CNN research.

4 January 2024 in IEEE Access
Verified
Journal Article
Open access
NGNil Llisterri Giménez
Nil Llisterri Giménez
JLJunkyu Lee
Junkyu Lee
FFFelix Freitag
Felix Freitag
HVHans Vandierendonck
Hans Vandierendonck
Many weight quantization approaches were explored to save the communication bandwidth between the clients and the server in federated learning using high-end computing machines. However, there is a lack of weight quantization research for online federated learning using TinyML devices which are restricted by the mini-batch size, the neural network size, and the communication method due to their severe hardware resource constraints and power budgets. We name Tiny Online Federated Learning (TinyOFL) for online federated learning using TinyML devices in the Internet of Things (IoT). This paper performs a comprehensive analysis of the effects of weight quantization in TinyOFL in terms of accuracy, stability, overfitting, communication efficiency, energy consumption, and delivery time, and extracts practical guidelines on how to apply the weight quantization to TinyOFL. Our analysis is supported by a TinyOFL case study with three Arduino Portenta H7 boards running federated learning clients for a keyword spotting task. Our findings include that in TinyOFL, a more aggressive weight quantization can be allowed than in online learning without FL, without affecting the accuracy thanks to TinyOFL’s quasi-batch training property. For example, using 7-bit weights achieved the equivalent accuracy to 32-bit floating point weights, while saving communication bandwidth by $4.6 \times $ . Overfitting by increasing network width rarely occurs in TinyOFL, but may occur if strong weight quantization is applied. The experiments also showed that there is a design space for TinyOFL applications by compensating for the accuracy loss due to weight quantization with an increase of the neural network size.

25 June 2018 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Closed access
JLJun Kyu Lee
Jun Kyu Lee
HVHans Vandierendonck
Hans Vandierendonck
MAMahwish Arif
Mahwish Arif
GPGregory D. Peterson
Gregory D. Peterson
DNDimitrios Nikolopoulos
Dimitrios Nikolopoulos
Mixed precision is a promising approach to save energy in iterative refinement algorithms since it obtains speedup without necessitating additional cores and parallelisation. However, conventional mixed precision methods utilise statically defined precision in a loop, thus hindering further speed-up and energy savings. We overcome this problem by proposing novel methods which allow iterative refinement to utilise variable precision arithmetic dynamically in a loop (i.e. a trans-precision approach). Our methods restructure a numeric algorithm dynamically according to runtime numeric behaviour and remove unnecessary accuracy checks. We implemented our methods by extending one conventional mixed precision iterative refinement algorithm on an Intel Xeon E5-2650 2GHz core with MKL 2017 and XBLAS 1.0. Our dynamic precision approach demonstrates 2.0–2.6× speed-up and 1.8–2.4× energy savings compared to mixed precision iterative refinement when double precision solution accuracy is required for forward error and with matrix dimensions ranging from 4K to 32K.

1 September 2020 in Parallel Computing
Verified
Journal Article
Closed access
JLJunKyu Lee
JunKyu Lee
GPGregory D. Peterson
Gregory D. Peterson
DNDimitrios S. Nikolopoulos
Dimitrios S. Nikolopoulos
HVHans Vandierendonck
Hans Vandierendonck

16 December 2020 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
JLJunKyu Lee
JunKyu Lee
DNDimitrios Nikolopoulos
Dimitrios Nikolopoulos
HVHans Vandierendonck
Hans Vandierendonck
Kernel recursive least squares (KRLS) is a widely used online machine learning algorithm for time series predictions. In this article, we present the mixed-precision KRLS, producing equivalent prediction accuracy to double-precision KRLS with a higher training throughput and a lower memory footprint. The mixed-precision KRLS applies single-precision arithmetic to the computation components being not only numerically resilient but also computationally intensive. Our mixed-precision KRLS demonstrates the 1.32, 1.15, 1.29, 1.09, and 1.08x training throughput improvements using 24.95%, 24.74%, 24.89%, 24.48%, and 24.20% less memory footprint without losing any prediction accuracy compared to double-precision KRLS for a 3-D nonlinear regression, a Lorenz chaotic time series, a Mackey-Glass chaotic time series, a sunspot number time series, and a sea surface temperature time series, respectively.

12 February 2025 in Internet of Things
Verified
Journal Article
Closed access
JPJong-Ik Park
Jong-Ik Park
SSSihoon Seong
Sihoon Seong
JLJunKyu Lee
JunKyu Lee
CHCheol-Ho Hong
Cheol-Ho Hong