Ju-Bong Kim

Mr. Ju-Bong Kim

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 March 2020 in Sensors
Verified
Journal Article
Open access
PDF file
HLHyun-Kyo Lim
Hyun-Kyo Lim
JKJu-Bong Kim
Ju-Bong Kim
JHJoo-Seong Heo
Joo-Seong Heo
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
Reinforcement learning has recently been studied in various fields and also used to optimally control IoT devices supporting the expansion of Internet connection beyond the usual standard devices. In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own IoT devices of the same type but with slightly different dynamics. For such multiple IoT devices, there is no guarantee that an agent who interacts only with one IoT device and learns the optimal control policy will also control another IoT device well. Therefore, we may need to apply independent reinforcement learning to each IoT device individually, which requires a costly or time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent IoT device shares their learning experience (i.e., the gradient of loss function) with each other, and transfers a mature policy model parameters into other agents. They accelerate its learning process by using mature parameters. We incorporate the actor–critic proximal policy optimization (Actor–Critic PPO) algorithm into each agent in the proposed collaborative architecture and propose an efficient procedure for the gradient sharing and the model transfer. Using multiple rotary inverted pendulum devices interconnected via a network switch, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple IoT devices and that the learning speed can be faster if more agents are involved.

21 June 2019 in Applied Sciences
Verified
Journal Article
Open access
PDF file
HLHyun-Kyo Lim
Hyun-Kyo Lim
JKJu-Bong Kim
Ju-Bong Kim
KKKwihoon Kim
Kwihoon Kim
YHYongGeun Hong
YongGeun Hong
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
Recently, with the advent of various Internet of Things (IoT) applications, a massive amount of network traffic is being generated. A network operator must provide different quality of service, according to the service provided by each application. Toward this end, many studies have investigated how to classify various types of application network traffic accurately. Especially, since many applications use temporary or dynamic IP or Port numbers in the IoT environment, only payload-based network traffic classification technology is more suitable than the classification using the packet header information as well as payload. Furthermore, to automatically respond to various applications, it is necessary to classify traffic using deep learning without the network operator intervention. In this study, we propose a traffic classification scheme using a deep learning model in software defined networks. We generate flow-based payload datasets through our own network traffic pre-processing, and train two deep learning models: 1) the multi-layer long short-term memory (LSTM) model and 2) the combination of convolutional neural network and single-layer LSTM models, to perform network traffic classification. We also execute a model tuning procedure to find the optimal hyper-parameters of the two deep learning models. Lastly, we analyze the network traffic classification performance on the basis of the F1-score for the two deep learning models, and show the superiority of the multi-layer LSTM model for network packet classification.

24 May 2021 in IEEE Access
Verified
Journal Article
Open access
HLHyun-Kyo Lim
Hyun-Kyo Lim
JKJu-Bong Kim
Ju-Bong Kim
IUIhsan Ullah
Ihsan Ullah
JHJoo-Seong Heo
Joo-Seong Heo
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
Nowadays, Reinforcement Learning (RL) is applied to various real-world tasks and attracts much attention in the fields of games, robotics, and autonomous driving. It is very challenging and devices overwhelming to directly apply RL to real-world environments. Due to the reality gap simulated environment does not match perfectly to the real-world scenario and additional learning cannot be performed. Therefore, an efficient approach is required for RL to find an optimal control policy and get better learning efficacy. In this paper, we propose federated reinforcement learning based on multi agent environment which applying a new federation policy. The new federation policy allows multi agents to perform learning and share their learning experiences with each other e.g., gradient and model parameters to increase their learning level. The Actor-Critic PPO algorithm is used with four types of RL simulation environments, OpenAI Gym's CartPole, MoutainCar, Acrobot, and Pendulum. In addition, we did real experiments with multiple Rotary Inverted Pendulum (RIP) to evaluate and compare the learning efficiency of the proposed scheme with both environments.

18 March 2019 in IEEE Access
Verified
Journal Article
Open access
JKJu-Bong Kim
Ju-Bong Kim
HLHyun-Kyo Lim
Hyun-Kyo Lim
CKChan-Myung Kim
Chan-Myung Kim
MKMin-Suk Kim
Min-Suk Kim
YHYong-Geun Hong
Yong-Geun Hong
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
Rotary inverted pendulum is an unstable and highly nonlinear device, and has been used as a common application model in nonlinear control engineering field. In this paper, we use a rotary inverted pendulum as a deep reinforcement learning environment. The real device is composed of a cyber environment and physical environment based on OpenFlow network, and the MQTT protocol is used on the Ethernet connection to connect the cyber environment and the physical environment. The reinforcement learning agent is learned to control the real device located remotely from the controller, and the classical PID controller is also utilized to implement the imitation reinforcement learning, and facilitate the learning process. From our CPS-based experimental system, we verify that a deep reinforcement learning agent can successfully control the real device located remotely from the agent, and our imitation learning strategy can make the learning time reduced effectively.

14 September 2022 in IEEE Access
Verified
Journal Article
Open access
HCHo-Bin Choi
Ho-Bin Choi
JKJu-Bong Kim
Ju-Bong Kim
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
SOSe-Won Oh
Se-Won Oh
KKKwihoon Kim
Kwihoon Kim
Automated guided vehicles (AGVs) are essential components for the automation of fulfillment centers, a type of warehouse, where goods are stored on shelves and carried by AGVs. To increase the productivity in inventory management, a well-organized cooperative path control is required to transport goods to the designated picking stations. In this paper, we propose a QMIX-based scheme for the cooperative path control of multiple AGVs. Although QMIX is the one of popular cooperative multi-agent reinforcement learning algorithms, we find that QMIX alone was not enough to increase productivity in warehouse systems. So, we develop two novel techniques that can be used with QMIX: 1) sequential action masking that eliminates all the collision cases and 2) additional local loss that improves collaboration of individual AGVs. They help to encourage the AGVs to cooperate more for high productivity. By extensive simulations, we present the superiority of the proposed scheme on several layouts in fulfillment centers. The effect of cooperation among AGVs in the proposed scheme is verified through the comparison study with the existing algorithms. Additionally, we show the generalization performance by investigating the reusability of the model trained with the proposed scheme.

16 June 2020 in Sensors
Verified
Journal Article
Open access
PDF file
JKJu-Bong Kim
Ju-Bong Kim
HCHo-Bin Choi
Ho-Bin Choi
GHGyu-Young Hwang
Gyu-Young Hwang
KKKwihoon Kim
Kwihoon Kim
YHYongGeun Hong
YongGeun Hong
Youn-Hee HanYoun-Hee Han
Youn-Hee Han
Intralogistics is a technology that optimizes, integrates, automates, and manages the logistics flow of goods within a logistics transportation and sortation center. As the demand for parcel transportation increases, many sortation systems have been developed. In general, the goal of sortation systems is to route (or sort) parcels correctly and quickly. We design an n-grid sortation system that can be flexibly deployed and used at intralogistics warehouse and develop a collaborative multi-agent reinforcement learning (RL) algorithm to control the behavior of emitters or sorters in the system. We present two types of RL agents, emission agents and routing agents, and they are trained to achieve the given sortation goals together. For the verification of the proposed system and algorithm, we implement them in a full-fledged cyber-physical system simulator and describe the RL agents’ learning performance. From the learning results, we present that the well-trained collaborative RL agents can optimize their performance effectively. In particular, the routing agents finally learn to route the parcels through their optimal paths, while the emission agents finally learn to balance the inflow and outflow of parcels.