Ryeonggu Kwon

Dr. Ryeonggu Kwon

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 November 2023 in Systems
Verified
Journal Article
Open access
PDF file
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
In the context of reinforcement learning (RL), ensuring both safety and performance is crucial, especially in real-world scenarios where mistakes can lead to severe consequences. This study aims to address this challenge by integrating temporal logic constraints into RL algorithms, thereby providing a formal mechanism for safety verification. We employ a combination of theoretical and empirical methods, including the use of temporal logic for formal verification and extensive simulations to validate our approach. Our results demonstrate that the proposed method not only maintains high levels of safety but also achieves comparable performance to traditional RL algorithms. Importantly, our approach fills a critical gap in the existing literature by offering a solution that is both mathematically rigorous and empirically validated. The study concludes that the integration of temporal logic into RL offers a promising avenue for developing algorithms that are both safe and efficient. This work lays the foundation for future research aimed at generalizing this approach to various complex systems and applications.

1 March 2023 in Scientific Journal of Gdynia Maritime University
Verified
Journal Article
Open access
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
As technology advances, hardware-centric systems are rapidly moving towards software-centric ones, and their complexity is rapidly increasing. In particular, systems directly related to safety require thorough verification. Model checking exhaustively explores the state space of the abstracted system to check whether properties written in a logical formula are achieved. In this paper, the control algorithm of the controller is verified using model checking to discover risk scenarios during the STPA steps. Two case studies are conducted using the widely used model checkers NuSMV and UPPAAL. We then explain the empirical results and compare two model checkers based on their characteristics. Finally, we discuss the benefits of applying model checking in the process of STPA.

10 February 2025 in IEEE Access
Verified
Journal Article
Open access
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
Ensuring the safety of Deep Reinforcement Learning (DRL) systems remains a significant challenge, particularly in real-time applications such as autonomous driving and robotics, where incorrect decisions can lead to catastrophic failures. This study proposes a novel safety verification framework that combines state abstraction with probabilistic model checking to quantitatively analyze failure risks. The continuous state space is clustered using the KMeans++ algorithm, enabling efficient state space reduction. A Discrete-Time Markov Chain (DTMC) model is then constructed for each cluster, capturing probabilistic transitions between abstracted states. The PRISM model checker is employed to verify failure probabilities and invariance properties, providing a rigorous quantitative evaluation of system safety. Counterexample analysis identifies critical failure paths, offering actionable insights for policy improvement. Experimental results demonstrate that the optimal number of clusters balances state space reduction with accurate failure analysis, enabling scalable verification. By leveraging model checking within an abstracted state space, this approach enhances the reliability and safety of DRL systems and establishes a pathway for their deployment in safety-critical domains.

27 March 2024 in Applied Sciences
Verified
Journal Article
Open access
PDF file
SPSohee Park
Sohee Park
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
In modern software development, OSS (Open Source Software) has become a crucial element. However, if OSS have few contributors and are lacking in maintenance activities, such as bug fixes, are used, it can lead to significant costs and resource allocation due to maintenance discontinuation. Since OSS are developed by a diverse group of contributors, the consistency of their involvement may vary, making continuous support and maintenance unpredictable. Therefore, it is necessary to identify the status of each OSS to avoid increased maintenance costs when selecting OSS for use. To address these issues, we use polynomial regression to predict trends in bug-fixing activities and evaluate the survivability of OSS accordingly. We predict the trend of bug-fixing activities in OSS, using factors such as popularity, number of contributors, and code complexity. A lower trend value indicates more vigorous activity. In this paper, we conduct data collection and extraction, generating model, and model testing and evaluation to predict survivability using these data. After collecting data through various tools, the models of different degrees are generated using K-fold cross-validation. The model with the highest performance is selected based on the RMSE (Root Mean Squared Error) and RSE (Residual Standard Error). Then, the chosen model is employed to predict the survivability of OSS and compare it with actual outcomes. This method is experimented with on OSS used in the KakaoTalk commercial messenger app. As a result, several OSS are predicted to have low survivability, and among them, five are analyzed. It is observed that, in reality, activities such as delayed or discontinued release updates occurred. These findings can support OSS users in selecting OSS for cost-saving purposes and alert OSS administrators to the need for solutions to ensure project survival.

29 March 2024 in Applied Sciences
Verified
Journal Article
Open access
PDF file
JCJiyoung Chang
Jiyoung Chang
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
Experience-based methods like reinforcement learning (RL) are often deemed less suitable for the safety field due to concerns about potential safety issues. To bridge this gap, we introduce STPA-RL, a methodology that integrates RL with System-Theoretic Process Analysis (STPA). STPA is a safety analysis technique that identifies causative factors leading to unsafe control actions and system hazards through loss scenarios. In the context of STPA-RL, we formalize the Markov Decision Process based on STPA analysis results to incorporate control algorithms into the system environment. The agent learns safe actions through reward-based learning, tracking potential hazard paths to validate system safety. Specifically, by analyzing various loss scenarios related to the Platform Screen Door, we assess the applicability of the proposed approach by evaluating hazard trajectory graphs and hazard frequencies in the system. This paper streamlines the RL process for loss scenario identification through STPA, contributing to self-guided loss scenarios and diverse system modeling. Additionally, it offers effective simulations for proactive development to enhance system safety and provide practical assistance in the safety field.

25 January 2024 in IEEE Access
Verified
Journal Article
Open access
RKRyeonggu Kwon
Ryeonggu Kwon
GKGihwon Kwon
Gihwon Kwon
SPSohee Park
Sohee Park
JCJiyoung Chang
Jiyoung Chang
SJSuhee Jo
Suhee Jo
Reinforcement learning (RL) is rapidly used in safety-centric applications. However, many studies focus on generating optimal policy that achieves maximum rewards. While maximum rewards are beneficial, safety constraints and non-functional requirements must also be considered in safety-centric applications to avoid dangerous situations. For example, in the case of food delivery robots in restaurants, RL should be used not only to find optimal policy that response to all customer requests through maximum rewards but also to consider safety constraints such as collision avoidance and non-functional requirements such as battery saving. In this paper, we investigated the fulfillment of safety constraints and non-functional requirements of learning models generated through RL with quantitative model checking. We experimented with various time steps and learning rates required for RL, targeting restaurant delivery robots. The functional requirement of these robots is to process all customer order requests, and the non-functional requirements are the number of steps and battery consumption to complete the task. Safety constraints include the amount of collision and the probability of collision. Through these experiments, we made three important findings. First, learning models that obtain maximum rewards may have a low degree of achievement of non-functional requirements and safety constraints. Second, as safety constraints are met, the degree of achievement of non-functional requirements may be low. Third, even if the maximum reward is not obtained, sacrificing non-functional requirements can maximize the achievement of safety constraints. These results show that learning models generated through RL can trade off rewards to achieve safety constraints. In conclusion, our work can contribute to selecting suitable hyperparameters and optimal learning models during RL.