The use of AI-based risk mitigation is increasing to provide safety in the areas of smart manufacturing, automated logistics etc, where the human-robot collaboration operations are in use. This paper presents our work on implementation of fuzzy logic system (FLS) and reinforcement learning (RL) to build risk mitigation modules for human-robot collaboration scenarios. Risk mitigation using FLS strategy is developed by manually defining the linguistic values, tuning the membership functions and generating the rules based on ISO/TS15066:2016. RL-based risk mitigation modules are developed using three different Qnetworks to estimate the Q-value function. Our purpose is twofold: to perform a comparative analysis of FLS and RL in terms of safety perspectives and further to evaluate the efficiency to accomplish the task. Our results present that all the proposed risk mitigation strategies improve the safety aspect by up to 26% as compared to a default setup where the robot is just relying on a navigation module without risk mitigation. The efficiency of using FLS model is maintained to the default setup, while the efficiency of using RL model is reduced by 26% from the default setup. We also compare the computation performance of risk mitigation between centralized and edge execution where the edge execution is 27.5 times faster than the centralized one.

Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and decisions made by the agent, different eXplainable RL (XRL) methods are developed; for example, feature importance methods are applied to analyze the contribution of the input side of the model, and reward decomposition methods are applied to explain the components of the output end of the RL model. In this study, we present a novel method to connect explanations from both input and output ends of a black-box model, which results in fine-grained explanations. Our method exposes the reward prioritization to the user, which in turn generates two different levels of explanation and allows RL agent reconfigurations when unwanted behaviors are observed. The method further summarizes the detailed explanations into a focus value that takes into account all reward components and quantifies the fulfillment of the explanation of desired properties. We evaluated our method by applying it to a remote electrical telecom-antenna-tilt use case and two openAI gym environments: lunar lander and cartpole. The results demonstrated fine-grained explanations by detailing input features’ contributions to certain rewards and revealed biases of the reward components, which are then addressed by adjusting the reward’s weights.

Generating detailed explanations that are easy to comprehend and interact with is a challenging problem for complex Reinforcement Learning (RL) agents. While various methods explain different aspects of the agents, it is difficult to aggregate and generate tailored insights for different users. Thus, we propose a comprehensive explainability approach that utilizes interactive natural language queries and generates different types of explanations. First, we introduce a new approach to generate meaningful counterfactual explanations using natural language queries. Further, we complement the natural language explanations with customized feature attributions for detailed insights. This helps in facilitating the interaction with explanations as well as tailoring the explanations for different purposes and levels of expertise. We demonstrate our proposal using an industrial telecommunication use case which shows its applicability and utility in a complex real-world scenario.

Understanding a Reinforcement Learning (RL) policy is crucial for ensuring that autonomous agents behave according to human expectations. This goal can be achieved using Explainable Reinforcement Learning (XRL) techniques. Although textual explanations are easily understood by humans, ensuring their correctness remains a challenge, and evaluations in state-of-the-art remain limited. We present a novel XRL framework for generating textual explanations, converting them into a set of transparent rules, improving their quality, and evaluating them. Expert's knowledge can be incorporated into this framework, and an automatic predicate generator is also proposed to determine the semantic information of a state. Textual explanations are generated using a Large Language Model (LLM) and a clustering technique to identify frequent conditions. These conditions are then converted into rules to evaluate their properties, fidelity, and performance in the deployed environment. Two refinement techniques are proposed to improve the quality of explanations and reduce conflicting information. Experiments were conducted in three open-source environments to enable reproducibility, and in a telecom use case to evaluate the industrial applicability of the proposed XRL framework. This framework addresses the limitations of an existing method, Autonomous Policy Explanation, and the generated transparent rules can achieve satisfactory performance on certain tasks. This framework also enables a systematic and quantitative evaluation of textual explanations, providing valuable insights for the XRL field.