15 January 2018 in IEEE Transactions on Automatic Control
This paper presents a novel approach to synthesize supremal nonblocking supervisors of discrete-event systems (DES), when the automaton models of specifications are not available. Extending the $L^*$ learning algorithm, an $S^*$ algorithm is developed to infer a tentatively correct supervisor. If the tentatively correct supervisor is nonblocking, it is indeed the supremal nonblocking supervisor with respect to the plant and specifications. Otherwise, the blocking automaton is regarded as a new plant, and the specification is the nonblocking property. Then, the supremal nonblocking supervisor with respect to the new problem is computed using supervisory control theory of DES. Two simplification rules are introduced to the $S^*$ algorithm to decrease the computational cost. Finally, the $S^*$ algorithm is implemented based on the LearnLib framework, and experiments are performed to verify the proposed approach.
7 August 2019 in IEEE Transactions on Automation Science and Engineering
The paper presents an approach to the control of black-box embedded systems by integrating automaton learning and supervisory control theory (SCT) of discrete-event systems (DES), where automaton models of both the system and requirements are unavailable or hard to obtain. First, the system is tested against the requirements. If all the requirements are satisfied, no supervisor is needed and the process terminates. Otherwise, a supervisor is synthesized to enforce the system to satisfy the requirements. To apply SCT and automaton learning technologies efficiently, the system is abstracted to be a finite-discrete model. Then, a C* learning algorithm is proposed based on the classical L* algorithm to infer a Moore automaton describing both the behavior of the system and the conjunctive behavior of the system and the requirements. Subsequently, a supervisor for the system is derived from the learned Moore automaton and patched on the system. Finally, the controlled system is tested again to check the correctness of the supervisor. If the requirements are still not satisfied, a larger Moore automaton is learned and a refined supervisor is synthesized. The whole process iterates until the requirements hold in the controlled system. The effectiveness of the proposed approach is manifested through two realistic case studies.
6 May 2024 in IEEE Transactions on Knowledge and Data Engineering
This study is devoted to the design of gradient boosted fuzzy rule-based models for regression problems. Fuzzy rule-based models are built on the basis of information granules formed in the input and output spaces whose structure involves a family of conditional ‘if-then’ statements. The architecture of fuzzy rule-based models contributes to the realization of a sound tradeoff between modeling accuracy and interpretability and computing overhead. Gradient boosting paradigm has emerged as a powerful learning method realized through sequentially fitting additive base learners to current residuals in the steepest descent way. However, surprisingly, studies on the design and analysis of gradient boosted fuzzy rule-based models are still lacking. In this study, fuzzy rule-based model is regarded as a base learner. Different loss functions and their influence on the performance of the final models are explored. We also thoroughly investigate an impact of the initial quality of the rule-based model (implied by the number of rules) on the process of gradient boosting. The performance of the proposed approach is illustrated by a series of experimental studies concerning synthetic and publicly available datasets.
17 September 2024 in IEEE Transactions on Automation Science and Engineering
In the control of discrete-event systems for current-state opacity enforcement, it is difficult to synthesize a supervisor by supervisory control theory (SCT) without explicit formal models of the systems. This study utilizes the reinforcement learning (RL) method to obtain supervisory policies for opacity enforcement in the case when the automaton model of the system is unavailable. The state space of the environment in the RL is dynamically generated through system simulation. Actions are defined according to the control patterns of the SCT. A reward function is proposed to evaluate whether the secret is exposed or not. Then, a sequence of state-action-reward chains are obtained as system simulation goes on. The frameworks of Q-learning and State-Action-Reward-State-Action (SARSA) algorithms are adopted to implement the proposed approach. The goal of the training is to maximize the total accumulative reward by optimizing the action selection in the learning process. Then, an optimal supervisory policy is obtained when the training process converges. Experiments are performed to illustrate the effectiveness of the proposed approach. The contributions are two aspects. Firstly, a supervisor for opacity enforcement is learned by RL training without an explicit formal model of the system. Secondly, the ability of the proposed method in computing supervisory policies without formal models addresses a significant gap in the literature and offers a new direction for research in opacity enforcement in discrete event systems. Note to Practitioners —Supervisory Control Theory (SCT) supplies an effective way to synthesize supervisors, which traditionally handles tasks with explicit system models for current-state opacity enforcement by restricting behavior of systems. However, formal models of systems are often confidential or otherwise unavailable. This paper presents a method for supervisor synthesis via reinforcement learning in the case of lacking formal models of systems. The proposed method leverages the characteristics of control patterns in SCT and optimizes the action selection in the training process through a reward mechanism that evaluates the secrecy of states. The approach can be applied to model-free RL frameworks such as Q-learning and State-Action-Reward-State-Action (SARSA) algorithms. The training is performed as the system simulation goes on. When the training process converges, the optimal policy can be used to enforce opacity for the system. However, Q-table is used to save the Q-value in both Q-learning and SARSA algorithms. In the worst case, the size of the Q-table grows exponentially with the number of states and controllable events increasing. This can lead to memory exhaustion when the system’s scale is large. To make the approach scalable, we will attempt to use DRL to train control policies for opacity enforcement in the future.
1 June 2021 in Journal of Systems Science and Information
Blockchain technology plays a very positive role in promoting the development of the port supply chain. Although there are some practical examples of blockchain in the port supply chain (eg. Trade Len or Cargo Smart), there are few application scenarios. Therefore, blockchain technology has yet to be widely used in the port industry. This may be related to the construction of blockchain. This paper analyzes the impact of blockchain technology on the port supply chain and the technology sharing on the shipping market. It is found that the investment of blockchain technology in a competitive environment has different characteristics. When the efficiency of technology investment and the proportion of market expansion reach a certain combination, there are two different equilibrium strategy choices: Only technology investment or technology sharing. Based on the above research results, this paper further investigates the results of the comprehensive influence of different factors such as competition intensity and market expansion proportion.