22 July 2024 in IEEE Transactions on Cybernetics
Policy iteration (PI), an iterative method in reinforcement learning, has the merit of interactions with a little-known environment to learn a decision law through policy evaluation and improvement. However, the existing PI-based results for output-feedback (OPFB) continuous-time systems relied heavily on an initial stabilizing full state-feedback (FSFB) policy. It thus raises the question of violating the OPFB principle. This article addresses such a question and establishes the PI under an initial stabilizing OPFB policy. We prove that an off-policy Bellman equation can transform any OPFB policy into an FSFB policy. Based on this transformation property, we revise the traditional PI by appending an additional iteration, which turns out to be efficient in approximating the optimal control under the initial OPFB policy. We show the effectiveness of the proposed learning methods through theoretical analysis and a case study.
17 October 2025 in Neural Networks
8 May 2026 in IEEE Transactions on Automation Science and Engineering
This work studies model-free H∞ policy design for a target tracking system with network-induced dropout and unknown system dynamics. By integrating reinforcement learning, robust output regulation theory, and state reconstruction, we address the target tracking problem while ensuring resilience to network-induced dropout. We show that this problem can be decomposed into two learning-based tasks: predicting the system state under packet dropout in the sensor/controller transmission channels, and designing an off-policy H∞ control law to reduce the impact of disturbances on system performance. On this basis, we develop a state-reconstruction technique that integrates historical system data into a Smith predictor to estimate the tracking-system state and improve resilience to network-induced dropout. Furthermore, we propose a robust output-regulation design with H∞ performance to control the target tracker effectively, even without prior knowledge of the system dynamics. Through extensive numerical simulations and hardware-in-the-loop (HIL) experiments, we validate the effectiveness of the proposed algorithms for robust target tracking under network dropout.
21 November 2024 in Unmanned Systems