This article addresses the online reinforcement Q-learning algorithms to design H∞ tracking controller for unknown discrete-time linear systems. An augmented system composed of the original system and the command generator is constructed, and a discounted performance function is introduced to establish a discounted game algebraic Riccati equation (GARE). The existence conditions of a solution to the GARE are proposed and a lower bound is found for the discount factor to assure the stability of the H∞ tracking control solution. The Q-function Bellman equation is then derived, based on which the reinforcement Q-learning algorithm is developed to learn the solution to H∞ tracking control problem without knowing the system dynamics. Both state-data-driven and output-data-driven reinforcement Q-learning algorithms toward finding the control policies are proposed. Unlike the value function approximation (VFA)-based approach, it is proved that the Q-learning scheme brings out no bias of solution to the Q-function Bellman equation under the probing noise satisfying the persistent excitation (PE) condition, and therefore, converges to the nominal discounted GARE solution. Moreover, the proposed output-data-driven method is more powerful than the state-data-driven method as it may not be available to completely measure the full system states in practical applications. A simulation example with a single-phase voltage-source UPS inverter is used to verify the effectiveness of the proposed Q-learning algorithms.

This paper investigates output voltage tracking control and harmonic distortion suppression in single-phase voltage-source uninterruptible power supply (UPS) inverters with an LC-filter. The control scheme is based on sensing the output voltage across the UPS load and using the voltage tracking error in a feedback loop of the internal model principle (IMP) framework. A robust controller is designed to drive LC-filter to operate with an adjustable amplitude and adaptive resonant frequency. Specifically, the proposed controller basically employs an internal model with a filter to examine the incremental dynamics of the voltage tracking error and regulate the voltage modulating signal in sine pulse width modulation (SPWM). With such a stable operation of the controlled UPS system, the inverter is able to achieve fast dynamics of voltage tracking, which results in the minimum harmonics while approaching the zero-error steady state under the external loads. The Matlab/Simulink simulation and experimental validation are both provided for the prototype controlled UPS inverter under the closed-loop operation. It is shown that the controller offers improved performance over the existing schemes. By merely using the voltage error to adjust the output voltage, the controlled inverter produces nearly perfect sinusoidal voltage at moderate switching frequency.

The micromirror based on micro-electro-mechanical systems (MEMS) technology is widely employed in different areas, such as scanning, imaging and optical switching. This paper studies the MEMS electromagnetic micromirror for scanning or imaging application. In these application scenarios, the micromirror is required to track the command sinusoidal signal, which can be converted to an output regulation problem theoretically. In this paper, based on the internal model principle, the output regulation problem is solved by designing a robust controller that is able to force the micromirror to track the command signal accurately. The proposed controller relies little on the accuracy of the model. Further, the proposed controller is implemented, and its effectiveness is examined by experiments. The experimental results demonstrate that the performance of the proposed controller is satisfying.