Summary: Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach.
1 September 2021 in Machine Learning with Applications
The ability to predict the onset of labour is seen to be an important tool in a clinical setting. Magnetomyography has shown promise in the area of labour imminency prediction, but its clinical application remains limited due to high resource consumption associated with its broad number of channels. In this study, five electrode channels, which account for 3.3% of the total, are used alongside a novel signal decomposition algorithm and low complexity classifiers (logistic regression and linear-SVM) to classify between labour imminency due within 0–48hrs and >48hrs. The results suggest that the parsimonious representation comprising of five electrode channels and novel signal decomposition method alongside the candidate classifiers could allow for greater affordability and hence clinical viability of the magnetomyography-based prediction model, which carries a good degree of model interpretability.
To date, effective means of predicting pregnancy labor continues to lack. Magnetic field signals during uterine contraction have shown, in recent studies, to be a good source of information for predicting labor state with a greater accuracy compared with existing methods. The means of labor prediction methods from such signals appear to rely on a supervised learning post-processing framework whose calibration relies on an effective labelling of the training sample set. As a potential solution to this, using a reduced electrode channel from magnetomyography instrumentation, we propose a multi-stage self-sorting cybernetic model that is comprised of an ensemble of various post-processing methods, and is underpinned by an unsupervised learning framework that allows for an automated method towards learning from the trend in the data to infer labor state and imminency. Experimental results showed a comparable accuracy with those from a supervised learning method adopted in a prior study. Additionally, an architecture of how an intelligent cybernetic model can be used for labor prediction and cost saving benefits within a clinical setting is offered by this study.