Even though the unprecedented success of AlphaGo Zero demonstrated reinforcement learning as a feasible complex problem solver, the research on reinforcement learning control of hydraulic systems is still void. We are motivated by the challenges presented in hydraulic systems to develop a new model-based reinforcement learning controller that achieves high-accuracy tracking at performance level and with asymptotic stability guarantees at system level. In this article, the proposed design consists of two frameworks: A recursive robust integral of the sign of the error (RISE) control approach to providing closed-loop system stability framework, and a reinforcement learning approach with actor-critic structure to dealing with the unknown dynamics or more specifically, the actor neural network is used to reduce the high feedback gain of the recursive RISE control approach by compensating the unknown dynamic while the critic neural network is integrated to improve the control performance by evaluating the filtered tracking error. A theoretical guarantee for the stability of the overall dynamic system is provided by using Lyapunov stability theory. Simulation and experimental results are provided to demonstrate improved control performance of the proposed controller.

Depth cameras have recently become popular and many vision problems can be better solved with depth information. But how to integrate depth information into a visual tracker to overcome the challenges such as occlusion and background distraction is still under-investigated in current literatures of visual tracking. In this paper, we investigate a 3D extension of classical mean-shift tracker whose greedy gradient ascend strategy is generally considered as unreliable in conventional 2D tracking. However, through careful study of the physical property of 3D point clouds, we reveal that objects which may appear to be adjacent on 2D image will form distinctive modes in the 3D probability distribution approximated by kernel density estimation, and finding the nearest mode using 3D mean-shift can always work in tracking. Based on the understanding of 3D mean-shift, we propose two important mechanisms to further boost the tracker's robustness: one is to enable the tracker be aware of potential distractions and make corresponding adjustments to the appearance model; and the other is to enable the tracker to be able to detect and recover from tracking failures caused by total occlusion. The proposed method is both effective and computationally efficient. On a conventional PC, it runs at more than 60 FPS without GPU acceleration.

Motivated by the challenges inherent in achieving high-accuracy tracking control of practical 6-degree-of-freedom (6-DOF) hydraulic robotic manipulators, we aim to conduct research on data-driven control methods in this article. To this end, we introduce actor-critic reinforcement learning to learn the value functions and their corresponding control actions along the trajectories associated with the tracking pattern evolution in hydraulic robotic manipulators. Furthermore, we extend our investigation to encompass the provision of performance guarantees at the system level, even in instances where value functions and control actions are approximated through the utilization of actor-critic neural networks. As proved in this article, the proposed reinforcement learning controller possesses pivotal properties, such as Bellman (sub)optimality in solutions, and the convergence of approximated value functions. We evaluated the proposed reinforcement learning controller in a well-established 6-DOF hydraulic robotic manipulator platform. The experimental results attest to the consistent improvement in control performance, progressively approaching the desired performance benchmarks through iterative updates of control actions. This study makes the first attempt to explore the realization of online adaptive optimal control for hydraulic robotic manipulators within a data-driven paradigm, an area where limited work has been reported, and also sets an example of how reinforcement learning interfaces with industrial automation.

Different from image inpainting, image outpainting has relatively less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (SiENet) is proposed. Specifically, in two stages, a novel border sensitive convolution named adaptive filling convolution is designed for allowing encoder to predict the unknown content, alleviating the burden of decoder. Besides, to introduce prior knowledge to network and reinforce the inferring ability of encoder, siamese adversarial mechanism is designed to enable our network to model the distribution of covered long range feature as that of uncovered image feature. The results on four datasets has demonstrated that our method outperforms existing state-of-the-arts and could produce realistic results. Our code is released on https://github.com/nanjingxiaobawang/SieNet-Image-extrapolation

Bifurcation control remains largely unresolved for fractional-order dynamical systems. This article addresses the optimal control issue of bifurcation for a delayed complex networks model with Caputo derivative, where the time delay is selected as the variable parameter. We first devise a fractional-order proportional-integral-derivative (PID) feedback synthesis for regulation of the Hopf bifurcation embedded in a delayed small-world network model with Caputo derivative. Dynamic stability criterion and Hopf bifurcation condition are obtained by carrying out the eigenvalue analysis of the controlled network. The stability range of the parameters of the PID control is evaluated completely for the small-world network. We can optimize the dynamics of stability and bifurcation of small-world networks by manipulating the gain parameters. Finally, we implement some simulations to show the performance of the presented PID scheme. The numerical simulations verify the advantage of the fractional PID controller in bifurcation control.