Since path planning for multi-arm manipulators is a complicated high-dimensional problem, effective and fast path generation is not easy for the arbitrarily given start and goal locations of the end effector. Especially, when it comes to deep reinforcement learning-based path planning, high-dimensionality makes it difficult for existing reinforcement learning-based methods to have efficient exploration which is crucial for successful training. The recently proposed soft actor–critic (SAC) is well known to have good exploration ability due to the use of the entropy term in the objective function. Motivated by this, in this paper, a SAC-based path planning algorithm is proposed. The hindsight experience replay (HER) is also employed for sample efficiency and configuration space augmentation is used in order to deal with complicated configuration space of the multi-arms. To show the effectiveness of the proposed algorithm, both simulation and experiment results are given. By comparing with existing results, it is demonstrated that the proposed method outperforms the existing results.

In order to enhance performance of robot systems in the manufacturing industry, it is essential to develop motion and task planning algorithms. Especially, it is important for the motion plan to be generated automatically in order to deal with various working environments. Although PRM (Probabilistic Roadmap) provides feasible paths when the starting and goal positions of a robot manipulator are given, the path might not be smooth enough, which can lead to inefficient performance of the robot system. This paper proposes a motion planning algorithm for robot manipulators using a twin delayed deep deterministic policy gradient (TD3) which is a reinforcement learning algorithm tailored to MDP with continuous action. Besides, hindsight experience replay (HER) is employed in the TD3 to enhance sample efficiency. Since path planning for a robot manipulator is an MDP (Markov Decision Process) with sparse reward and HER can deal with such a problem, this paper proposes a motion planning algorithm using TD3 with HER. The proposed algorithm is applied to 2-DOF and 3-DOF manipulators and it is shown that the designed paths are smoother and shorter than those designed by PRM.

In this study, an output‐feedback model predictive controller (MPC) is presented for capacitor voltage regulation on the basis of a non‐linear DC/DC converter. The proposed output‐feedback MPC scheme is constructed by combining a state‐feedback MPC scheme with a Luenberger‐type observer. The proposed MPC does not carry out any online optimisation. Although the converter model is non‐linear, the proposed observer gain ensures global exponential convergence of the state estimation errors. On the basis of this observer, an output‐feedback MPC scheme is designed by certainty equivalence. It is also shown that the capacitor voltage globally converges to its reference while fulfilling input constraints. In order to eliminate the capacitor voltage offset error, a PI controller is employed in the outer loop. The experiment results demonstrate that the closed‐loop performance is satisfactory despite the load resistance variations.

In the workspace of robot manipulators in practice, it is common that there are both static and periodic moving obstacles. Existing results in the literature have been focusing mainly on the static obstacles. This paper is concerned with multi-arm manipulators with periodically moving obstacles. Due to the high-dimensional property and the moving obstacles, existing results suffer from finding the optimal path for given arbitrary starting and goal points. To solve the path planning problem, this paper presents a SAC-based (Soft actor–critic) path planning algorithm for multi-arm manipulators with periodically moving obstacles. In particular, the deep neural networks in the SAC are designed such that they utilize the position information of the moving obstacles over the past finite time horizon. In addition, the hindsight experience replay (HER) technique is employed to use the training data efficiently. In order to show the performance of the proposed SAC-based path planning, both simulation and experiment results using open manipulators are given.

This article presents a complex gain margin of discrete-time linear quadratic regulator (DLQR) and its application to a consensus problem of multi-agent higher order linear systems. Since the consensus problem can be converted into a robust control problem with perturbation expressed by complex numbers, and since the classical gain and phase margins are not enough to handle the current case, we study the so-called ‘disc margin’ which is somehow a combination of gain and phase margins. We first compute the disc margin of DLQR controller based on a Lyapunov argument, which is simple but yields a relaxed result over those previously reported in the literature. Then, it is shown that the disc margin can be enlarged arbitrarily when the system is asymptotically null controllable with bounded controls and when a low-gain feedback is employed. Based on this fact, the discrete-time consensus problem is solved by a DLQR-based consensus controller. Simulation study shows that the DLQR-based consensus controller has better robustness property against model uncertainties in the input channel.