This brief considers the problem of trajectory tracking of a planar snake robot without a lateral constraint. The reference trajectory of the head position and the orientation of link 1 are given, and torque control is determined to reduce tracking errors. The performance of the controller was tested in a number of simulations. The robustness during actuator failure was also studied. We assumed that one of the actuators was broken and the corresponding joint became passive. Furthermore, as a more realistic situation, we considered an instance when some of the states were not readily accessible from the sensor readings and needed to be estimated by an observer. The extended Kalman filter was employed for this purpose, and the performance of the closed-loop system with the observer was also tested in simulations.
24 May 2023 in IEEE Transactions on Neural Networks and Learning Systems
The morphology and controller design of robots is often a labor-intensive task performed by experienced and intuitive engineers. Automatic robot design using machine learning is attracting increasing attention in the hope that it will reduce the design workload and result in better-performing robots. Most robots are created by joining several rigid parts and then mounting actuators and their controllers. Many studies limit the possible types of rigid parts to a finite set to reduce the computational burden. However, this not only limits the search space, but also prohibits the use of powerful optimization techniques. To find a robot closer to the global optimal design, a method that explores a richer set of robots is desirable. In this article, we propose a novel method to efficiently search for various robot designs. The method combines three different optimization methods with different characteristics. We apply proximal policy optimization (PPO) or soft actor-critic (SAC) as the controller, the REINFORCE algorithm to determine the lengths and other numerical parameters of the rigid parts, and a newly proposed method to determine the number and layout of the rigid parts and joints. Experiments with physical simulations confirm that when this method is used to handle two types of tasks—walking and manipulation—it performs better than simple combinations of existing methods. The source code and videos of our experiments are available online (https://github.com/r-koike/eagent).
Path integral policy improvement (PI²) is known to be an efficient reinforcement learning algorithm, particularly, if the target system is a high-dimensional dynamical system. However, PI², and its existing extensions, have adjustable parameters, on which the efficiency depends significantly. This article proposes an extension of PI² that adjusts all of the critical parameters automatically. Motion acquisition tasks for three different types of simulated legged robots were performed to test the efficacy of the proposed algorithm. The results show that the proposed method cannot only eliminate the burden on the user to set the parameters appropriately but also improve the optimization performance significantly. For one of the acquired motions, a real robot experiment was conducted to show the validity of the motion.
16 August 2023 in IEEE Transactions on Neural Networks and Learning Systems
This article describes a novel sufficient condition concerning approximations with reservoir computing (RC). Recently, RC using a physical system as the reservoir has attracted attention. Because many physical systems are modeled as state-space systems, it is necessary to guarantee the approximations given by reservoirs represented as nonlinear state-space systems. There are two problems with existing approaches: a reservoir must have a property called fading memory and must be represented as a set of maps between input and output signals on the bi-infinite-time (BIT) interval. These two conditions are too strict for reservoirs represented as nonlinear state-space systems as they require the reservoir to have a unique equilibrium state for the zero input. This article proposes an approach that employs operators from right-infinite-time (RIT) inputs to RIT outputs. Furthermore, we develop a novel extension of the Stone–Weierstrass theorem to handle discontinuous functions. To apply the extended theorem, we define functionals corresponding to operators and introduce a metric on the domain of the functionals. The resulting sufficient condition does not require the reservoir to have fading memory or continuity with respect to inputs and time. Therefore, our result guarantees the approximations with very common reservoirs and provides a rationale for physical RC. We present an example of a physical reservoir without fading memory. With the example reservoir, the RC model successfully approximates NARMA10, a benchmark task for time series predictions.