Bipedal walking is a challenging task for humanoid robots. In this study, we develop a lightweight reinforcement learning method for real-time gait planning of the biped robot. We regard bipedal walking as a process in which the robot constantly interacts with the environment, judges the quality of control action through the walking state, and then adjusts the control strategy. A mean-asynchronous advantage actor-critic (M-A3C) reinforcement learning algorithm is proposed to obtain the continuous state space and action space, and directly obtain the final gait of the robot without introducing the reference gait. We use multiple sub-agents of M-A3C algorithm to train multiple virtual robots independently at the same time in the physical simulation platform. Then we transfer the trained model to the walking control of the actual robot to reduce the number of training on the actual robot, improve the training speed, and ensure the acquisition of the final gait. Finally, a biped robot is designed and fabricated to verify the effectiveness of the proposed method. Various experiments show that the proposed method can achieve the biped robot’s continuous and stable gait planning.

This paper presents an exciting and meaningful design to make mobile robots capable of adapting to various terrains. We designed a relatively simple and novel composite motion mechanism called the flexible spoked mecanum (FSM) wheel and created a mobile robot, LZ-1, with multiple motion modes based on the FSM wheel. Based on the motion analysis of the FSM wheel, we designed an omnidirectional motion mode for this robot, allowing it to move flexibly in all directions and successfully traverse rugged terrains. In addition, we designed a crawl motion mode for this robot, which can climb stairs effectively. We used a multilayer control method to move the robot according to the designed motion modes. Multiple experiments showed that these two motion modes for the robot are effective on various terrains.

This paper proposes a novel fluxible spoked Mecanum wheel structure and a mobile robot based on this wheel, and introduces in detail the motion mode and corresponding control methods of the fluxible spoked Mecanum wheel in the omni-directional movement mode and crawl mode of the mobile robot.