The reaction wheel is able to help improve the balancing ability of a bicycle robot on curved pavement. However, preserving good control performances for such a robot that is driving on unstructured surfaces under matched and mismatched disturbances is challenging due to the underactuated characteristic and the nonlinearity of the robot. In this paper, a controller combining proximal policy optimization algorithms with terminal sliding mode controls is developed for controlling the balance of the robot. Online reinforcement-learning-based adaptive terminal sliding mode control is proposed to attenuate the influence of the matched and mismatched disturbance by adjusting parameters of the controller online. Different from several existing adaptive sliding mode approaches that only tune parameters of the reaching controller, the proposed method also considers the online adjustment of the sliding surface to provide adequate robustness against mismatched disturbances. The co-simulation experimental results in MSC Adams illustrate that the proposed controller can achieve better control performances than four existing methods for a reaction wheel bicycle robot moving on curved pavement, which verifies the robustness and applicability of the method.

The single-track two-wheeled (STTW) robot has the advantages of small size and flexibility, and it is suitable for traveling in narrow terrains of mountains and jungles. In this article, a reinforcement learning control method for STTW robots is proposed for driving fast in narrow terrain with limited visibility and line-of-sight occlusions. The proposed control scheme integrates path planning, trajectory tracking, and balancing control in a single framework. Based on this method, the state, action, and reward function are defined for narrow terrain passing tasks. At the same time, we design the actor network and the critic network structures and use the twin delayed deep deterministic policy gradient (TD3) to train these neural networks to construct a controller. Next, a simulation platform is formulated to test the performances of the proposed control method. The simulation results show that the obtained controller allows the STTW robot to effectively pass the training terrain, as well as the four test terrains. In addition, this article conducts a simulation comparison to prove the advantages of the integrated framework over traditional methods and the effectiveness of the reward function.