Achieving omnidirectional walking for bipedal robots is considered one of the most challenging tasks in robotics technology. Reinforcement learning (RL) methods have proved effective in bipedal walking tasks. However, most existing methods use state machines to switch between multiple policies and achieve omnidirectional gait, which results in shaking during the policy switching process for bipedal robots. To achieve a seamless transition between omnidirectional gait and transient motion for full-size bipedal robots, we propose a novel multi-agent RL method. Firstly, a multi-agent RL algorithm based on the actor–critic framework is designed, and policy entropy is introduced to improve exploration efficiency. By learning agents with parallel initial state distributions, we minimize reliance on gait planner effectiveness in the Robot Operating System (ROS). Additionally, we design a novel heterogeneous policy experience replay mechanism based on Euclidean distance. Secondly, considering the periodicity of bipedal robot walking, we develop a new periodic gait function. Including periodic objectives in the policy can accelerate the convergence speed of training periodic gait functions. Finally, to enhance the robustness of the policy, we construct a novel curriculum learning method by discretizing Gaussian distribution and incorporate it into the robot’s training task. Our method is validated in a simulation environment, and the results show that our method can achieve multiple gaits through a policy network and achieve smooth transitions between different gaits.

The parallel dual-slider telescopic leg bipedal robot (L04) is characterized by its simple structure and low leg rotational inertia, which contribute to its walking efficiency. However, end-to-end methods often overlook the robot’s physical structure, leading to difficulties in maintaining the parallel alignment of the dual sliders, which in turn compromises walking stability. One potential solution to this issue involves utilizing imitation learning to replicate human motion data. However, the dual telescopic leg structure of the L04 robot makes it difficult to perform motion retargeting of human motion data. To enable L04 walking, we design a method that integrates prior feedforward with reinforcement learning (PFRL), specifically tailored for the parallel dual-slider structure. We utilize prior knowledge as a feedforward action to compensate for system nonlinearities; meanwhile, the feedback action generated by the policy network adaptively regulates dynamic balance and, combined with the feedforward action, jointly controls the robot’s walking. PFRL enforces constraints within the motion space to mitigate the chaotic behavior of the parallel dual sliders. Experimental results show that our method successfully achieves sim2real transfer on a real bipedal robot without the need for domain randomization techniques or intricate reward functions. L04 achieves omnidirectional walking with minimal energy consumption and exhibits robustness against external disturbances.

The presence of sensor noise, missing states and inadequate future prediction capabilities imposes significant limitations on the locomotion performance of bipedal robots operating in unstructured terrain. Conventional methods generally depend on long-term history observations to reconstruct single-frame privileged information. However, these methods fail to acknowledge the pivotal function of short-term history in rapid state responses and the significance of future state prediction in anticipating potential risks. The proposed framework is a Long–Short World Model (LSWM), which integrates state reconstruction and future state prediction to enhance the locomotion capabilities of bipedal robots in complex environments. The LSWM framework comprises two modules: a state reconstruction module (SRM) and a future state prediction module (SPM). The state reconstruction module employs long-term history observations to reconstruct privileged information in the current short-term history, thereby effectively improving the system’s robustness to sensor noise and enhancing state observability. The future state prediction module enhances the robot’s adaptability to complex environments and unpredictable scenarios by predicting the robot’s future short-term privileged information. We conducted extensive comparative experiments in simulation as well as in a variety of real-world indoor and outdoor environments. In the indoor stair-climbing task, LSWM achieved a 94% success rate, outperforming the current state-of-the-art baseline methods by at least 34%, thereby demonstrating its substantial performance advantages in complex and dynamic environments.

Blind locomotion policies trained in simulation for unstructured terrains often suffer from significant performance degradation when deployed on real bipedal robots. A typical manifestation is the emergence of abnormal behaviors (e.g., sudden high-stepping motions) even on simple terrains like flat ground. This issue arises from environmental noise interference and the inability to reliably obtain certain critical state information (such as terrain heightmaps and linear velocities) through proprioception. Existing methods commonly attempt to address this problem by injecting large amounts of noise and learning explicit or implicit representations of the missing information. Nevertheless, excessive noise can severely restrict the potential performance of the policy. The more fundamental challenge lies in the fact that both explicit and implicit representations constructed solely from historical proprioceptive data are unable to accurately reconstruct key unobservable state variables. This inevitable estimation bias ultimately leads to abnormal behaviors. To address these challenges, we propose a bipedal training framework based on a Smoothed and Denoised World Model (SDWM), which explicitly mitigates the effects of observation noise, incomplete state information, and inaccurate reconstruction. We validate the effectiveness of SDWMthrough comparative tests conducted in simulation as well as indoor and outdoor realworld scenarios.

Tendon-driven bipedal robots exhibit complex time-varying dynamics due to elastic deformations, multi-joint coupling, and transmission delays. These characteristics lead to significant sim-to-real discrepancies and limit the robot’s performance in complex terrains. To address this issue, we propose a two-stage locomotion control training framework based on Real2Sim2Real (RSR). In the first stage, joint motion data collected from the real robot are used to train a torque refinement policy in simulation, implicitly modeling the time-varying dynamics of the tendon-driven system and reducing the body dynamics gap during sim-to-real transfer. In the second stage, we introduce a reinforcement learning approach that integrates explicit estimation with implicit representation. By explicitly estimating body linear velocity and local terrain information under the feet, and simultaneously learning task-relevant latent features through implicit representation, the robot’s adaptability to complex terrains is enhanced. Experimental results show that, for a forward velocity tracking task of 2.5 m/s, the proposed explicit–implicit learning method achieves a 15.9% reduction in velocity tracking error compared to the purely implicit representation baseline (IWM). When further combined with the torque refinement policy (RSR), the tracking error is further reduced by 86.4% compared to the explicit–implicit baseline (EIWM). Moreover, the proposed method enables stable locomotion across various complex terrains, demonstrating its effectiveness in improving sim-to-real transfer performance and terrain adaptability.

Robust blind locomotion over complex unstructured terrains relies on accurate estimation of robot states and surrounding terrain geometry. However, under real-world deployment conditions without exteroceptive perception, it remains challenging to accurately estimate robot states and infer surrounding terrain structures solely from noisy proprioceptive observations. Existing methods commonly learn single-scale implicit terrain representations from historical proprioceptive observations and explicitly estimate robot states. However, they lack explicit terrain estimation and may lose critical geometric details. Moreover, single-scale terrain information is insufficient to capture both local geometric structures and global terrain trends. To address these issues, we propose a Terrain and State Implicit-Explicit Estimation (TSIE) framework to improve the locomotion capability of bipedal robots over complex terrains. TSIE encodes long-horizon proprioceptive observations using a Long Short-Term Memory (LSTM) network and introduces a dual-branch architecture consisting of a Terrain Implicit-Explicit Estimator (Terrain-IE) and a State Implicit-Explicit Estimator (State-IE). Terrain-IE performs multi-scale terrain implicit-explicit estimation by explicitly estimating a local high-resolution height map and implicitly reconstructing a global low-resolution height map. By preserving gradient connections between the two terrain branches, Terrain-IE enables joint implicit-explicit training of multi-scale terrain representations, improving terrain understanding and estimation accuracy. State-IE explicitly estimates the base linear velocity and foot-centered height map, while implicitly reconstructing future proprioceptive states to further improve tracking performance and locomotion robustness. We validate TSIE through simulation and real-world experiments on a full-sized bipedal robot platform with a height of $170\mathrm{cm}$ and a mass of $35\mathrm{kg}$. Experimental results show that TSIE outperforms baseline methods in complex-terrain traversal capability, terrain and state estimation accuracy, and velocity-tracking stability. Real-world deployment further demonstrates the robustness of TSIE across indoor and outdoor complex terrains, achieving a 95% success rate in continuous stair ascent and descent tasks.