Hybrid zero dynamics (HZD) has emerged as a popular framework for dynamic walking but has significant implementation difficulties when applied to the high degrees of freedom humanoids. The primary impediment is the process of gait design—it is difficult for optimizers to converge on a viable set of virtual constraints defining a gait. This paper presents a methodology that allows for fast and reliable generation of dynamic robotic walking gaits through the HZD framework, even in the presence of underactuation. Specifically, we describe an optimization formulation that builds upon the novel combination of HZD and direct collocation methods. Furthermore, achieving a scalable implementation required developing a defect-variable substitution formulation to simplify expressions, which ultimately allows us to generate compact analytic Jacobians of the constraints. We experimentally validate our methodology on an underactuated humanoid, DURUS, a spring-legged machine designed to facilitate energy-economical walking. We show that the optimization approach, in concert with the HZD framework, yields dynamic and stable walking gaits in hardware with a total electrical cost of transport of 1.33.

This paper presents the methodology used to achieve efficient and dynamic walking behaviors on the prototype humanoid robotics platform, DURUS. As a means of providing a hardware platform capable of these behaviors, the design of DURUS combines highly efficient electromechanical components with “control in the loop” design of the leg morphology. Utilizing the final design of DURUS, a formal framework for the generation of dynamic walking gaits which maximizes efficiency by exploiting the full body dynamics of the robot, including the interplay between the passive and active elements, is developed. The gaits generated through this methodology form the basis of the control implementation experimentally realized on DURUS; in particular, the trajectories generated through the formal framework yield a feedforward control input which is modulated by feedback in the form of regulators that compensate for discrepancies between the model and physical system. The end result of the unified approach to control-informed mechanical design, formal gait design and regulator-based feedback control implementation is efficient and dynamic locomotion on the humanoid robot DURUS. In particular, DURUS was able to demonstrate dynamic locomotion at the DRC Finals Endurance Test, walking for just under five hours in a single day, traveling 3.9 km with a mean cost of transport of 1.61-the lowest reported cost of transport achieved on a bipedal humanoid robot.

This paper presents a novel reinforcement learning (RL) framework to design cascade feedback control policies for 3D bipedal locomotion. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint or task space trajectories. Unlike these studies, we propose a policy structure that decouples the bipedal locomotion problem into two modules that incorporate the physical insights from the nature of the walking dynamics and the well-established Hybrid Zero Dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, sample efficiency, and less dependence on prior knowledge. The proposed solution learns stable and robust walking gaits from scratch and allows the controller to realize omnidirectional walking with accurate tracking of the desired velocity and heading angle. The learned policies also perform robustly against various adversarial forces applied to the torso and walking blindly on a series of challenging and unstructured terrains. These results demonstrate that the proposed cascade feedback control policy is suitable for navigation of 3D bipedal robots in indoor and outdoor environments.

In this work, we demonstrate robust walking in the bipedal robot Digit on uneven terrains by just learning a single linear policy. In particular, we propose a new control pipeline, wherein the high-level trajectory modulator shapes the end-foot ellipsoidal trajectories, and the low-level gait controller regulates the torso and ankle orientation. The foot-trajectory modulator uses a linear policy and the regulator uses a linear PD control law. As opposed to neural network based policies, the proposed linear policy has only 13 learnable parameters, thereby not only guaranteeing sample efficient learning but also enabling simplicity and interpretability of the policy. This is achieved with no loss of performance on challenging terrains like slopes, stairs and outdoor landscapes. We first demonstrate robust walking in the custom simulation environment, MuJoCo, and then directly transfer to hardware with no modification of the control pipeline. We subject the biped to a series of pushes and terrain height changes, both indoors and outdoors, thereby validating the presented work.

This paper presents an optimal gait synthesis method that exploits the full body dynamics of robots using the Hybrid Zero Dynamics (HZD) control framework and-for the first time-experimentally realizes online HZD gait generation for a planar underactuated robot. Hybrid zero dynamics is an established theoretical framework that formally enables stable control of dynamic locomotion by enforcing virtual constraints through feedback controllers. An essential part of successfully realizing dynamic walking with HZD framework is determining parameters of the virtual constraints that satisfy hybrid invariant condition via nonlinear constrained optimization. Due to the complexity of the full hybrid system model of the robot, these optimization problems often suffer from slow convergence and local minima. In this paper, we improve the reliability of the HZD gait optimization and significantly increase the convergence speed by taking advantage of the direct transcription formulation and the exponential convergence of the global orthogonal collocation (a.k.a. pseudospectral) method. As a result, generating HZD gaits online becomes feasible with an average computation time less than 0.5 seconds, as will be demonstrated experimentally on a bipedal robot.

Correct-by-construction techniques, such as control barrier functions (CBFs), can be used to guarantee closed-loop safety by acting as a supervisor of an existing legacy controller. However, supervisory-control intervention typically compromises the performance of the closed-loop system. On the other hand, machine learning has been used to synthesize controllers that inherit good properties from a training dataset, though safety is typically not guaranteed due to the difficulty of analyzing the associated learning structure. In this paper, supervised learning is combined with CBFs to synthesize controllers that enjoy good performance with provable safety. A training set is generated by trajectory optimization that incorporates the CBF constraint for an interesting range of initial conditions of the truck model. A control policy is obtained via supervised learning that maps a feature representing the initial conditions to a parameterized desired trajectory. The learning-based controller is used as the performance controller and a CBF-based supervisory controller guarantees safety. A case study of lane keeping (LK) for articulated trucks shows that the controller trained by supervised learning inherits the good performance of the training set and rarely requires intervention by the CBF supervisor.