21 August 2023 in IEEE Transactions on Robotics
The empirical success of reinforcement learning (RL) in contact-rich manipulation leaves much to be understood from a model-based perspective, where the key difficulties are often attributed to 1) the explosion of contact modes, 2) stiff, nonsmooth contact dynamics and the resulting exploding/discontinuous gradients, and 3) the nonconvexity of the planning problem. The stochastic nature of RL addresses 1) and 2) by effectively sampling and averaging the contact modes. On the other hand, model-based methods have tackled the same challenges by smoothing contact dynamics analytically. Our first contribution is to establish the theoretical equivalence of the two smoothing schemes for simple systems, and provide qualitative and empirical equivalence on several complex examples. In order to further alleviate 2), our second contribution is a convex, differentiable, and quasi-dynamic formulation of contact dynamics, which is amenable to both smoothing schemes, and has proven to be highly effective for contact-rich planning. Our final contribution resolves 3), where we show that classical sampling-based motion planning algorithms can be effective in global planning when contact modes are abstracted via smoothing. Applying our method on several challenging contact-rich manipulation tasks, we demonstrate that efficient model-based motion planning can achieve results comparable to RL, but with dramatically less computation.
13 September 2023 in IEEE Robotics and Automation Letters
Sums-of-squares (SOS) optimization is a promising tool to synthesize certifiable controllers for nonlinear dynamical systems. Building upon prior works (Lasserre et al., 2008), (Jiang and Jiang, 2015), we demonstrate that SOS can synthesize dynamic controllers with bounded suboptimal performance for various underactuated robotic systems by finding good approximations of the value function. We summarize a unified SOS framework to synthesize both under- and over- approximations of the value function for continuous-time, control-affine systems, use these approximations to generate approximate optimal controllers, and perform regional analysis on the closed-loop system driven by these controllers. We then extend the formulation to handle hybrid systems with contacts. We demonstrate that our method can generate tight under- and over- approximations of the value function with low-degree polynomials, which are used to provide stabilizing controllers for continuous-time systems including the inverted pendulum, the cart-pole, and the quadrotor as well as a hybrid system, the planar pusher. To the best of our knowledge, this is the first time that a SOS-based time-invariant controller can swing up and stabilize a cart-pole, and push the planar slider to the desired pose.