10 September 2018 in IEEE Robotics and Automation Letters
This letter presents a case study of a learning-based approach for target-driven mapless navigation. The underlying navigation model is an end-to-end neural network, which is trained using a combination of expert demonstrations, imitation learning (IL) and reinforcement learning (RL). While RL and IL suffer from a large sample complexity and the distribution mismatch problem, respectively, we show that leveraging prior expert demonstrations for pretraining can reduce the training time to reach at least the same level of the performance compared to plain RL by a factor of 5. We present a thorough evaluation of different combinations of expert demonstrations, different RL algorithms, and reward functions, both in simulation and on a real robotic platform. Our results show that the final model outperforms both standalone approaches in the amount of successful navigation tasks. In addition, the RL reward function can be significantly simplified when using pretraining, e.g., by using a sparse reward only. The learned navigation policy is able to generalize to unseen and real-world environments.
13 February 2025 in IEEE Transactions on Automatic Control
Safely exploring environments with a-priori unknown constraints is a fundamental challenge that restricts the autonomy of robots. While safety is paramount, guarantees on sufficient exploration are also crucial for ensuring autonomous task completion. To address these challenges, we propose a novel safe guaranteed exploration framework using optimal control, which achieves first-of-its-kind results: guaranteed exploration for nonlinear systems with finite-time sample complexity bounds, while being provably safe with arbitrarily high probability. The framework is general and applicable to many real-world scenarios with complex nonlinear dynamics and unknown domains. We improve the efficiency of this general framework by proposing an algorithm, SAfe Guaranteed Exploration using Model Predictive Control (SageMPC). SageMPC leverages three key techniques: first, exploiting a Lipschitz bound; second, goal-directed exploration; and third, receding horizon style replanning, all while maintaining the desired sample complexity, safety, and exploration guarantees of the framework. Lastly, we demonstrate safe efficient exploration in challenging unknown environments using SageMPC with a car model.