25 October 2022 in IEEE Robotics and Automation Letters
Reinforcement Learning (RL) has been shown to be effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. An increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a difficulty for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. Moreover, we also propose an approach to modularly learn the underlying reward-driven task, independent of safety constraints. We demonstrate that this approach both ensures safety and effectively guides exploration during training in a range of experiments, including zero-shot transfer when the reward is learned in a constraint-agnostic fashion.
8 September 2021 in IEEE Transactions on Robotics
In the context of heterogeneous multirobot teams deployed for executing multiple tasks, this article develops an energy-aware framework for allocating tasks to robots in an online fashion. With a primary focus on long-duration autonomy applications, we opt for a survivability-focused approach. Toward this end, the task prioritization and execution—through which the allocation of tasks to robots is effectively realized—are encoded as constraints within an optimization problem aimed at minimizing the energy consumed by the robots at each point in time. In this context, an allocation is interpreted as a prioritization of a task over all others by each of the robots. Furthermore, we present a novel framework to represent the heterogeneous capabilities of the robots, by distinguishing between the features available on the robots and the capabilities enabled by these features. By embedding these descriptions within the optimization problem, we make the framework resilient to situations, where environmental conditions make certain features unsuitable to support a capability and when component failures on the robots occur. We demonstrate the efficacy and resilience of the proposed approach in a variety of use-case scenarios, consisting of simulations and real robot experiments.
26 June 2019 in IEEE Transactions on Robotics
This paper presents a safe learning framework that employs an adaptive model learning algorithm together with barrier certificates for systems with possibly nonstationary agent dynamics. To extract the dynamic structure of the model, we use a sparse optimization technique. We use the learned model in combination with control barrier certificates that constrain policies (feedback controllers) in order to maintain safety, which refers to avoiding particular undesirable regions of the state space. Under certain conditions, recovery of safety in the sense of Lyapunov stability after violations of safety due to the nonstationarity is guaranteed. In addition, we reformulate an action-value function approximation to make any kernel-based nonlinear function estimation method applicable to our adaptive learning framework. Lastly, solutions to the barrier-certified policy optimization are guaranteed to be globally optimal, ensuring the greedy policy improvement under mild conditions. The resulting framework is validated via simulations of a quadrotor, which has previously been used under stationarity assumptions in the safe learnings literature, and is then tested on a real robot, the brushbot, whose dynamics is unknown, highly complex, and nonstationary.
2 July 2020 in IEEE Control Systems Letters
This paper introduces integral control barrier functions (I-CBFs) as a means to enable the safety-critical integral control of nonlinear systems. Importantly, I-CBFs allow for the holistic encoding of both state constraints and input bounds in a single framework. We demonstrate this by applying them to a dynamically defined tracking controller, thereby enforcing safety in state and input through a minimally invasive I-CBF controller framed as a quadratic program.
4 January 2018 in IEEE Robotics and Automation Letters
In this paper we consider the problem of rendering robotic tasks persistent by ensuring that the robots' energy levels are never depleted, which means that the tasks can be executed over long time horizons. This process is referred to as the persistification of the task. In particular, the state of each robot is augmented with its battery level so that the desired persistent behavior can be encoded as the forward invariance of a set such that the robots never deplete their batteries. Control barrier functions are employed to synthesize controllers that ensure that this set is forward invariant and, therefore, that the robotic task is persistent. As an application, this paper considers the persistification of a robotic sensor coverage task in which a group of robots has to cover an area of interest. The successful persistification of the coverage task is shown in simulation and on a team of mobile robots.
31 March 2020 in IEEE Transactions on Control Systems Technology
In this article, we propose a control framework that enables robots to execute tasks persistently, i.e., over time horizons much longer than robots' battery life. This is achieved by ensuring that the energy stored in the batteries of the robots is never depleted. This condition is framed as a set invariance constraint in an optimization problem whose objective is that to minimize the difference between the robots' control inputs and nominal control inputs corresponding to the task that is to be executed. We refer to this process as the persistification of a robotic task. Forward invariance of subsets of the state space of the robots is turned into a control input constraint by using control barrier functions. The solution to the formulated optimization problem with energy constraints ensures that the robotic task is persistent. To illustrate the operation of the proposed framework, we consider two tasks whose persistent execution is particularly relevant: environment exploration and environment surveillance. We show the persistification of these two tasks both in simulation and on a team of wheeled mobile robots on the Robotarium.