10 October 2016 in IEEE Transactions on Automation Science and Engineering
In this paper, we define a novel census signal temporal logic (CensusSTL) that focuses on the number of agents in different subsets of a group that complete a certain task specified by the STL. CensusSTL consists of an “inner logic” STL formula and an “outer logic” STL formula. We present a new inference algorithm to infer CensusSTL formulas from the trajectory data of a group of agents. We first identify the “inner logic” STL formula and then infer the subgroups based on whether the agents' behaviors satisfy the “inner logic” formula at each time point. We use two different approaches to infer the subgroups based on similarity and complementarity, respectively. The “outer logic” CensusSTL formula is inferred from the census trajectories of different subgroups. We apply the algorithm in analyzing data from a soccer match by inferring the CensusSTL formula for different subgroups of a soccer team.
1 January 2019 in IJCAI : Proceedings of the Conference
Transferring high-level knowledge from a source task to a target task is an effective way to expedite reinforcement learning (RL). For example, propositional logic and first-order logic have been used as representations of such knowledge. We study the transfer of knowledge between tasks in which the timing of the events matters. We call such tasks temporal tasks. We concretize similarity between temporal tasks through a notion of logical transferability, and develop a transfer learning approach between different yet similar temporal tasks. We first propose an inference technique to extract metric interval temporal logic (MITL) formulas in sequential disjunctive normal form from labeled trajectories collected in RL of the two tasks. If logical transferability is identified through this inference, we construct a timed automaton for each sequential conjunctive subformula of the inferred MITL formulas from both tasks. We perform RL on the extended state which includes the locations and clock valuations of the timed automata for the source task. We then establish mappings between the corresponding components (clocks, locations, etc.) of the timed automata from the two tasks, and transfer the extended Q-functions based on the established mappings. Finally, we perform RL on the extended state for the target task, starting with the transferred extended Q-functions. Our implementation results show, depending on how similar the source task and the target task are, that the sampling efficiency for the target task can be improved by up to one order of magnitude by performing RL in the extended state space, and further improved by up to another order of magnitude using the transferred extended Q-functions.
7 June 2018 in IEEE Transactions on Automation Science and Engineering
In this paper, we present a method to learn (infer) and refine a set of advices from the trajectories generated in the successful and failed attempts in a task or game, in the form of advisory signal temporal logic (STL) formulas. Each advice consists of an advisory motion STL formula that characterizes the spatial-temporal pattern of the motion as a feature of success and an advisory selection STL formula as a criterion for the environment to select the advice. For the inference of advisory STL formulas, we provide a theoretical framework of perfect classification with a labeled set of trajectories with different time lengths. We design an advisory controller that can drive the robots to satisfy an advisory motion STL formula based on the advice selected according to the advisory selection STL formula. The advisory controller can advise or guide the human operators or the robots for better performance with the shared autonomy between the human operator and the controller. We provide two case studies to test the effectiveness of the advisory controller, one with a Baxter-On-Wheels simulator and the other with two quadrotors in an experimental testbed in iteratively improving the success rates of completing the tasks with the help of the designed advisory controller.
23 October 2017 in IEEE Systems Journal
We present an energy storage controller synthesis method for power systems with respect to metric temporal logic (MTL) specifications. The power systems with both constant impedance loads and constant power loads are modeled as a set of differential–algebraic equations. After a fault is cleared, with uncertainties in the fault clearing time, the generator machine angles and rotor speed deviations will enter a set of postfault initial states. We use the robust neighborhood approach to cover this set using the initial robust neighborhoods of finitely many simulated postfault trajectories. These simulated postfault trajectories meet the frequency regulation requirements specified in MTL as they are driven by the optimal control input signals obtained through a functional gradient descent approach. In this way, all the possible postfault trajectories with the given uncertainties in the fault clearing time are guaranteed to satisfy the MTL specification. Furthermore, we learn a piecewise linear control law from the data of the simulated trajectories to generate a feedback controller.
24 June 2019 in IEEE Robotics and Automation Letters
In this paper, we propose a method to perform swarm behavior analysis with a novel swarm signal temporal logic (SwarmSTL). We define generalized moments to describe swarm features and propose a logical proposition to represent an event, where the Boolean value of the logical proposition at a certain time is known a priori. We develop methods for SwarmSTL monitoring and inference. As the swarm size can be large, we also propose methods to perform the above tasks by sampling. In particular, we apply our methods to three case studies that aim to monitor swarm "maneuver" behavior using an agent sample, infer a SwarmSTL formula to describe the cause of the split, and explore a SwarmSTL formula to describe "mixed-species foraging flock" of birds.