We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed to be optimizing. The skill boundaries and the number of skills making up each demonstration are unknown. We use a Bayesian nonparametric approach to propose skill segmentations and maximum entropy inverse reinforcement learning to infer reward functions from the segments. This method produces a set of Markov Decision Processes (MDPs) that best describe the input trajectories. We evaluate this approach in a car driving domain and a simulated quadcopter obstacle course, showing that it is able to recover demonstrated skills more effectively than existing methods.
We introduce a model-free algorithm for learning in Markov decision processes with parameterized actions-discrete actions with continuous parameters. At each step the agent must select both which action to use and which parameters to use with that action. We introduce the Q-PAMDP algorithm for learning in these domains, show that it converges to a local optimum, and compare it to direct policy search in the goal-scoring and Platform domains.
We present a method for learning options from segmented demonstration trajectories. The trajectories are first segmented into skills using nonparametric Bayesian clustering and a reward function for each segment is then learned using inverse reinforcement learning. From this, a set of inferred trajectories for the demonstration are generated. Option initiation sets and termination conditions are learned from these trajectories using the one-class support vector machine clustering algorithm. We demonstrate our method in the four rooms domain, where an agent is able to autonomously discover usable options from human demonstration. Our results show that these inferred options can then be used to improve learning and planning.
7 November 2025 in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
This paper investigates the application of supervised learning for the purpose of match outcome prediction from Dota 2 in game chat logs. We analyze a dataset of 50,000 ranked matches, evaluating the predictive power of communication data alone and in combination with game events. Using LSTM and DistilBERT architectures, alongside a logistic regression baseline, we demonstrate that chat logs alone enable accurate prediction (up to 81.4% accuracy), while incorporating game events substantially improves performance (up to 98.4% accuracy). Our temporal analysis reveals that prediction reliability increases significantly during the mid-game phase (15-30 minutes), with models exhibiting different strengths - LSTM achieves higher accuracy while DistilBERT demonstrates greater prediction confidence. This study contributes to esports analytics by establishing chat logs as a viable predictive data source.