Ariyan Bighashdel

Ariyan Bighashdel

Auto-generated profile, based on publicly available information

Ariyan Bighashdel

Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 December 2022 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
YWYa Wu
Ya Wu
ABAriyan Bighashdel
Ariyan Bighashdel
GCGuang Chen
Guang Chen
GDGijs Dubbelman
Gijs Dubbelman
PJPavol Jancura
Pavol Jancura
Learning to predict the trajectories of pedestrians is essential for improving safety and efficiency of mobile robots. The prediction is challenging since the robot needs to operate in multiple environments in which the motion patterns of pedestrians are different between environments. Existing pedestrian trajectory prediction models heavily rely on the availability of representative data samples during training. In the presence of additional training data from a new environment, these models must be retrained on all datasets to avoid catastrophic forgetting of the knowledge obtained from the already supported environments. In this paper, we address this catastrophic forgetting problem in the context of learning to predict the trajectories of pedestrians. We propose a pseudo-rehearsal approach based on a novel Generative Replay (GR) model, referred to as Social-GR . The proposed method is consistent with crowd motion patterns and is free of any explicit reference to past experiences. To demonstrate the problem of catastrophic forgetting and evaluate our solution, we develop the Continual Trajectory Prediction Benchmark , which consists of four tasks, each representing a real-world pedestrian trajectory dataset from a different environment. By conducting several experiments, we show that our proposed Social-GR approach significantly outperforms other continual learning methods that depend on explicit experience replay, including the state-of-the-art conditional-GR model. We further illustrate the robustness of our proposed approach to mitigating catastrophic forgetting by switching the order of environments and employing a more complex prediction model.

10 September 2021
Verified
Book Chapter
Closed access
PDF file
ABAriyan Bighashdel
Ariyan Bighashdel
PMPanagiotis Meletis
Panagiotis Meletis
PJPavol Jancura
Pavol Jancura
GDGijs Dubbelman
Gijs Dubbelman

30 November 2022 in Machine Learning
Verified
Journal Article
Open access
PDF file
ABAriyan Bighashdel
Ariyan Bighashdel
PJPavol Jancura
Pavol Jancura
GDGijs Dubbelman
Gijs Dubbelman
In this paper, we define a novel inverse reinforcement learning (IRL) problem where the demonstrations are multi-intention, i.e., collected from multi-intention experts, unlabeled, i.e., without intention labels, and partially overlapping, i.e., shared between multiple intentions. In the presence of overlapping demonstrations, current IRL methods, developed to handle multi-intention and unlabeled demonstrations, cannot successfully learn the underlying reward functions. To solve this limitation, we propose a novel clustering-based approach to disentangle the observed demonstrations and experimentally validate its advantages. Traditional clustering-based approaches to multi-intention IRL, which are developed on the basis of model-based Reinforcement Learning (RL), formulate the problem using parametric density estimation. However, in high-dimensional environments and unknown system dynamics, i.e., model-free RL, the solution of parametric density estimation is only tractable up to the density normalization constant. To solve this, we formulate the problem as a mixture of logistic regressions to directly handle the unnormalized density. To research the challenges faced by overlapping demonstrations, we introduce the concepts of shared pair, which is a state-action pair that is shared in more than one intention, and separability, which resembles how well the multiple intentions can be separated in the joint state-action space. We provide theoretical analyses under the global optimality condition and the existence of shared pairs. Furthermore, we conduct extensive experiments on four simulated robotics tasks, extended to accept different intentions with specific levels of separability, and a synthetic driver task developed to directly control the separability. We evaluate the existing baselines on our defined problem and demonstrate, theoretically and experimentally, the advantages of our clustering-based solution, especially when the separability of the demonstrations decreases.

4 April 2023 in arXiv
Verified
Preprint
Open access
PDF file
ABAriyan Bighashdel
Ariyan Bighashdel
DGDaan de Geus
Daan de Geus
PJPavol Jancura
Pavol Jancura
GDGijs Dubbelman
Gijs Dubbelman
Learning anticipation in Multi-Agent Reinforcement Learning (MARL) is a reasoning paradigm where agents anticipate the learning steps of other agents to improve cooperation among themselves. As MARL uses gradient-based optimization, learning anticipation requires using Higher-Order Gradients (HOG), with so-called HOG methods. Existing HOG methods are based on policy parameter anticipation, i.e., agents anticipate the changes in policy parameters of other agents. Currently, however, these existing HOG methods have only been applied to differentiable games or games with small state spaces. In this work, we demonstrate that in the case of non-differentiable games with large state spaces, existing HOG methods do not perform well and are inefficient due to their inherent limitations related to policy parameter anticipation and multiple sampling stages. To overcome these problems, we propose Off-Policy Action Anticipation (OffPA2), a novel framework that approaches learning anticipation through action anticipation, i.e., agents anticipate the changes in actions of other agents, via off-policy sampling. We theoretically analyze our proposed OffPA2 and employ it to develop multiple HOG methods that are applicable to non-differentiable games with large state spaces. We conduct a large set of experiments and illustrate that our proposed HOG methods outperform the existing ones regarding efficiency and performance.

28 December 2022 in Machine Learning
Verified
Journal Article
Open access
PDF file
ABAriyan Bighashdel
Ariyan Bighashdel
PJPavol Jancura
Pavol Jancura
GDGijs Dubbelman
Gijs Dubbelman
There is a missing character "i" in equation 3 of the published manuscript. Here is the incorrect equation: and here is the corrected equation: q(s, a) ∝ i∼p(i)

2 April 2023 in arXiv
Verified
Preprint
Open access
PDF file
ABAriyan Bighashdel
Ariyan Bighashdel
DGDaan de Geus
Daan de Geus
PJPavol Jancura
Pavol Jancura
GDGijs Dubbelman
Gijs Dubbelman
Learning anticipation is a reasoning paradigm in multi-agent reinforcement learning, where agents, during learning, consider the anticipated learning of other agents. There has been substantial research into the role of learning anticipation in improving cooperation among self-interested agents in general-sum games. Two primary examples are Learning with Opponent-Learning Awareness (LOLA), which anticipates and shapes the opponent's learning process to ensure cooperation among self-interested agents in various games such as iterated prisoner's dilemma, and Look-Ahead (LA), which uses learning anticipation to guarantee convergence in games with cyclic behaviors. So far, the effectiveness of applying learning anticipation to fully-cooperative games has not been explored. In this study, we aim to research the influence of learning anticipation on coordination among common-interested agents. We first illustrate that both LOLA and LA, when applied to fully-cooperative games, degrade coordination among agents, causing worst-case outcomes. Subsequently, to overcome this miscoordination behavior, we propose Hierarchical Learning Anticipation (HLA), where agents anticipate the learning of other agents in a hierarchical fashion. Specifically, HLA assigns agents to several hierarchy levels to properly regulate their reasonings. Our theoretical and empirical findings confirm that HLA can significantly improve coordination among common-interested agents in fully-cooperative normal-form games. With HLA, to the best of our knowledge, we are the first to unlock the benefits of learning anticipation for fully-cooperative games.