Modeling pedestrian dynamics and their implementation in a computer are challenging and important issues in the knowledge areas of transportation and computer simulation. The aim of this article is to provide a bibliographic outlook so that the reader may have quick access to the most relevant works related to this problem. We have used three main axes to organize the article’s contents: pedestrian models, validation techniques, and multiscale approaches. The backbone of this work is the classification of existing pedestrian models; we have organized the works in the literature under five categories, according to the techniques used for implementing the operational level in each pedestrian model. Then the main existing validation methods, oriented to evaluate the behavioral quality of the simulation systems, are reviewed. Furthermore, we review the key issues that arise when facing multiscale pedestrian modeling, where we first focus on the behavioral scale (combinations of micro and macro pedestrian models) and second on the scale size (from individuals to crowds). The article begins by introducing the main characteristics of walking dynamics and its analysis tools and concludes with a discussion about the contributions that different knowledge fields can make in the near future to this exciting area.

Reinforcement learning is one of the most promising machine learning techniques to get intelligent behaviors for embodied agents in simulations. The output of the classic Temporal Difference family of Reinforcement Learning algorithms adopts the form of a value function expressed as a numeric table or a function approximator. The learned behavior is then derived using a greedy policy with respect to this value function. Nevertheless, sometimes the learned policy does not meet expectations, and the task of authoring is difficult and unsafe because the modification of one value or parameter in the learned value function has unpredictable consequences in the space of the policies it represents. This invalidates direct manipulation of the learned value function as a method to modify the derived behaviors. In this paper, we propose the use of Inverse Reinforcement Learning to incorporate real behavior traces in the learning process to shape the learned behaviors, thus increasing their trustworthiness (in terms of conformance to reality). To do so, we adapt the Inverse Reinforcement Learning framework to the navigation problem domain. Specifically, we use Soft Q-learning, an algorithm based on the maximum causal entropy principle, with MARL-Ped (a Reinforcement Learning-based pedestrian simulator) to include information from trajectories of real pedestrians in the process of learning how to navigate inside a virtual 3D space that represents the real environment. A comparison with the behaviors learned using a Reinforcement Learning classic algorithm (Sarsa($\lambda$)) shows that the Inverse Reinforcement Learning behaviors adjust significantly better to the real trajectories.