Personalized cardiac electrophysiology simulations have demonstrated great potential to study cardiac arrhythmias and help in therapy planning of radio-frequency ablation. Its application to analyze vulnerability to ventricular tachycardia and sudden cardiac death in infarcted patients has been recently explored. However, the detailed multi-scale biophysical simulations used in these studies are very demanding in terms of memory and computational resources, which prevents their clinical translation. In this work, we present a fast phenomenological system based on cellular automata (CA) to simulate personalized cardiac electrophysiology. The system is trained on biophysical simulations to reproduce cellular and tissue dynamics in healthy and pathological conditions, including action potential restitution, conduction velocity restitution and cell safety factor. We show that a full ventricular simulation can be performed in the order of seconds, emulate the results of a biophysical simulation and reproduce a patient’s ventricular tachycardia in a model that includes a heterogeneous scar region. The system could be used to study the risk of arrhythmia in infarcted patients for a large number of scenarios.

Reinforcement learning is one of the most promising machine learning techniques to get intelligent behaviors for embodied agents in simulations. The output of the classic Temporal Difference family of Reinforcement Learning algorithms adopts the form of a value function expressed as a numeric table or a function approximator. The learned behavior is then derived using a greedy policy with respect to this value function. Nevertheless, sometimes the learned policy does not meet expectations, and the task of authoring is difficult and unsafe because the modification of one value or parameter in the learned value function has unpredictable consequences in the space of the policies it represents. This invalidates direct manipulation of the learned value function as a method to modify the derived behaviors. In this paper, we propose the use of Inverse Reinforcement Learning to incorporate real behavior traces in the learning process to shape the learned behaviors, thus increasing their trustworthiness (in terms of conformance to reality). To do so, we adapt the Inverse Reinforcement Learning framework to the navigation problem domain. Specifically, we use Soft Q-learning, an algorithm based on the maximum causal entropy principle, with MARL-Ped (a Reinforcement Learning-based pedestrian simulator) to include information from trajectories of real pedestrians in the process of learning how to navigate inside a virtual 3D space that represents the real environment. A comparison with the behaviors learned using a Reinforcement Learning classic algorithm (Sarsa($\lambda$)) shows that the Inverse Reinforcement Learning behaviors adjust significantly better to the real trajectories.