Despite the advances in the autonomous driving domain, autonomous vehicles (AVs) are still inefficient and limited in terms of cooperating with each other or coordinating with vehicles operated by humans. A group of autonomous and human-driven vehicles (HVs) which work together to optimize an altruistic social utility can co-exist seamlessly and assure safety and efficiency on the road. Achieving this mission without explicit coordination among agents is challenging, mainly due to the difficulty of predicting the behavior of humans with heterogeneous preferences in mixed-autonomy environments. Formally, we model an AV’s maneuver planning in mixed-autonomy traffic as a partially-observable stochastic game and attempt to derive optimal policies that lead to socially-desirable outcomes using a multi-agent reinforcement learning framework (MARL), and propose a semi-sequential multi-agent training and policy dissemination algorithm for our MARL problem. We introduce a quantitative representation of the AVs’ social preferences and design a distributed reward structure that induces altruism into their decision-making process. Altruistic AVs are able to form alliances, guide the traffic, and affect the behavior of the HVs to handle competitive driving scenarios. We compare egoistic AVs to our altruistic autonomous agents in a highway merging setting and demonstrate the emerging behaviors that lead to improvement in the number of successful merges and the overall traffic flow and safety.
6 May 2022 in IEEE Open Journal of Intelligent Transportation Systems
Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.
Many studies in the field of robot navigation have focused on environment representation and localization. The goal of map representation is to summarize spatial information in topological and geometrical abstracts. By providing strong priors, maps improve the performance and reliability of automated robots. Due to the transition to fully automated driving in recent years, there has been a constant effort to design methods and technologies to improve the precision of road participants and the environment’s information. Among these efforts is the high-definition (HD) map concept. Making HD maps requires accuracy, completeness, verifiability, and extensibility. Because of the complexity of HD mapping, it is currently expensive and difficult to implement, particularly in an urban environment. In an urban traffic system, the road model is at least a map with sets of roads, lanes, and lane markers. While more research is being dedicated to mapping and localization, a comprehensive review of the various types of map representation is still required. This paper presents a brief overview of map representation, followed by a detailed literature review of HD maps for automated vehicles. The current state of autonomous vehicle (AV) mapping is encouraging, the field has matured to a point where detailed maps of complex environments are built in real time and have been proved useful. Many existing techniques are robust to noise and can cope with a large range of environments. Nevertheless, there are still open problems for future research. AV mapping will continue to be a highly active research area essential to the goal of achieving full autonomy.
20 March 2023 in IEEE Open Journal of Intelligent Transportation Systems
Utilizing Vehicle-to-everything (V2X) communication technologies, vehicle platooning systems are expected to realize a new paradigm of cooperative driving with higher levels of traffic safety and efficiency. Connected and Autonomous Vehicles (CAVs) need to have proper awareness of the traffic context. The cooperative platoon’s performance will be influenced by the communication strategy. In particular, time-triggered or event-triggered are of interest here. The expenses related to communication will increase significantly as the number of connected entities increases. Periodic communication can be relaxed to more flexible aperiodic or event-triggered implementations while maintaining desired levels of performance. This paper proposes a predictive model-based and control-aware communication solution for vehicle platoons. The method uses a fully distributed Event-Triggered Communication (ETC) strategy combined with Model-Based Communication (MBC) and aims to minimize communication resource usage while preserving desired closed-loop performance characteristics. In our method, each vehicle runs a remote vehicle state estimator based on the most recently communicated model and the event-driven communication scheme only updates the model when the performance metric error exceeds a certain threshold. Our approach achieves a significant reduction in the average communication rate (82%) while only slightly reducing control performance (e.g., less than 1% speed deviation).
23 October 2023 in IEEE Transactions on Intelligent Transportation Systems
Autonomous vehicle (AV) navigation in the presence of Human-driven vehicles (HVs) is challenging, as HVs continuously update their policies in response to AVs. In order to navigate safely in the presence of complex AV-HV social interactions, the AVs must learn to predict these changes. Humans are capable of navigating such challenging social interaction settings because of their intrinsic knowledge about other agents’ behaviors and use that to forecast what might happen in the future. Inspired by humans, we provide our AVs the capability of anticipating future states and leveraging prediction in a cooperative reinforcement learning (RL) decision-making framework, to improve safety and robustness. In this paper, we propose an integration of two essential and earlier-presented components of AVs: social navigation and prediction. We formulate the AV’s decision-making process as a RL problem and seek to obtain optimal policies that produce socially beneficial results utilizing a prediction-aware planning and social-aware optimization RL framework. We also propose a Hybrid Predictive Network (HPN) that anticipates future observations. The HPN is used in a multi-step prediction chain to compute a window of predicted future observations to be used by the value function network (VFN). Finally, a safe VFN is trained to optimize a social utility using a sequence of previous and predicted observations, and a safety prioritizer is used to leverage the interpretable kinematic predictions to mask the unsafe actions, constraining the RL policy. We compare our prediction-aware AV to state-of-the-art solutions and demonstrate performance improvements in terms of efficiency and safety in multiple simulated scenarios.
25 July 2023 in IEEE Open Journal of Intelligent Transportation Systems
As autonomous vehicles (AVs) become increasingly prevalent on the roads, their ability to accurately interpret and understand traffic signs is crucial for ensuring reliable navigation. While most previous research has focused on addressing specific aspects of the problem, such as sign detection and text extraction, the development of a comprehensive visual processing method for traffic sign understanding remains largely unexplored. In this work, we propose a robust and scalable traffic sign perception system that seamlessly integrates the essential sensor signal processing components, including sign detection, text extraction, and text recognition. Furthermore, we propose a novel method to estimate the sign relevance with respect to the ego vehicle, by computing the 3D orientation of the sign from the 2D image. This critical step enables AVs to prioritize the detected signs based on their relevance. We evaluate the effectiveness of our perception solution through extensive validation across various real and simulated datasets. This includes a novel dataset we created for sign relevance that features sign orientation. Our findings highlight the robustness of our approach and its potential to enhance the performance and reliability of AVs navigating complex road environments.