Autonomous driving has been widely applied in commercial and industrial applications, along with the upgrade of environmental awareness systems. Tasks such as path planning, trajectory tracking, and obstacle avoidance are strongly dependent on the ability to perform real-time object detection and position regression. Among the most commonly used sensors, camera provides dense semantic information but lacks accurate distance information to the target, while LiDAR provides accurate depth information but with sparse resolution. In this paper, a LiDAR-camera-based fusion algorithm is proposed to improve the above-mentioned trade-off problems by constructing a Siamese network for object detection. Raw point clouds are converted to camera planes to obtain a 2D depth image. By designing a cross feature fusion block to connect the depth and RGB processing branches, the feature-layer fusion strategy is applied to integrate multi-modality data. The proposed fusion algorithm is evaluated on the KITTI dataset. Experimental results demonstrate that our algorithm has superior performance and real-time efficiency. Remarkably, it outperforms other state-of-the-art algorithms at the most important moderate level and achieves excellent performance at the easy and hard levels.

This paper presents a latent space reinforcement learning method for interpretable decision-making of autonomous vehicles at highway on-ramps. This method is based on the latent model and the combination model of the hidden Markov model and Gaussian mixture regression (HMM-GMR). It is difficult for the traditional decision-making method to understand the environment because its input is high-dimensional and lacks an understanding of the task. By utilizing the HMM-GMR model, we can obtain the interpretable state providing semantic information and environment understanding. A framework is proposed to unify representation learning with the deep reinforcement learning (DRL) approach, in which the latent model is used to reduce the dimension of interpretable state by extracting underlying task-relevant information. Experimental results are presented and the results show the right balance between driving safety and efficiency in the challenging scenarios of highway on-ramps merging.

Merging at highway on-ramps while interacting with other human-driven vehicles is challenging for autonomous vehicles (AVs). An efficient route to this challenge requires exploring and exploiting knowledge of the interaction process from demonstrations by humans. However, it is unclear what information (or environmental states) is utilized by the human driver to guide their behavior throughout the whole merging process. This paper provides quantitative analysis and evaluation of the merging behavior at highway on-ramps with congested traffic in a volume of time and space. Two types of social interaction scenarios are considered based on the social preferences of surrounding vehicles: courteous and rude . The significant levels of environmental states for characterizing the interactive merging process are empirically analyzed based on the real-world INTERACTION dataset. Experimental results reveal two fundamental mechanisms in the merging process: 1) Human drivers select different states to make sequential decisions at different moments of task execution; and 2) the social preference of surrounding vehicles can impact variable selection for making decisions. It implies that efficient decision-making design should filter out irrelevant information while considering social preference to achieve comparable human-level performance. These essential findings shed light on developing new decision-making approaches for AVs.

In this paper, we focus on the problem of highway merge via parallel-type on-ramp for autonomous vehicles (AVs) in a decentralized non-cooperative way. This problem is challenging because of the highly dynamic and complex road environments. A deep reinforcement learning-based approach is proposed. The kernel of this approach is a Deep Q-Network (DQN) that takes dynamic traffic state as input and outputs actions including longitudinal acceleration (or deceleration) and lane merge. The total reward for this on-ramp merge problem consists of three parts, which are the merge success reward, the merge safety reward, and the merge efficiency reward. For model training and testing, we construct a highway on-ramp merging simulation experiments with realistic driving parameters. The experimental results show that the proposed approach can make reasonable merging decisions based on the observation of the traffic environment. We also compare our approach with a state-of-the-art approach and the superior performance of our approach is demonstrated by making challenging merging decisions in complex highway parallel-type on-ramp merging scenarios.

In this study, a deep reinforcement learning approach is proposed to handle tactical driving in complex highway traffic environments for unmanned ground vehicles. Tactical driving is a challenging topic for unmanned ground vehicles because of its interplay with routing decisions as well as real-time traffic dynamics. The core of our deep reinforcement learning approach is a deep Q-network that takes dynamic traffic information as input and outputs typical tactical driving decisions as action. The reward is designed with the consideration of successful highway exit, average traveling speed, and driving safety and comfort. In order to endow an unmanned ground vehicle with situational traffic information that is critical for tactical driving, the vehicle’s sensor information such as vehicle position and velocity are further augmented through the assessment of the ego-vehicle’s collision risk, potential field, and kinematics and used as input for the deep Q-network model. A convolutional neural network is built and fine-tuned to extract traffic features which facilitate the decision-making process of Q-learning. For model training and testing, a highway simulation platform is constructed with realistic parameter settings obtained from a real-world highway traffic dataset. The performance of the deep Q-network model is validated with extensive simulation experiments under different parameter settings such as traffic density and risk level. The results exhibit the important potentials of our deep Q-network model in learning challenging tactical driving decisions given multiple objectives and complex traffic environment.

Humans make daily routine decisions based on their internal states in intricate interaction scenarios. This article presents a probabilistically reconstructive learning approach to identify the internal states of multivehicle sequential interactions when merging at highway on-ramps. We treated the merging task’s sequential decision as a dynamic, stochastic process and then integrated the internal states into a hidden Markov model (HMM)-Gaussian mixture regression (GMR) model, a probabilistic combination of an extended GMR and HMM. We also developed a variant of the expectation–maximization (EM) algorithm to estimate the model parameters and verified it based on a real-world dataset. Experiment results reveal that three interpretable internal states can semantically describe the interactive merge procedure at highway on-ramps. This finding provides a basis for developing an efficient model-based decision-making algorithm for autonomous vehicles (AVs) in a partially observable environment. Note to Practitioners—Model-based learning approaches have obtained increasing attention in decision-making design due to their stability and interpretability. This article was built upon two facts: 1) intelligent agents can only receive partially observable environmental information directly through their equipped sensors in the real world and 2) humans mainly utilize the internal states and associated dynamics inferred from observations to make proper decisions in complex environments. Similarly, autonomous vehicles (AVs) need to understand, infer, anticipate, and exploit the internal states of dynamic environments. Applying probabilistic decision-making models to AVs requires updating the internal states’ beliefs and associated dynamics after getting new observations. The designed and verified emission model in hidden Markov model (HMM)-Gaussian mixture regression (GMR) provides a modifiable functional module for online updates of the associated internal states. Experiment results based on the real-world driving dataset demonstrate that the internal states extracted using HMM-GMR can represent the dynamic decision-making process semantically and make an accurate prediction.