Merge scenarios on highway are often challenging for autonomous driving, due to its lack of sufficient tacit understanding on and subtle interaction with human drivers in the traffic flow. This, as a result, may impose serious safety risks, and often cause traffic jam with autonomous driving. Therefore, human-like autonomous driving becomes important, yet imperative. This paper presents an interaction-aware decision-making and planning method for human-like autonomous driving in merge scenarios. Rather than directly mimicking human behavior, deep inverse reinforcement learning is employed to learn the human-used reward function for decision-making and planning from naturalistic driving data to enhance interpretability and generalizability. To consider the interaction factor, the reward function for planning is utilized to evaluate the joint trajectories of the autonomous driving vehicle (ADV) and traffic vehicles. In contrast to predicting trajectories of traffic vehicles with the fixed trajectory of ADV given by the upstream prediction model, the trajectories of traffic vehicles are predicted by responding to the ADV's behavior in this paper. Additionally, the decision-making module is employed to reduce the solution space of planning via the selection of a proper gap for merging. Both the decision-making and planning algorithms follow a “sampling, evaluation, and selection” framework. After being verified through experiments, the results indicate that the planned trajectories with the presented method are highly similar to those of human drivers. Moreover, compared to the interaction-unaware planning method, the interaction-aware planning method behaves closer to human drivers.

The steering feedback torque (SFT) is a key part of driving simulator and steer-by-wire system, which provides driver with desired road feel and vehicle motion dynamics. Therefore, accurately modeling of SFT is of great significance for driver to get better steering feel. Since SFT can be affected by many linear or nonlinear factors, it is appropriate to model SFT using data-driven method. In this article, we adopt artificial neural network (ANN) and Gaussian process regression (GPR) to build the SFT model, and analyze the performance. Considering the fact that the contributing factors for SFT may vary under different driving conditions, we employ K-Means to precluster the training dataset to improve the model accuracy. The model training and validation processes are mainly data-driven, and the results show that GPR and ANN can achieve similar prediction accuracy with the mean square error to be around 0.10. Since the GPR model can be trained much faster than ANN model, it is more suitable for real-time application. It is further demonstrated that using preclustered data based on K-Means for model training can significantly improve its accuracy without sacrificing its computational efficiency.

Traditional modeling methods of steering feedback torque (SFT) widely used in driving simulator are mainly physics-based, but suffers from high model complexity and low model accuracy due to the inaccurate or unknown structure, dynamics, and related parameters of the steering system, such as backlash or dry friction. For these problems, data-driven methods were considered to have advantageous in modeling such a complex yet dynamic system. However, due to the limitation of the data collected for model training from real vehicle test, the commonly used artificial neural network (ANN) cannot fully reflect the steering dynamics, and does not work well in many cases where collected data do not cover, and thus lead to poor generalization performance. In this article, a long short-term memory (LSTM) model is proposed and proved to simulate well on the dynamics of SFT, and works well in most driving conditions. A driving simulator is developed to verify the performance of different models in terms of generalization of working conditions. Based on the driving simulator data, we found that ANN failure generally arises in the transition between conventional and unconventional operating conditions, which is recognized by the proposed driving pattern recognition method. The LSTM-based modeling method can effectively solve this problem. The comparative analysis reveals that the improved LSTM model can simulate the SFT with higher accuracy and broader adaptability to driving simulator conditions.

Modeling driving behavior plays a pivotal role in advancing the development of human-like autonomous driving. In light of this, this paper proposes a car-following behavior modeling method with sample-based deep inverse reinforcement learning (DIRL). Due to the challenges associated with feature extraction and the limited fitting capacity of linear functions, traditional IRL, which employs feature-based linear functions to represent reward functions, exhibits low modeling accuracy. Accordingly, DIRL leverages deep neural networks to represent reward functions. However, the requirement for reinforcement learning to determine the optimal policy for DIRL's reward function makes the training and inference processes computationally resource-intensive and inefficient. To address this issue, this paper proposes the sample-based DIRL. Through solution space discretization, sample-based DIRL streamlines the integration calculation of the partition function into a summation, resulting in improved computational efficiency. Specifically, it is a three-stage framework: sampling candidate trajectories, evaluating candidate trajectories, and selecting the trajectory with the highest reward. In order to evaluate DIRL at both the level of driving behavior and the reward function, the MPC-based virtual driver with the explicit reward function is utilized to collect driving data for training and assessing the convergence of the learned reward function. The experimental results confirm that the proposed method can accurately model the car-following behavior, and acquire the driver's reward function from the driving data.

Steering feel holds a main role in driving, making steering feedback torque (SFT) generation an important research of steer-by-wire (SBW) system. Authentic steering feel provides sensations comparable to conventional steering systems, while the SFT model aligns with driver preferences, reducing cognitive load and fatigue, thereby improving vehicle safety. Current research on driver-preferred SFT is limited to real-world trials, a method often time-intensive and suboptimal. This paper presents an innovative approach for obtaining optimal SFT with reinforcement learning (RL) guided by human reward model. The human reward model, trained on human feedback data, is introduced to guide SFT model’s optimization. Given complexity of assessing human evaluations, the reward model is validated with virtual data. Building upon previous data-driven SFT modeling, a fine-tuning method leveraging the RL method is presented, using human reward model’s outputs. This approach retains the nonlinear predictive capabilities of the pre-trained model while aligning with human evaluations. Results confirm the human reward model’s ability to replicate human SFT standards. Models adapted using this network retain the nonlinear predictive capabilities of the initial network, even under high-frequency inputs. Overall, this work pioneers the integration of human feedback into SFT optimization, providing novel insights for the advancement of SBW systems.

Accurately modeling steering feedback torque (SFT) is crucial for the performance of both steer-by-wire systems and driving simulators of vehicles. Physics-based methods have poor model accuracy and real-time performance, due to the model complexity and unknown or inaccurate model parameters. Therefore data-driven methods have gained increasing attention in recent years for their advantages on SFT modeling. However, developing satisfactory data-driven models requires a large amount of data with sufficient coverage on various driving conditions, which can be time-consuming and expensive. To address this issue, this paper proposes an innovative generalization method that can transfer an SFT model trained for a specific vehicle to other target vehicles, dramatically reducing the dependence on large amounts of data and the coverage of various driving conditions, thus, saving time and cost. First, a pre-trained SFT model with high accuracy and strong generalization capability is built based on large, full-coverage data from source domain vehicles. Then, a network is designed to fine-tune the pre-trained model with a small amount of data from target domain vehicles with only a few driving conditions. To identify the most suitable driving conditions for training the fine-tuning network, the performance of five networks trained by different driving conditions is compared. Finally, a control network with the same structure as the fine-tuned network is trained based on the same target domain data, and the model accuracy of the transfer network and the controlled network under different conditions is compared and analyzed. The proposed transfer learning method along with the designed transfer network is proved to be effective on improving prediction accuracy of the target domain by 58%–74.7% compared to the pre-trained network.