To address traffic congestion in urban expressways during the transition from wide to narrow sections, this study proposed a car-following strategy based on deep reinforcement learning. Firstly, a car-following strategy was developed based on a twin-delayed deep deterministic policy gradient (TD3) algorithm, and a multi-objective constrained reward function was designed by comprehensively considering safety, traffic efficiency, and ride comfort. Secondly, 214 car-following periods and 13 platoon-following periods were selected from the natural driving database for the strategies training and testing. Finally, the effectiveness of the proposed strategy was verified through simulation experiments of car-following and platoon-following. The results showed that compared to human-driven vehicles (HDV), the TD3 and deep deterministic policy gradient (DDPG)-based strategies enhanced traffic efficiency by over 29% and ride comfort by more than 60%. Furthermore, compared to DDPG, the relative errors between the following distance and desired safety distance using TD3 could be reduced by 1.28% and 1.37% in simulation experiments of car-following and platoon-following, respectively. This study provides a new approach to alleviate traffic congestion for wide-to-narrow road sections in urban expressways.

Despite the significant impact of network hyperparameters on deep learning car-following models, there has been relatively little research on network hyperparameters of deep learning car-following models. Therefore, this study proposes a car-following model that combines particle swarm optimization (PSO) and gated recurrent unit (GRU) networks. The PSO-GRU car-following model is trained and tested using data from the natural driving database. The results demonstrate that compared to the intelligent driver model (IDM) and the GRU car-following model, the PSO-GRU car-following model reduces the mean squared error (MSE) for the speed simulation of following vehicles by 88.36% and 72.92%, respectively, and reduces the mean absolute percentage error (MAPE) by 64.81% and 50.14%, respectively, indicating a higher prediction accuracy. Dataset 3 from the drone video trajectory database of Southeast University and NGSIM’s I-80 dataset are used to study the car-following model’s cross-dataset adaptability, that is, to verify its transferability. Compared to the GRU car-following model, the PSO-GRU car-following model reduces the standard deviation of the test results by 60.64% and 32.89%, highlighting its more robust prediction stability and better transferability. Verifying the ability of the car-following model to produce the stop-and-go phenomenon can evaluate its transferability more comprehensively. The PSO-GRU car-following model outperforms the GRU car-following model in creating stop-and-go sensations through platoon simulation tests, demonstrating its superior transferability. Therefore, the proposed PSO-GRU car-following model has higher prediction accuracy and cross-dataset adaptability compared to other car-following models.