3 September 2025 in IEEE Transactions on Intelligent Transportation Systems
Reinforcement learning (RL) offers a powerful paradigm for adaptive motion planning in complex driving environments. However, applying RL to autonomous driving remains challenging due to uncertainty from partial observability and the stochastic, multimodal behaviors of traffic participants. This paper presents a novel risk-aware RL framework for non-conservative motion planning under uncertainty. By integrating Partially Observable Markov Decision Processes (POMDP) with a deep RL-based policy optimization scheme, the proposed approach explicitly models aleatoric uncertainty via a Gaussian Mixture Bayesian Belief Updater and a time-varying risk field. Additionally, an Adaptive Context-aware Attention (ACA) module is employed to prioritize critical targets for enhanced interaction modeling dynamically. Extensive experiments on the CARLA simulator show that the framework generalizes well across diverse traffic conditions, improving average reward by 65.74% and 64.02% in low-speed dense and high-speed sparse scenarios. It remains robust in challenging situations such as overtaking and sudden lane changes in the PeMS dataset. Furthermore, distributed deployment tests confirm a real-time performance of 10 Hz on a hardware-in-the-loop platform, demonstrating the feasibility of practical deployment.
22 November 2024 in Advanced Engineering Informatics
25 April 2025 in IEEE Transactions on Intelligent Transportation Systems
Autonomous driving has become an inevitable trend in automotive development. Reinforcement Learning (RL) is extensively used in autonomous vehicle motion planning, demonstrating good generalization but facing challenges of long training times and lack of consideration of uncertainty. To address these challenges, an Adaptive Heuristic Reinforcement Learning (AHRL) approach is proposed. First, this study improves upon the Dueling Double Deep Q Network (D3QN) algorithm by proposing the Adaptive Heuristic Dueling Double Deep Q Network (Adapt-HD3QN) algorithm. Specifically, heuristic functions from search-based planning algorithms are incorporated into the RL reward terms to guide heuristic learning and enhance learning efficiency. Additionally, considering the uncertainties in real-world driving environments, such as the movement of other traffic participants and building occlusions, a Mixed Artificial Potential Field (Mix-APF) is implemented to address static and dynamic obstacle avoidance. Furthermore, potential collisions between the autonomous vehicle and other vehicles in occluded areas are modeled as a zero-sum game, with a Dynamic Bayesian Network (DBN) used for prior modeling of potential vehicles, aiding in constructing the potential vehicle’s forward hidden set. Finally, a signal-free intersection scenario, a typical crash-prone road type, is constructed on CARLA, incorporating static obstacles and other traffic participants. Experimental results demonstrate that the proposed Adapt-HD3QN algorithm exhibits superior safety, training efficiency, and traffic efficiency in scenarios with road environment perception uncertainties caused by occlusion.
23 January 2026 in IEEE Transactions on Intelligent Vehicles
Path planning and tracking of terrain vehicles are key to navigation safety in the field considering complex working conditions such as uncertain rough surfaces and deformable soil. This paper proposes a hierarchical path-planning and tracking framework to address inherent stochasticity and safety-critical problems in deformable terrain navigation. Firstly, a safe planning method based on distributional reinforcement learning is proposed, where route safety is strengthened by Conditional Value at Risk (CVaR) optimization based on terrain risk evaluation and constraints. Then, the terramechanics establishes the dynamics model for terrain vehicles driving on soft surfaces. A tracking controller based on the Adaptive Prescribed Performance Sliding Mode Control (APPSMC) is developed, where an improved prescribed performance function and a Fuzzy Logic System (FLS) are introduced to estimate the ground vehicle dynamics as well as the environmental disturbances. Simulation conducted on high-fidelity terrain environments shows satisfactory planning and tracking performances. It exhibits engineering transferability for autonomous operations such as planetary exploration and precision agriculture, providing modular design for seamless integration into off-road robotic platforms, and safety-critical tasks in unpredictable, unstructured environments.
1 October 2026 in IEEE Transactions on Intelligent Transportation Systems