Dynamic path planning enables vehicles to autonomously navigate in unknown or continuously changing environments, thereby reducing reliance on fixed maps. Deep reinforcement learning (DRL), with its superior performance in handling high-dimensional state spaces and complex dynamic environments, has been widely applied to dynamic path planning. Traditional DRL methods are prone to capturing unnecessary noise information and irrelevant features during the training process, leading to instability and decreased adaptability of models in complex dynamic environments. To address this challenge, we propose a dynamic path-planning method based on our Causal State-Masking Twin-delayed Deep Deterministic Policy Gradient (CSM-TD3) algorithm. CSM-TD3 integrates a causal inference mechanism by introducing dynamic state masks and intervention mechanisms, allowing the policy network to focus on genuine causal features for decision optimization and thereby enhancing the convergence speed and generalization capabilities of the agent. Furthermore, causal state-masking DRL allows the system to learn the optimal mask configurations through backpropagation, enabling the model to adaptively adjust the causal features of interest. Extensive experimental results demonstrate that this method significantly enhances the convergence of the TD3 algorithm and effectively improves its performance in dynamic path planning.
We propose a compound control framework to improve the path tracking accuracy of a four-wheel independent steering and driving (4WISD) vehicle in complex environments. The framework consists of a deep reinforcement learning (DRL)-based auxiliary controller and a dual-layer controller. Samples in the 4WISD vehicle control framework have the issues of skewness and sparsity, which makes it difficult for the DRL to converge. We propose a group intelligent experience replay (GER) mechanism that non-dominantly sorts the samples in the experience buffer, which facilitates within-group and between-group collaboration to achieve a balance between exploration and exploitation. To address the generalization problem in the complex nonlinear dynamics of 4WISD vehicles, we propose an actor-critic architecture based on the method of two-stream information bottleneck (TIB). The TIB method is used to remove redundant information and extract high-dimensional features from the samples, thereby reducing generalization errors. To alleviate the overfitting of DRL to known data caused by IB, the reverse information bottleneck (RIB) alters the optimization objective of IB, preserving the discriminative features that are highly correlated with actions and improving the generalization ability of DRL. The proposed method significantly improves the convergence and generalization capabilities of DRL, while effectively enhancing the path tracking accuracy of 4WISD vehicles in high-speed, large-curvature, and complex environments.