Autonomous driving technologies have received notable attention in the past decades. In autonomous driving systems, identifying a precise dynamical model for motion control is nontrivial due to the strong nonlinearity and uncertainty in vehicle dynamics. Recent efforts have resorted to machine learning techniques for building vehicle dynamical models, but the generalization ability and interpretability of existing methods still need to be improved. In this paper, we propose a pure data-driven vehicle modeling approach based on deep neural networks with an interpretable Koopman operator. The main advantage of using the Koopman operator is to represent the nonlinear dynamics in a linear lifted feature space. In the proposed approach, a deep learning-based extended dynamic mode decomposition algorithm is presented to learn a finite-dimensional approximation of the Koopman operator. A multi-step prediction loss function is used in the training process, enabling a long-term prediction capability. Furthermore, a data-driven model predictive controller with the learned Koopman model is designed for velocity profile tracking control of autonomous vehicles. Simulation results in a high-fidelity CarSim environment show that our approach outperforms previously developed traditional and advanced modeling methods. Velocity profile tracking tests of the autonomous vehicle are also performed in the CarSim environment. The results show that our approach has better tracking accuracy and higher computational efficiency than the model predictive control algorithms using a nonlinear model and a linear time-varying model.

Robust model predictive control (MPC) is a well-known control technique for model-based control with constraints and uncertainties. In classic robust tube-based MPC approaches, an open-loop control sequence is computed via periodically solving an online nominal MPC problem, which requires prior model information and frequent access to onboard computational resources. In this article, we propose an efficient robust MPC solution based on receding horizon reinforcement learning, called r-LPC, for unknown nonlinear systems with state constraints and disturbances. The proposed r-LPC utilizes a Koopman operator-based prediction model obtained offline from precollected input–output datasets. Unlike classic tube-based MPC, in each prediction time interval of r-LPC, we use an actor–critic structure to learn a near-optimal feedback control policy rather than a control sequence. The resulting closed-loop control policy can be learned offline and deployed online or learned online in an asynchronous way. In the latter case, online learning can be activated whenever necessary; for instance, the safety constraint is violated with the deployed policy. The closed-loop recursive feasibility, robustness, and asymptotic stability are proven under function approximation errors of the actor–critic networks. Simulation and experimental results on two nonlinear systems with unknown dynamics and disturbances have demonstrated that our approach has better or comparable performance when compared with tube-based MPC and linear quadratic regulator, and outperforms a recently developed actor–critic learning approach.

Path tracking control of intelligent vehicles has to deal with the difficulties of model uncertainties and nonlinearities. As a class of adaptive optimal control methods, reinforcement learning (RL) has received increasing attention in solving difficult control problems. However, feature representation and online learning ability are two major problems to be solved for learning control of uncertain dynamic systems. In this article, we propose a multi-kernel online RL approach for path tracking control of intelligent vehicles. In the proposed approach, a multiple kernel feature learning framework is designed for online learning control based on dual heuristic programming (DHP) and the new online learning control algorithm is called multi-kernel DHP (MKDHP). In MKDHP, instead of the expert knowledge for selecting and fine-tuning of a suitable kernel function, only a set of basic kernel functions is required to be predefined and the multi-kernel features can be learned for value function approximation in the critic. The simulation studies on path tracking control for intelligent vehicles have been conducted under $S$ -curve and urban road conditions. The results demonstrated that compared with other typical path tracking controllers for intelligent vehicles, such as the linear quadratic regulator (LQR), the pure pursuit controller and the ribbon-based controller, the proposed multi-kernel learning controller can achieve better performance in terms of tracking precision and smoothness.

Kinodynamic motion planning is critical for autonomous vehicles with high maneuverability in dynamic environments. However, obtaining near-optimal motion planning solutions with low computational costs and inaccurate prior model information is challenging. To address this issue, this paper proposes a receding-horizon reinforcement learning approach for kinodynamic motion planning (RHRL-KDP) of autonomous vehicles in the presence of inaccurate dynamics information and moving obstacles. Specifically, a receding-horizon actor-critic reinforcement learning algorithm is presented, resulting in a neural network-based planning strategy that can be learned both offline and online. A neural network-based model is built and learned online to approximate the modeling uncertainty of the prior nominal model in order to improve planning performance. Furthermore, active collision avoidance in dynamic environments is realized by constructing safety-related terms in actor and critic networks using potential fields. In theory, the uniformly ultimate boundedness property of the modeling uncertainty’s approximation error is proven, and the convergence of the proposed RHRL-KDP is analyzed. Simulation tests show that our approach outperforms the previously developed motion planners based on model predictive control (MPC), safe RL, and RRT ${^\star }$ in terms of planning performance. Furthermore, in both online and offline learning scenarios, RHRL-KDP outperforms MPC and RRT $^{\star }$ in terms of computational efficiency.

In recent years, safe reinforcement learning (RL) with the actor-critic structure has gained significant interest for continuous control tasks. However, achieving near-optimal control policies with safety and convergence guarantees remains challenging. Moreover, few works have focused on designing RL algorithms that handle time-varying safety constraints. This article proposes a safe RL algorithm for optimal control of nonlinear systems with time-varying state and control constraints. The algorithm's novelty lies in two key aspects. Firstly, the approach introduces a unique barrier force-based control policy structure to ensure control safety during learning. Secondly, a multistep policy evaluation mechanism is employed, enabling the prediction of policy safety risks under time-varying constraints and guiding safe updates. Theoretical results on learning convergence, stability, and robustness are proven. The proposed algorithm outperforms several state-of-the-art RL algorithms in the simulated Safety Gym environment. It is also applied to the real-world problem of integrated path following and collision avoidance for two intelligent vehicles—a differential-drive vehicle and an Ackermann-drive one. The experimental results demonstrate the impressive sim-to-real transfer capability of our approach, while showcasing satisfactory online control performance.