Deep reinforcement learning (DRL) enables autonomous vehicles to perform complex decision making using neural networks. However, previous DRL networks only output decisions, so there is no way to determine whether the decision is proper. Reinforcement learning agents may continue to produce wrong decisions in unexpected environments not encountered during the learning process. In particular, one wrong decision can lead to an accident in autonomous driving. Therefore, it is necessary to indicate whether the action is a reasonable decision. As one such method, uncertainty can inform whether the agent’s decision is appropriate for practical application where safety must be guaranteed. Therefore, this paper provides uncertainty in the decision by proposing DeepSet-Q with Gaussian mixture (DwGM-Q), which converges the existing DeepSet-Q and mixture density network (MDN). Calculating uncertainty with the Gaussian mixture model (GMM) produced from MDN made it possible to calculate faster than the existing ensemble method. Moreover, it verified how the agent responds to the unlearned situation through the Simulation of Urban MObility (SUMO) simulator and compared the uncertainty of the decision between the learned and nontrained situation.

Model Predictive Control (MPC) is a widely used optimal control strategy, particularly effective in managing complex constraints. It excels at optimizing performance within feasible limits, such as minimizing lap times in vehicle racing. However, its effectiveness can be hindered by computational burdens and model inaccuracies. To address these challenges, MPC problems are often simplified, and parameters are manually tuned through iterative adjustments—a process that is both time-consuming and labor-intensive. This paper introduces a Reinforcement Learning-based Speed Profile Model Predictive Control (RLSP-MPC) approach, which leverages deep reinforcement learning (DRL) to approximate global optima via an exploration-exploitation framework, eliminating the need for manual tuning. The RLSP-MPC method reformulates the traditional MPC speed optimization task into a DRL-enhanced speed profile tracking problem, reducing parameter dependency and improving lap times. The DRL network provides boundary constraints and a reference velocity profile to MPC, which then reconstructs the trajectory while adhering to these constraints. Polynomial functions are employed to reduce the dimensionality of the network output in velocity profile design. Extensive experiments conducted using the F1Tenth Simulator confirm that the proposed method effectively integrates the strengths of MPC and DRL, achieving superior lap times and enhanced trajectory tracking performance while maintaining real-time computational feasibility. Specifically, the proposed method reduces lap times by an average of approximately 22.07% compared to conventional MPC and 14.52% compared to LMPC across multiple racing tracks, demonstrating consistent and statistically significant performance improvements.

Incorrect autonomous driving decisions on highways can lead to traffic congestion and accidents. Therefore, accurate decision- making in highways is essential. However, decision-making in highways is a challenging task due to the complexity of traffic situations. Thus, reinforcement learning is an appropriate approach for decision-making in highways. However, a domain gap often arises between the training and testing environments due to differences in them. Therefore, a curriculum-based domain randomization approach is applied, which progressively adjusts the level of random parameters. The RL model is trained using the SUMO simulator and tested in the MORAI simulator, where white noise is added to simulate real-world conditions. Consequently, applying curriculum domain randomization has the lowest collision rate.