Autonomous driving involves multi-timescale and multi-objective tasks coupled with long-term driving decision and short-term motion planning. However, existing studies tend to investigate them separately or assume they were synchronous and instantaneous, failing to deeply interpret their interconnections in autonomous driving. To address it, this study explored driving decision and motion planning in an integrated manner, and proposed a double-layer reinforcement learning (RL) framework to couple and optimize these two modules. Specifically, on the upper layer of the framework, a trajectory-level reward function associated with safety, efficiency, comfort and decision execution was proposed, and a decision-making model was established based on value function-based deep reinforcement learning (DRL) algorithms. Then, the reward function of the upper layer was taken as the objective function of the lower layer with relevant state constraints. The model predictive control (MPC) method was used to derive the optimal maneuver sequence guided by driving decisions, and the performance evaluation of motion planning was fed back to the decision-making for DRL parameter optimization. Accordingly, a closed-loop mechanism was developed consisting of forward decision-making output guidance and backward motion planning feedback optimization. Then, two-lane and three-lane interactive scenarios were set for framework training and testing. Last, comparative experiments were conducted using NGSIM dataset. The results demonstrated the effectiveness and the enhanced driving performance of the proposed framework in contrast to four benchmarks.

Trajectory planning for autonomous driving is challenging in unstructured scenarios such as mining sites. Existing studies mainly resort to a heuristic search-based planner to find a feasible trajectory. In the case of the narrow area, the heuristic function of the planner suffers from over-expansion problems, which may result in heavy computation burden and memory usage as well as potential failure. Therefore, a novel trajectory planning approach, namely learning and optimization-based trajectory planning (LOTP), is proposed, which is featured by a hierarchical structure consisting of two modules: (1) path searching, (2) speed profile generation. Firstly, a path searching method based on deep learning and Monte-Carlo tree search is proposed to generate a coarse path connecting starting and terminal points. Then, the path is smoothed using path optimization and provided to the speed-planning module as the reference. Next, a speed planning method based on quadratic optimization is developed, which allows to seek maximum driving comfort and energy saving. Last, extensive simulation experiments were conducted in the real environment of mining sites. The results verified that LOTP enhances the computational efficiency and success rate of path planning and helps generate an optimal speed profile. Furthermore, LOTP exhibits desirable potential for the practical application of autonomous driving at mining sites. Source implementation will be released as an open-source code.

Maneuver decision-making and trajectory planning play important roles in autonomous driving since a safe and flexible decision module is indispensable for navigation. Typical algorithms apply sampling methods to generate feasible trajectories. However, the fixed sampling distance and maneuver execution time in a sampling approach sacrifice the flexibility of algorithm. Moreover, since motion planning can be represented as a high-dimensional problem, it usually results in unnecessary samples that require additional resources to search the solution. Therefore, a semantic-level maneuver sampling and trajectory planning algorithm is proposed to solve the above problems. In the upper-level maneuver decision, the decision-making problem is formulated as a selection of the forward leading object. A semantic-level decision tree is built to sample long-term maneuver sequences, and the safety corridor of each maneuver sequence is calculated according to the surrounding environment. In the lower-level trajectory planning, the process is decoupled into longitudinal and lateral directions. First, a heuristic search method is proposed to generate longitudinal trajectory candidates for each maneuver sequence. Then, an exhaustive search algorithm is employed to synchronously generate the lateral trajectory within safety corridor. Among the generated trajectory candidates, the one with minimum cost will be chosen as the searching result. Furthermore, in order to improve the driving comfort, numerical optimization is adopted to refine the result by accounting for the constraints of kinematics and safety. Finally, the proposed method was evaluated through simulations of typical on-road dynamic scenarios, which help verify its performance with desirable computation efficiency of less than 32 ms.

Lateral lane-changing and longitudinal car-following behavior decision-making is crucial for the implementation of autonomous driving (AD) in complex and dynamic traffic environment. Reinforcement learning (RL) has been increasingly applied to these behaviors due to its powerful environmental adaptability. However, typical RL-based research focused solely on one directional decision-making or output discrete actions in both lateral and longitudinal directions, which is not sufficiently flexible for AD. In view of the close interaction between lane-changing and car-following tasks, this study proposed an integrated framework to realize coupled decision-making using RL. Specifically, double deep Q-network (DDQN) was used to make the lateral lane change command decision-making, and twin delay deep deterministic policy gradient (TD3) used to make longitudinal car-following acceleration decision-making. The integration of DDQN and TD3 was deployed to tackle the problem of discrete-continuous hybrid action space in RL. The lateral decision-making was integrated into the actor-critic network for longitudinal decision-making to generate accurate acceleration in lane-changing or lane-keeping conditions. A series of simulation experiments were conducted in typical two-lane straight driving scenarios, validating the feasibility and applicability of the proposed integrated framework. The results reveal that the RL-based coupled decision-making helps enhance the implementation of AD.

The bottleneck of trajectory planning for autonomous driving at mining site has shifted from single truck to multi trucks, particularly in the loading area where collaborative trajectory planning (CTP) in the loading area is crucial for efficient operation. The existing optimization-based methods that integrate all vehicle states and fail to generate the trajectories according to real time requirements. Other station-time graph-based method have significant limitations as they rely on the prior reference path and only adjust the speed of trucks but often incur infeasible trajectory when the scale of the trucks increases, causing traffic deadlock even collision. To facilitate a more productive CTP, we propose an efficient method by combing heuristic search and group priority optimization-based approach that strives to realize the efficient generation of safe trajectories. The proposed method essentially integrates two radical trajectory planners: i) heuristic search trajectory planning and ii) advanced sequential optimization method, and introduces a novel adaptive group priority based on the characteristics of mining site for easing the computational time of sequential optimization. Extensive simulation results demonstrate that the proposed method can resolve the collision between trucks and reduce the waiting time of trucks while overcoming excessive time consumption of CTP