6 April 2022 in IEEE Transactions on Intelligent Transportation Systems
Road shoulders and slopes often appear in unstructured environments. They make 2.5D vehicle trajectory planning commonly seen in our daily life, which lies on a 2D manifold embedded in a 3D space. The height difference of these terrains brings spatially dependent constraints on vehicle maneuvers, such as the limit on vehicle steering for vehicle tire protection when a vehicle approaches a road shoulder edge. These constraints have an “if-else” structure since they are activated only when the vehicle passes through the local area with a height difference, making the restriction on variables coupled with the judgment of variables. This makes the application of state-of-art optimization-based planners challenging. To solve this problem, we devise an approximation formulation for these constraints in the trajectory planning optimization problem, whose solution depends on a proper initial guess for the optimizer. We propose a two-stage trajectory planning framework, where the first stage improves the hybrid A* algorithm by adding spatially dependent constraints into node expansion to provide the initial guess. Then, the optimization problem with the formulated spatially dependent constraints is solved for further trajectory smoothness and quality. Finally, the simulation results validate the fast and high-quality planning performance of our proposed framework.
29 March 2022 in IEEE Transactions on Intelligent Vehicles
Optimization-based trajectory planners for an autonomous vehicle have received increasing attention due to their convenience for handling various complex constraints. However, their formulated trajectory planning optimization problems in unstructured environments are highly non-convex and nonlinear, requiring a proper initial guess to help with convergence. Given the low efficiency of common sampling-and-search-based initialization strategies due to the dimensionality brought by complex vehicle dynamics and the maneuverable space narrowed by cluttered obstacles, the sequential optimization of a series of softened intermediate problems is emerging as a new initialization strategy. Unlike sequential optimization-based strategies, which optimize all decision variables (w.r.t. the positions, velocities, vehicle orientations, steering angles, and so on of all waypoints) for initialization, the proposed strategy only optimizes critical variables, whose values should help satisfy the inequalities strictly with a small equation violation. Thus, the original scheme for initialization is decoupled into two sub-tasks with down-sized variables. One reduces the nonlinear equation violation, and the other ensures strict inequality satisfaction. Experimental results show that the proposed initialization strategy can facilitate the solution of an optimization-based trajectory planner, thereby making the trajectory planner outperform the state-of-the-art competitors in this community.
25 April 2022 in IEEE Transactions on Neural Networks and Learning Systems
Undiscounted return is an important setup in reinforcement learning (RL) and characterizes many real-world problems. However, optimizing an undiscounted return often causes training instability. The causes of this instability problem have not been analyzed in-depth by existing studies. In this article, this problem is analyzed from the perspective of value estimation. The analysis result indicates that the instability originates from transient traps that are caused by inconsistently selected actions. However, selecting one consistent action in the same state limits exploration. For balancing exploration effectiveness and training stability, a novel sampling method called last-visit sampling (LVS) is proposed to ensure that a part of actions is selected consistently in the same state. The LVS method decomposes the state-action value into two parts, i.e., the last-visit (LV) value and the revisit value. The decomposition ensures that the LV value is determined by consistently selected actions. We prove that the LVS method can eliminate transient traps while preserving optimality. Also, we empirically show that the method can stabilize the training processes of five typical tasks, including vision-based navigation and manipulation tasks.