Predicting vulnerableroad user behavior is an essential prerequisite for deploying Automated Driving Systems (ADS) in the real-world. Pedestrian crossing intention should be recognized in real-time, especially for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. However, these models are not robust and certain issues still need to be resolved. First, the global spatio-temporal context that accounts for the interaction between the target pedestrian and the scene has not been properly utilized. Second, the optimal strategy for fusing different sensor data has not been thoroughly investigated. This work addresses the above limitations by introducing a novel neural network architecture to fuse inherently different spatio-temporal features for pedestrian crossing intention prediction. We fuse different phenomena such as sequences of RGB imagery, semantic segmentation masks, and ego-vehicle speed in an optimal way using attention mechanisms and a stack of recurrent neural networks. The optimal architecture was obtained through exhaustive ablation and comparison studies. Extensive comparative experiments on the JAAD and PIE pedestrian action prediction benchmarks demonstrate the effectiveness of the proposed method, where state-of-the-art performance was achieved. Our code is open-source and publicly available: https://github.com/OSU-Haolin/Pedestrian_Crossing_Intention_Prediction .

Social distancing (SD) is an effective measure to prevent the spread of the infectious Coronavirus Disease 2019 (COVID-19). However, a lack of spatial awareness may cause unintentional violations of this new measure. Against this backdrop, we propose an active surveillance system to slow the spread of COVID-19 by warning individuals in a region-of-interest. Our contribution is twofold. First, we introduce a vision-based real-time system that can detect SD violations and send non-intrusive audio-visual cues using state-of-the-art deep-learning models. Second, we define a novel critical social density value and show that the chance of SD violation occurrence can be held near zero if the pedestrian density is kept under this value. The proposed system is also ethically fair: it does not record data nor target individuals, and no human supervisor is present during the operation. The proposed system was evaluated across real-world datasets.

This paper focuses on the problem of autonomous valet parking trajectory planning in complex environments. The task normally can be well described by an optimal control problem (OCP) for the rapid, accurate, and optimal trajectory generation. Appropriate initial guesses obtained by sampling or searching-based methods are important for the numerical optimization procedure. Still, in highly complex environments with narrow passages, it may incur high computational costs or fail to find the proper initial guess. To address this challenge, an enhanced hybrid A* (EHA) algorithm is proposed to address the issue. The EHA includes four steps. The first step is to quickly obtain the global coarse trajectory using a traditional A* search. The second step is constructing a series of driving corridors along the rough trajectory, then evaluating and extracting nodes from each wide or narrow passage based on the length of the box's side. The third step is to extract each passage's boundary points. The final step is connecting boundary points by hybrid A* and generating a feasible initial guess for OCP. To reduce safety risks, vehicles in particular areas (Fig. 1b) should travel as slowly as possible. The global and local speed restrictions are distinct, and local restrictions are only activated when the vehicle enters a particular area. This “if-else” structure makes the optimization problem difficult. A novel approximation formulation for these local state constraints is introduced to overcome this issue. The experimental results demonstrate that the proposed method for trajectory planning is effective and robust.

Pedestrian motion modeling in mixed traffic scenarios is crucial to the development of autonomous systems in transportation related applications. This work investigated how pedestrian motion is affected by surrounding pedestrians and vehicles, i.e., vehicle-pedestrian interaction. A social force based pedestrian motion model was proposed, in which the effect of surrounding pedestrians was improved and the effect of vehicles was newly designed. Variable constraints dependent on vehicle influence as well as nearby pedestrian density were imposed on the velocity and acceleration of the pedestrian motion. This work focuses on fundamental patterns of multi-pedestrian interaction with a low speed vehicle (front, back, and lateral interaction in open space). In other words, the application of the model is not restricted to specific scenarios such as crosswalks. Parameters of the proposed model were calibrated by the genetic algorithm (GA) based on trajectory data of the same vehicle-pedestrian interaction patterns from controlled experiments. The proposed model is able to simulate complex self-designed vehicle-pedestrian interaction scenarios. The effectiveness of the proposed model was validated by comparing the simulated trajectories with ground truth trajectories under the same initial conditions, and by evaluating the pedestrian behavior of avoiding vehicle in the simulation of self-designed scenarios.

Trajectory prediction of heterogeneous road agents such as vehicles, cyclists, and pedestrians in dense traffic plays an essential role in self-driving. Despite breakthroughs in trajectory prediction technology in recent years, challenges remain in world state representation, social interaction modeling, real-time computing, and road agent heterogeneity. To address these challenges, we propose a new model that employs hierarchical convolutional networks and multi-task learning to predict agents' trajectories. The model first achieves effective and unified representation of agent and scene context by rendering heterogeneous world states in a top-down multi-channel raster map. Based on this representation, we propose hierarchical convolutional networks to extract global interaction and local features of all agents simultaneously, enabling the model to predict multiple agents' trajectories in real-time in a single forward inference. In addition, we specifically design multi-task learning branches with dynamic adaptive anchors to capture differences in behavioral patterns of heterogeneous agents, allowing a single model to accurately predict multimodal trajectories of multi-class agents. Extensive experiments on public nuScenes and Lyft datasets demonstrate top model performance. Importantly, our model is faster (2.2x) and more computationally stable than state-of-the-art models, making it well-suited for mass-produced self-driving systems that require both performance and computational efficiency.

Due to their strong dependence on high-definition (HD) maps, mainstream methods cannot make accurate predictions of vehicle trajectories under missing map conditions or in dynamically changing scenarios. In this paper, the interaction features intrinsic in traffic scenes are utilized and the implicit traffic priors related to vehicle motion in the scene are extracted to improve the accuracy of vehicle trajectory prediction when no HD map is available. First, a graph attention network is constructed to encode the motion state and local interactions between vehicles at each historical timestamp. Then a bi-axial Transformer is introduced to alternately update the global interaction and vehicle motion features. At the same time, a multi-scale structure is proposed to fuse the high-level behavior logic and low-level motion primitives of the agent vehicle. Finally, a trajectory decoder is used to output multi-modal vehicle trajectories. The proposed model was trained and evaluated using the Argoverse1 Forecasting dataset. The experimental results show that all metrics of this method were better than the no-HD map ablations of mainstream prediction models, and even better than some mainstream prediction methods with HD map encoding. In addition, a Lidar-based physical perception platform was set up on an experimental vehicle, and the generalization capability of the proposed method was validated in real traffic scenes.