24 July 2020 in IEEE Transactions on Vehicular Technology
Timely recognition of driving intention is crucial in the design of a safe and effective driving assistance system. This study proposes an efficient recognition approach based on Nonlinear Polynomial Regression (NPR) and Recurrent Hidden Semi-Markov Model (R-HSMM) to recognize the driver lane-change intention accurately in the early stage. The NPR model is utilized to transform the input signal amplitude into the standard form in order to improve system adaptability. Besides, an unsupervised time-series segmentation method named the Toeplitz Inverse Covariance-based Clustering (TICC) is applied to label the driving data automatically. The R-HSMM is utilized as a time-series classifier to classify the driving intention during the lane-change process into predefined categories based on the signals processed by the NPR. The proposed method is verified by the experiments with a driving simulator. The experimental results show that the proposed method can recognize driver intention earlier than the popular recognition methods, and also can reduce the number of false warnings during the lane-change process, which has great significance for driving safety improvement. Moreover, the proposed method can adapt well to various vehicle speeds achieving stable recognition performance.
25 May 2023 in IEEE Transactions on Intelligent Vehicles
Effective trajectory prediction is of great significance for the design of intelligent driving systems. To overcome the problems of low algorithm efficiency and insufficient scenario adaptation in urban environments, this paper proposes a trajectory prediction framework based on the rule and learning-based frameworks fusion (TP-FRL). By augmenting the observed trajectory under the guidance of a physically possible trajectory set, the proposed framework improves algorithmic adaptation in the presence of disturbed trajectory observations. Meanwhile, rule-based spatio-temporal semantic corridors are used to vectorize agent trajectory-related regions, which improves the rationality of each agent region division and enhances the adaptability in different scenarios. Finally, based on the trajectory-related region modeling of each agent, the interest region of the central agent is learned by a graph convolutional network, and only the scenario elements in the interest region are encoded to improve the trajectory prediction efficiency of the transformer multi-head attention network. Traffic data from Argoverse and ApolloScape datasets are used to evaluate the performance of the TP-FRL model. Specifically, the TP-FRL model shows efficient performance and strong adaptability on datasets from two countries, which shows the promise of predicting trajectories under various scenarios. The TP-FRL framework code is publicly available at https://github.com/hanyuxuan123/TP-FRL-code .
26 July 2023 in IEEE Transactions on Industrial Electronics
An object-level attention prediction framework for drivers in the urban environment with rich semantic and motion information is proposed in this article. The proposed framework is based on the visual working memory mechanism, which decomposes the perception process into three phases, external stimuli, cognitive constructing, and memory search. In the external stimuli phase, semantic and motion information of surrounding objects is obtained. In the cognitive constructing phase, the neighbor-based hierarchical clustering method is applied to extract both independent and dependent features of traffic participants and driving events. In the memory search phase, the heterogeneous motif graph neural network is utilized to construct visual memory layers and integrate multilevel features for attention reasoning. Finally, the feature embedding is fed into a multilayer perceptron to predict the object-level visual attention. Training and testing data are collected from crowded and dynamic traffic scenes. Experimental results show that the proposed framework can achieve a superior object-level prediction performance in the information-rich environments compared with the state-of-the-art methods. In addition, the proposed framework can reduce the time bias of visual attention effectively.
12 August 2024 in IEEE Transactions on Intelligent Vehicles
LiDAR-based place recognition contributes significantly to identifying loop closure candidates for long-term simultaneous localization and mapping (SLAM) systems and obtaining global position for intelligent vehicles. Although the existing Transformer-based methods have achieved impressive results, they lack the capability of harnessing Transformers to establish long-range dependencies and suffer from high computational costs. To address these limitations, this study proposes a hierarchical range image transformer network named RangePlace to generate discriminative global descriptors for place recognition. By leveraging shifted window-based attention and depth-wise convolution, the proposed network can effectively and efficiently extract both global and local features. This not only enhances the network's capability to learn long-range information but also reduces the computational costs. In addition, a Pyramid Feature Mix module is incorporated to aggregate multi-scale feature maps to unified global descriptor, thus significantly enhancing place recognition performance. The proposed method is verified by experiments on the KITTI Odometry dataset, NCLT dataset, Ford Campus dataset and NuScenes dataset. The results demonstrate that the proposed method can achieve state-of-the-art performance in terms of recall rates. Moreover, it has good robustness to viewpoint yaw-rotation and strong generalization ability in various environments. Furthermore, it can be applied to real-time localization systems. The open-sourced codes used in this study are publicly available at https://github.com/JiLiBIT/RangePlace .
22 September 2023 in IEEE/ASME Transactions on Mechatronics
Wall climbing represents a crucial task in a heavy unmanned ground vehicle (HUGV) working under poor road conditions. Inspired by the human perception-action behavior, this study proposes an effective human-like planning system for solving the wall-climbing problem in the HUGVs. In the proposed model, the Gaussian mixture model (GMM) and hidden semi-Markov model (HSMM) are combined to imitate the stochastic dynamic driving behavior. Based on the decision made by the GMM-HSMM-based driver model, the speed planner uses the dynamic motion primitive (DMP) to generate the expected speed curve. Furthermore, in the climbing-over process, the DMP is employed to predict the motor torque curve. This study considers a driver's perception of acceleration, which strengthens the planning system's adaptability and flexibility. The proposed method is verified by experiments with a real electric tracked HUGV. The experimental results show that the proposed system with duration constraints can maintain a reasonable acceleration rate, balancing driving safety and efficiency in the wall-climbing task. Moreover, the proposed method can adapt well to different road conditions without using a vehicle dynamics model.
22 August 2024 in IEEE Sensors Journal
Road network extraction from remote sensing images can provide precise map information for global positioning and planning. While existing transformer-based methods show promising performance in road network extraction, they suffer from misleading results of crossroad and low generalization ability. In our study, a prompt-based multi-modal Transformer for road network extraction (PT-RE) is proposed. In PT-RE, a swin transformer is used as the backbone network to extract image features from remote sensing images. Then, a fine-tuned prompt-based method is employed to generate the road topology classification contexts. The prompt-based information generation and cross-modal loss function are designed to deal with the fine-tuning task. Compared with the original uni-modal loss function in fine-tuning, the cross-modal method processes the different modals information and improves the generalization ability. Finally, the topology decoder utilizes cross-attention architecture to predict the relationship by the information from images and classification contexts. With the help of different views and modals information, the framework strengthens the accuracy of crossroad detection rather than the uni-modal type. The proposed topological road network extraction method demonstrates superior accuracy across 20 US City datasets and SpaceNet datasets, showcasing its accuracy and generalization ability.