Driving behavior classification is an important direction in the field of social transportation systems and advanced driving assistance system (ADAS), which has attracted more and more attention in recent years. An accurate driving behavior classification algorithm plays a great role in traffic safety, energy saving, and other fields. In this article, we propose a novel vehicle speed and image fusion for driving behavior classification network (SIFDriveNet), which classifies driver behaviors into normal driving, aggressive driving, and drowsy driving. Our method has the following key advantages. First, in the research of driving behavior classification, we are the first to introduce a 2-D image with rich roadside information and convert speeds into a 2-D spectrogram expressing the time–frequency characteristics of speeds through short-time Fourier transform (STFT) while unifying the data space of image information and speed information. Second, we propose a tensor fusion method based on weight decomposition to fully fuse the vectors of the two modalities. This method maps the tensor outer product results to the low-dimensional space through weight decomposition and has a low computational cost while maintaining the fusion effect of the tensor outer product. In addition, we evaluated our model on the public UAH-DriveSet and compared it with the most advanced model. Experimental results show that our model has a better performance, and F1-score is 97.9% on all roads. Especially on the secondary road, our F1-score is 99.4%. Also, our model has strong generalization, and we have reached 99.3% F1 in distracted driving multimodal dataset. In addition, the inference speed reaches 411 FPS, enabling real-time needs. The code is available on https://github.com/alu222/SIFDriveNet .

Driving behavior classification is a critical component of social transportation systems and advanced driver assistance systems, and it has gained increasing attention in recent years. Accurate classification algorithms for driving behavior play a significant role in enhancing traffic safety, energy conservation, and related fields. In this article, we propose a novel driving behavior classification network named feature-attention- embedding-based multimodal-fusion driving-behavior-classification network (FMDNet). FMDNet incorporates eight types of data, including acceleration along the x-axis, y-axis, z-axis, roll angle, pitch angle, yaw angle, roadside image, and vehicle speed, to classify driving behavior. To effectively fuse features extracted from different modalities, taking into account their varying importance, we introduce the feature attention embedding-based fusion module (FAEF) as our fusion strategy. This fusion strategy enhances the network’s capability to capture meaningful features by incorporating two feature attention embedding units that delve deeper into the interplay between different modes. Furthermore, we provide further validation of the effectiveness of our approach through extensive ablation experiments to investigate and analyze the impact of various modal data on the classification of driving behavior. Our proposed FMDNet achieves state-of-the-art performance on the public UAH-DriveSet dataset, demonstrating its effectiveness with an impressive F1-score of 99.0%. Additionally, the robustness of our model is confirmed on distracted dataset, achieving a remarkable F1-score of 99.7%. The model’s outstanding performance on both the UAH-DriveSet dataset and the distracted-dataset highlights its capabilities and potential for real-world applications.

Lane detection plays an important role in the field of automatic driving. Conventional convolutional operations typically focus on local block-like region, while lane often span across multiple image regions as strip lines. How to make up for the limitations of ordinary convolution in extracting lane information and make the network focus on the strip lanes is a challenging problem. For this purpose, we propose an oblique convolution idea to effectively extract the information of the whole lane. Specifically, an Oblique Rotation Module (ORM) is proposed to rotate the lane feature map into a vertical orientation, enabling better lane positioning and recognition by using row anchors. Additionally, we introduce a Strip Spatial Attention Module (SSAM) to extract global features of the entire lane and employ deformable convolutions to adapt to curved lane lines. Finally, we restore the feature map using Anti-oblique Rotation Module (AORM) and obtain lane detection results through classification-based predictions. During training, we incorporate a multi-scale auxiliary classification loss to predict the presence of lane lines at different network levels, aiding the network in learning the structural information of lane lines. A large number of experiments on the lane detection benchmark dataset show that our method can achieve advanced performance, and we have achieved good results in public datasets. The accuracy on the Tusimple dataset is 96.50%, the F1 on the CULane dataset is 79.33%. Both results rank in the top 10 on the leaderboard, and the inference speed reaches 35 FPS. Code and data are available on https://github.com/gongyan1/Oblique-Convolution

In recent years, researchers have focused on identifying tasks related to driver state, traffic environment, and others to enhance the safety of autonomous driving assistance systems. However, current research on these tasks is conducted independently, neglecting the interconnections between the driver, traffic environment, and vehicle. In this paper, we propose a Unified Multi-task Assistive Driving Network Based on Multimodal Fusion (UMD-Net), the first unified model capable of recognizing four tasks simultaneously by utilizing multimodal data: driver behavior recognition, driver emotion recognition, traffic context recognition, and vehicle behavior recognition. In order to better enhance the synergistic effects between multiple tasks, we designed the position-sensitive multi-directional attention feature extraction subnetwork and recursive dynamic feature fusion module. The former captures the key features of multi-view images by different directions of attention mechanism to improve the generalization of the model across multiple tasks. The latter dynamically adjusts the fusion weight according to the multimodal features to enhance the representation ability of important features in multi-task learning. Our model was evaluated on the public dataset AIDE, achieving the best performance across all four tasks and a high accuracy of 95.31% in the traffic context recognition task, demonstrating the superiority of our approach. The code is available on https://github.com/Wenzhuo-Liu/UMD-Net.

Understanding bimanual hand interactions is essential for realistic 3D pose and shape reconstruction. However, existing methods struggle with occlusions, ambiguous appearances, and computational inefficiencies. To address these challenges, we propose Vision Mamba Bimanual Hand Interaction Network (VM-BHINet), introducing state space models (SSMs) into hand reconstruction to enhance interaction modeling while improving computational efficiency. The core component, Vision Mamba Interaction Feature Extraction Block (VM-IFEBlock), combines SSMs with local and global feature operations, enabling deep understanding of hand interactions. Experiments on the InterHand2.6M dataset show that VM-BHINet reduces Mean per-joint position error (MPJPE) and Mean per-vertex position error (MPVPE) by 2-3 \(\%\) , significantly surpassing state-of-the-art methods.