Human driving behaviors are personalized and unique, and the automobile fingerprint of drivers could be helpful to automatically identify different driving behaviors and further be applied in fields such as auto-theft systems. Current research suggests that in-vehicle Controller Area Network-BUS (CAN-BUS) data can be used as an effective representation of driving behavior for recognizing different drivers. However, it is difficult to capture complex temporal features of driving behaviors in traditional methods. This paper proposes an end-to-end deep learning framework by fusing convolutional neural networks and recurrent neural networks with an attention mechanism, which is more suitable for time series CAN-BUS sensor data. The proposed method can automatically learn features of driving behaviors and model temporal features without professional knowledge in features modeling. Moreover, the method can capture salient structure features of high-dimensional sensor data and explore the correlations among multi-sensor data for rich feature representations of driving behaviors. Experimental results show that the proposed framework performs well in the real world driving behavior identification task, outperforming the state-of-the-art methods.

A large number of studies have shown that most vehicle collisions are caused by drivers' abnormal operations. To ensure the safety of all people on the road network as much as possible, it is crucial to be able to predict the drivers' driving safety risks in real time. In this paper, we propose a novel cost-sensitive L1/L2-nonnegativity-constrained deep autoencoder network for driving safety risk prediction. Unfortunately, with existing research methods, the size of the sliding time window is too large, the feature extraction is relatively subjective, and class imbalances occur, which leads to low identification accuracy, long prediction times, and poor applicability. We first propose using a three-layer L1/L2-nonnegativity-constrained autoencoder to adaptively search the optimal size of the sliding window and then construct a deep L1/L2-nonnegativity-constrained autoencoder network to automatically extract the hidden features of the driving behaviors. Finally, we build a new L1/L2-nonnegativity-constrained focal loss classifier to predict the driving behaviors under different safety risk levels. The results from the public 100-Car naturalistic driving study dataset indicate that our method can effectively find the optimal window size, reduce the data volume and reconstruction error, and extract more distinctive features. Furthermore, this method effectively curbs the class imbalance, improves the driving safety risk prediction performance, reduces overfitting, shortens the prediction time, and improves the timeliness.

Driving behavior recognition is a challenging task that exploits the acceleration and angular velocity information of the vehicle collected by smartphone to identify various driving events. Traditional methods usually extract hand-crafted features from raw data, leading to under-explored temporal features of driving behaviors. To address the issue of hand-designed limitation for features, this paper proposes an end-to-end deep learning framework to automatically extract the features of driving behaviors. The mechanism behind our method is to model temporal features, capture salient structure features, and explore the correlation among the high-dimensional sensor data by fusing convolutional neural network (CNN) and recurrent neural network (RNN) with an attention unit. Moreover, a novel approach is introduced to build driving behavior dataset, which considers the effect of gravity in modeling smartphone sensor data. Subsequently, sensor data with device position independence is collected, and six types of driving events (straight driving, static, left turn, right turn, breaking, and acceleration) are annotated, which provides rich sensor information compared with other methods. The experimental results indicate that the proposed model outperforms other competing methods significantly, which possesses good generalization ability in the identification of driving behaviors.

Graph convolutional networks (GCNs) have achieved remarkable performance on skeleton-based action recognition. Existing GCN-based methods usually apply the fixed graph topology and one fixed temporal convolution kernel to extract the spatial features of joints and temporal features, which is from a single-scale perspective. Actually, human actions are coordinated by various body parts in the spatial domain, and exhibit different characteristics in the temporal domain. Therefore, it is appropriate to model the multi-scale information that can enhance both the explainability and stability, which is ignored in current literatures. To address this issue, we propose a multi-scale spatial-temporal graph neural network (MSTGNN) to discover multi-scale discriminative features from spatial and temporal aspects simultaneously. Our contributions are three-folds: 1) For the spatial domain, inspired by the kinematics of the human action, we develop a three-scale graph data structures in a fine-to-coarse way. A novel hybrid spatial pooling module is then proposed to dynamically exploit the global and comprehensive information step-by-step. 2) For the temporal domain, we design a multi-scale temporal convolution module adaptively fusing the temporal features extracted by different scale convolution kernels. 3) As utilizing one-stream architecture instead of multi-stream architecture, the proposed model can be trained in an end-to-end manner. MSTGNN achieves state-of-the-art performance with less computation complexity. Experimental results conducted on two large datasets (NTU-RGB+D and NTU-RGB+D-120) demonstrate the superiority of MSTGNN.

Traffic accidents caused by distracted drivers account for a large proportion of traffic accidents each year, and monitoring the driving state of drivers to avoid traffic accidents caused by distracted driving has become a very important research direction. At present, the field of driver distraction detection mainly adopts supervised learning methods, which have problems such as poor generalization ability, large labeling cost, and weak artificial intelligence. This paper is oriented toward driver distraction fine-grained detection and innovatively proposes a new unsupervised deep learning algorithm, which is referred to as UDL, to achieve a more human-like level of intelligence. First, we build a new unsupervised deep learning algorithm; furthermore, we integrate the multilayer perceptron (MLP) architecture to build a new backbone and projection head to strengthen feature extraction capabilities; and finally, a new loss function based on contrast learning and a stop-gradient strategy is designed to guide the model to learn more robust features. The comparison results on large-scale driver distraction detection datasets show that our UDL method can accurately detect driver distraction without labels and exhibits excellent generalization performance with a linear evaluation accuracy of 97.38%; In addition, after fine-tuning with fewer labels, our UDL method can achieve superior performance close to state-of-the-art supervised learning methods, achieving 99.07% accuracy after fine-tuning using only 50% of the labeled data, which greatly reduces the cost and limitations of manual annotation.

According to the surveys of the World Health Organization, distracted driving is one of main causes of road traffic accidents. To improve road traffic safety, real-time detection of drivers’ driving behavior is very important for the development of highly reliable Advanced Driver Assistance System (ADAS). At present, the deep learning architecture based on a Convolutional Neural Network (CNN) has disadvantages such as large number of parameters and weak global feature extraction ability. Therefore, this paper proposes an innovative driver distraction detection model based on the fusion of a transformer and a CNN, referred to as FPT, which is the first exploration in the field of driver distraction detection. First, we introduce the latest Twins transformer as a benchmark. Then, we design residual embedding to replace block embedding, which can further integrate the convolutional neural network with Transformer and improve the feature extraction ability. In addition, the Multilayer Perceptron (MLP) module with a large parameter occupancy rate in the original transformer structure is replaced with a lightweight group convolution module to reduce computational complexity. Finally, a cross-entropy loss function for label smoothing is designed to guide network learning with significantly differentiated features. Comparison results on two large-scale driver distraction detection datasets show that the proposed FPT offers a better compromise between computational cost and performance compared to the state-of-the-art CNN and Transformer architectures.