Researches on machine vision-based driver fatigue detection algorithm have improved traffic safety significantly. Generally, many algorithms do not analyze driving state from driver characteristics. It results in some inaccuracy. The paper proposes a fatigue driving detection algorithm based on facial multi-feature fusion combining driver characteristics. First, we introduce an improved YOLOv3-tiny convolutional neural network to capture the facial regions under complex driving conditions, eliminating the inaccuracy and affections caused by artificial feature extraction. Second, on the basis of the Dlib toolkit, we introduce the Eye Feature Vector(EFV) and Mouth Feature Vector(MFV), which are the evaluation parameters of the driver’s eye state and mouth state, respectively. Then, the driver identity information library is constructed by offline training, including driver eye state classifier library, driver mouth state classifier library, and driver biometric library. Finally, we construct the driver identity verification model and the driver fatigue assessment model by online assessment. After passing the identity verification, calculate the driver’s closed eyes time, blink frequency and yawn frequency to evaluate the driver’s fatigue state. In simulated driving applications, our algorithm detects the fatigue state at a speed of over 20fps with an accuracy of 95.10%.

RGB-D-based human action recognition has attracted much attention recently because it can provide more complementary information than a single modality. However, it is difficult for two modalities to effectively learn spatial-temporal information from each other. To facilitate information interaction between different modalities, a cross-modality compensation convolutional neural network (ConvNet) is proposed for human action recognition, which enhances the discriminative ability by jointly learning compensation features from the RGB and depth modalities. Moreover, we design a cross-modality compensation block (CMCB) to extract compensation features from the RGB and depth modalities. Specifically, CMCB is incorporated into two typical network architectures, ResNet and VGG, to verify the ability to improve the performance of our model. The proposed architecture has been evaluated on three challenging datasets: NTU RGB+D 120, THU-READ and PKU-MMD. We experimentally verify that our proposed model with CMCB is effective for different input types, such as pairs of raw images and dynamic images constructed from the entire RGB-D sequence, and the experimental results show that the proposed framework achieves state-of-the-art performance on all three datasets.

Vision based depth estimation plays a significant role in Intelligent Transportation Systems (ITS) because of its low cost and high efficiency, which can be used to analyze driving environment, improve driving safety, etc. Although recently proposed approaches abandon time consuming pre-processing or postprocessing steps and achieve an end-to-end prediction manner, fine details may be lost through max-pooling based encode modules. To tackle this problem, we propose Multi-Scale Dilated Convolution Network (MSDC-Net), a dilated convolution based deep network. For the feature encoding and decoding part, dilated layers maintain the scale of original image and reduce lost details. After that, a pyramid dilated feature extraction module is added to integrate the knowledge learned through forward steps with different receptive fields. The proposed approach is evaluated on KITTI dataset, and achieves a state-of-the-art result on the dataset.