Recently, people’s demand for action recognition has extended from the initial high classification accuracy to the high accuracy of the temporal action detection. It is challenging to meet the two requirements simultaneously. The key to behavior recognition lies in the quantity and quality of the extracted features. In this paper, a two-stream convolutional network is used. A three-dimensional convolutional neural network (3D-CNN) is used to extract spatiotemporal features from the consecutive frames. A two-dimensional convolutional neural network (2D-CNN) is used to extract spatial features from the key-frames. The integration of the two networks is excellent for improving the model’s accuracy and can complete the task of distinguishing the start–stop frame. In this paper, a multi-scale feature extraction method is presented to extract more abundant feature information. At the same time, a multi-task learning model is introduced. It can further improve the accuracy of classification via sharing the data between multiple tasks. The experimental result shows that the accuracy of the modified model is improved by 10%. Meanwhile, we propose the confidence gradient, which can optimize the distinguishing method of the start–stop frame to improve the temporal action detection accuracy. The experimental result shows that the accuracy has been enhanced by 11%.

Automobile safety driving technology is a hot topic in today’s society, which is very significant to the social transportation system. Vehicle driving behavior monitoring is the foundation and core of safe driving techniques. The research on existing vehicle safety technology can not only improve the understanding of current safe driving research progress, but also provide reference for future researchers. This paper proposes a state recognition system based on a three-dimensional convolutional neural network, which can identify several improper states frequently encountered by drivers during driving, including drinking, making phone calls, and smoking, and can also issue alarm interventions. The system takes the collected continuous video frame information as the input of the three-dimensional convolutional network, carries out multi-level feature extraction and spatio-temporal information fusion, and identifies the driver state according to the extracted spatio-temporal features. The state is judged by the facial feature points of the video stream, and the design of the video surveillance driver state recognition system is completed. Then, the driver status recognition is improved and optimized, and finally, the actual deployment of the driver status recognition system on the mobile terminal is completed. A large number of experimental results show that the driver status recognition system proposed in this paper has achieved upper identification accuracy.