30 July 2024 in Expert Systems with Applications
25 February 2022 in IEEE Transactions on Intelligent Transportation Systems
One of the most significant problem in traffic accidents is distracted driving that causes a higher number of deaths and injury every year. In recent times car manufacturing companies are integrating driver distraction detection systems for detecting distracted behavior. In this work we have suggested an mobile usage detection system in driving images taken from inside of a car. Previously proposed distracted driver detection systems that focus only on gaze and head poses. We have suggested another alternative using driver hand images by which distracted driving detection system can be benefited. If the driver looks at the road ahead, previous head pose and gaze based systems are unable to detect distractions caused by cell phone use. To reliably detect these modes of distraction, we trained an unsupervised low rank nonnegative dictionary and used a threshold based criterion over reconstruction error. We have created a new driving dataset based on YouTube videos that includes real-world driving scenarios as well as mobile phone usage while driving. We have also tested our method using a number of publicly available driving datasets, including the State farm driving dataset, the AUC driving dataset, the VIVA dataset, and the CVRR-3D dataset. According to our knowledge of the current literature, this method is the first to use an unsupervised approach to address the problem of cell phone usage while driving. We achieved a competitive test accuracy of 79.17%, 77.19%, 84.0%, 76.0%, 75.45% in the proposed Youtube ego driving, State farm driving, CVRR-3D, VIVA, AUC distracted driver dataset, respectively.
18 May 2021 in The Visual Computer
16 May 2024 in ACM Transactions on Multimedia Computing, Communications, and Applications
With the advent of egocentric cameras, there are new challenges where traditional computer vision is not sufficient to handle this kind of video. Moreover, egocentric cameras often offer multiple modalities that need to be modeled jointly to exploit complimentary information. In this article, we propose a sparse low-rank bilinear score pooling approach for egocentric hand action recognition from RGB-D videos. It consists of five blocks: a baseline CNN to encode RGB and depth information for producing classification probabilities; a novel bilinear score pooling block to generate a score matrix; a sparse low-rank matrix recovery block to reduce redundant features, which is common in bilinear pooling; a one-layer CNN for frame-level classification; and an RNN for video-level classification. We proposed to fuse classification probabilities instead of traditional CNN features from RGB and depth modality, involving an effective yet simple sparse low-rank bilinear score pooling to produce a fused RGB-D score matrix. To demonstrate the efficacy of our method, we perform extensive experiments over two large-scale hand action datasets, namely, THU-READ and FPHA, and two smaller datasets, GUN-71 and HAD. We observe that the proposed method outperforms state-of-the-art methods and achieves accuracies of 78.55% and 96.87% over the THU-READ dataset in cross-subject and cross-group settings, respectively. Further, we achieved accuracies of 91.59% and 43.87% over the FPHA and Gun-71 datasets, respectively.
7 May 2024 in IEEE Transactions on Human-Machine Systems
Robust segmentation of hands in a cluttered environment for hand gesture recognition has remained a challenge in computer vision. In this work, a two-stage gesture recognition framework is proposed. In the first stage, we segment hands using the proposed deep learning algorithm, and in the second stage, we use these segmented hands to classify gestures using a novel structurally incoherent nonnegative matrix factorization approach. We propose a new deep learning framework for hand segmentation called densely dilated U-Net. We exploit recently proposed dense blocks and dilated convolution layers in our work. To cope with the scarcity of labeled datasets we extend our densely dilated U-Net for semisupervised hand segmentation using hand bounding boxes as cues. We provide quantitative and qualitative evaluation of proposed hand segmentation model on several public hand segmentation datasets including EgoHands, GTEA, EYTH, EDSH, and HOF. Semisupervised segmentation results are also obtained on two hand detection datasets including VIVA and CVRR. As an extension of our work, we show semisupervised segmentation and gesture recognition results using segmented hands on NUS-II cluttered hand gesture dataset. To validate the efficiency of our semisupervised algorithm we evaluate it on OUHands dataset with real ground truth labels. For gesture classification, we propose a novel structurally incoherent nonnegative matrix factorization algorithm. We propose to use CNN features extracted from segmented images for nonnegative matrix factorization. Experimental results on NUS-II and OUHands datasets demonstrate that our two-stage approach for gesture recognition yields superior results.