Recent research on hand detection and gesture recognition has attracted increasing interest due to its broad range of potential applications, such as human-computer interaction, sign language recognition, hand action analysis, driver hand behavior monitoring, and virtual reality. In recent years, several approaches have been proposed with the aim of developing a robust algorithm which functions in complex and cluttered environments. Although several researchers have addressed this challenging problem, a robust system is still elusive. Therefore, we propose a deep learning-based architecture to jointly detect and classify hand gestures. In the proposed architecture, the whole image is passed through a one-stage dense object detector to extract hand regions, which, in turn, pass through a lightweight convolutional neural network (CNN) for hand gesture recognition. To evaluate our approach, we conducted extensive experiments on four publicly available datasets for hand detection, including the Oxford, 5-signers, EgoHands, and Indian classical dance (ICD) datasets, along with two hand gesture datasets with different gesture vocabularies for hand gesture recognition, namely, the LaRED and TinyHands datasets. Here, experimental results demonstrate that the proposed architecture is efficient and robust. In addition, it outperforms other approaches in both the hand detection and gesture classification tasks.
12 December 2023 in Engineering Applications of Artificial Intelligence
Digital publishing’s exponential growth has created vast scholarly collections. Guiding researchers to relevant resources is crucial, and knowledge graphs (KGs) are key tools for unlocking hidden knowledge. However, current methods focus on external links between concepts, ignoring the rich information within individual papers. Challenges like insufficient multi-relational data, name ambiguity, and cold-start issues further limit existing KG-based methods, failing to capture the intricate attributes of diverse entities. To solve these issues, we propose GLAMOR, a robust KG framework encompassing entities e.g., authors, papers, fields of study, and concepts, along with their semantic interconnections. GLAMOR uses a novel random walk-based KG text generation method and then fine-tunes the language model using the generated text. Subsequently, the acquired context-preserving embeddings facilitate superior top@k predictions. Evaluation results on two public benchmark datasets demonstrate our GLAMOR’s superiority against state-of-the-art methods especially in solving the cold-start problem.
27 July 2024 in Progress in Neuro-Psychopharmacology and Biological Psychiatry
Accurate and real-time recognition of skeleton-based dynamic hand gestures has gained increasing attention in recent years with the development of depth sensors and the improved hand joints estimation algorithms. This task is challenging due to the spatial and temporal features that exacerbate the task complexity. Although previous works have applied different techniques, it remains challenging to efficiently and simultaneously encode the spatiotemporal features. To address this problem, this work presents a Deep Convolutional LSTM (DConvLSTM) model to learn more discriminative spatiotemporal features from skeleton data implicitly. The model employs multi-layer ConvLSTM to accurately capture the multiscale spatial and sequential information of the gesture and preserves a fast inference and lightweight size. Extensive experiments on three publicly available datasets show strong performance and demonstrate the superiority of our method by outperforming other methods. Furthermore, our method can achieve comparable recognition accuracy while maintaining small models and short inference time.