27 September 2023 in IEEE Transactions on Multimedia
Skeleton-based action recognition has been substantially driven by the development of artificial intelligence technology and deep sensors. Recently, graph convolutional networks (GCNs) have achieved excellent performances in skeleton-based action recognition. However, the performances of GCN-based methods are impaired by inappropriate node partitioning strategy and obstructed long-range information flow. To solve these issues, a novel Select-Assemble-Normalize Graph Convolution Network (SAN-GCN) is proposed to model the spatio-temporal features of skeleton. First, all skeleton joints are selected as root nodes, and the neighborhoods of the root joints are assembled and normalized according to the body structure, which explicitly and interpretably expresses the spatial geometric relation of the skeleton joints. Second, we propose an attention-based assembly and normalization strategy to adaptively capture non-local joints. The adaptive assembly and normalization can avoid the dilution of key long-range features. Moreover, a bi-level aggregation strategy is introduced to learn spatio-temporal dependencies of joints, where the low-level aggregation aligns the normalized neighborhood graphs, and the high-level aggregation aggregates the features of neighbor nodes by a standard convolution kernel. In high-level aggregation, it is convenient to realize factorized spatio-temporal aggregation or unified spatio-temporal aggregation. Extensive experiments on four datasets with different numbers of action patterns demonstrate that our model achieves comparable performance with the state-of-the-art works.
10 January 2024 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Gait impairment in Parkinson’s Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24–50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.
2 February 2026 in IEEE Internet of Things Journal
For Underwater Internet of Things (UIoT) applications, the accurate and efficient estimation of the direction of arrival (DOA) is fundamental to technologies such as node localization and autonomous underwater vehicle (AUV) node cooperative communication. However, the low signal-to-noise ratio (SNR) and limited energy in underwater environments pose severe challenges to DOA estimation. Furthermore, existing methods typically require a large number of snapshots. To address these issues, this article proposes the use of an adaptive wavelet denoising model to enhance the quality of underwater acoustic signals. Subsequently, a dual-branch space-time state space network (ST-SSNet) is proposed. This network consists of a time feature extraction branch (TFEB) and a space feature extraction branch (SFEB). The time branch incorporates gating units and time mixing functions into the state space model (SSM) within the Mamba framework to extract temporal features. The spatial branch uses 1-D convolutions in different directions and the convolutional block attention module (CBAM) to extract spatial features. Extensive simulation and sea trial experiments demonstrate that ST-SSNet outperforms other deep learning methods in various scenarios, while having lower computational complexity than other methods. Compared to ResNet18, the accuracy improves by 2.14%, and RMSE is reduced by 43.4%.