10 May 2024 in IEEE Internet of Things Journal
Human activity recognition (HAR) enables real-time monitoring of human movement, posture, and activity level, and can provide valuable information for the health management. With the continuous advancement of Internet of Things (IoT) technology, wearable sensors and smartphones equipped with various types of sensors have become widely utilized to collect multimodal data for HAR. However, in multimodal HAR, current fusion methods fall short in capturing intermodality correlations, hampering the full exploitation of complementary information between the modalities and leading to lower recognition accuracy. We thus propose a novel multiscale cross-modal interactive fusion network (MCIFN), which can fully capture correlations between various modalities and obtain an effective fused representation for HAR. Specifically, we employ a multiscale parallel convolution module to extract features from each modality at multiple scales. Then, an interactive fusion strategy based on the cross-modal attention mechanism is introduced to adjust and enhance each modality based on its correlations with other modalities. Additionally, to resolve the information redundancy caused by the interactive fusion strategy, we utilize a hybrid attention module to focus on important information in the fusion representation. Extensive experiments conducted on three publicly available data sets and one private data set demonstrate that our proposed network outperforms the previous baseline networks for HAR. Additionally, our proposed fusion strategy yielded a notable improvement in accuracy ranging from 1.87% to 9.96% compared to existing strategies. These findings imply that our newly proposed network can realize comprehensive multimodal fusion and effectively enhance HAR accuracy, potentially contributing to advancements in individual health management and personalized healthcare interventions.
19 August 2024 in IEEE Sensors Journal
Obstructive sleep apnea (OSA) is a common and chronic respiratory disorder. Autonomic dysfunction and heart rate change usually occur during apneic events. Damaged autonomic nerve system modulation can lead to morphological differences in R-to-R intervals (RRI) series. Capable of capturing structural properties of RRI series and thus assessing the heart rate dynamics, the local difference visibility graphs (LDVG) method was proposed as a novel OSA detection method in this study. The LDVG method focuses on long continuous ascent range, which is likely to be induced by OSA attack, demonstrating the characteristics of bradycardia and tachycardia in OSA. Combined with the local strategy, it avoids heavy workload while maintaining major correlations. Degree mean of LDVG (LDVG_M) and degree uncertainty of LDVG (LDVG_SEn) were calculated as the mean and Shannon entropy of the degree sequence of LDVG, which can quantify the tightness and uncertainty of structural characteristics of the RR interval series, respectively. 60 ECG recordings from the Physionet database were used in this study. Both LDVG_M and LDVG_SEn achieved a good performance in differentiating the severity of diseases, while the classic index low-frequency power to high-frequency power ratio value (LF/HF) cannot. Then, binary classification was performed. In OSA detection, the accuracy, specificity, sensitivity of both LDVG-related indices were all above 90%. LDVG_SEn achieved the highest accuracy (96.7%), while maintaining a good balance between sensitivity (97.5%) and specificity (95.0%). Therefore, this LDVG method provides an insight into the underlying pathological mechanism of OSA and is promising for clinical and home monitoring.