Both the discrete motion states and continuous joint kinematics are essential for controlling assistive robots under changeable environmental conditions. However, few studies investigate both the discrete and continuous motion intents. This paper is the first work to propose an end-to-end motion intent decoding method that integrates recognition of discrete locomotion modes and prediction of continuous joint kinematics and gait events. First, we propose a data-driven approach to segment the transitional periods of the adjacent locomotion modes and determine their boundaries. Second, we build a CNN-LSTM network that utilizes locomotion mode classification as the prior of joint kinematics and gait events prediction and thus decodes discrete and continuous motion intents. Finally, we evaluate our method through extensive experiments and comparisons. The proposed method for locomotion mode recognition achieves an accuracy of 98.73% for steady-state and 97.53% for transitional periods. For knee angle prediction, the method achieves the ahead-of-time prediction with NRMSE lower than 8.41%, R-value higher than 0.93, and R-square higher than 0.88 for various prediction time. For gait event prediction, it predicts the future timing of events with an error of 8.46 ms for initial contact, 32.76 ms for heel off, and 8.91 ms for toe off. This study reveals the feasibility of predicting locomotion-based kinematics and gait events on various terrains. The rich motion intents studied in this work highlight the potential of a more detailed lower-limb rehabilitation system.

The cybertwin-driven 6G that can obtain static and dynamic data stream of users provide an exciting potential for a novel muscular human cybertwin beyond traditonally used artificial neural networks (ANNs) and musculoskeletal models (MSMs). In this article, we propose the conceptual design of the muscular human cybertwin and construct a baseline model with an improved generalization ability over ANN and an easier adaptation to new data distributions over MSMs. In particular, we for the first time propose to combine ANN and MSM, which benefits from the combination of learning-based approaches and analytical approaches. We then experimentally compare different manners of the combination and demonstrate the better combining manner on our testing case. Finally, we evaluate our method on an open-sourced dataset and on data from wearable sensors from the aspects of joint moment prediction accuracy, data efficiency, generalization ability, and time efficiency of personalization. Our proposed method achieves accuracy similar with ANN and over 30 $\%$ better than MSM with sufficient training data. Compared with ANN, the improved data efficiency is presented by the better accuracies with a small amount of training data, and the generalization ability to unseen walking conditions and new subjects are demonstrated by the over 70 $\%$ accuracy improvements. Moreover, when fine-tuning the model, our algorithm is demonstrated by the time 75 $\%$ shorter than calibrated MSM and the accuracy improvements.

Surface Electromyography (sEMG) enables an intuitive control of wearable robots. The muscle fatigue-induced changes of sEMG signals might limit the long-term usage of the sEMG-based control algorithms. This paper presents the performance deterioration of sEMG-based gait phase classifiers, explains the deterioration by analyzing the time-varying changes of the extracted features, and proposes a training strategy that can improve the classifiers’ robustness against muscle fatigue. In particular, we first select some features that are commonly used in fatigue-related studies and use them to classify gait phases under muscle fatigue. Then, we analyze the time-varying characteristics of extracted features, with the aim of explaining the performance of the classifiers. Finally, we propose a training strategy that effectively improves the robustness against muscle fatigue, which contributes to an easy-to-use method. Ten subjects performing prolonged walking are recruited. Our study contributes to a novel perspective of designing gait phase classifiers under muscle fatigue.

Objective: To develop a robust group-level brain parcellation method using deep learning based on resting-state functional magnetic resonance imaging (rs-fMRI), aiming to release the model assumptions made by previous approaches. Methods: We proposed Brain Deep Embedded Clustering (BDEC), a deep clustering model that employs a loss function designed to maximize inter-class separation and enhance intra-class similarity, thereby promoting the formation of functionally coherent brain regions. Results: Compared to ten widely used brain parcellation methods, the BDEC model demonstrates significantly improved performance in various functional homogeneity metrics. It also showed favorable results in parcellation validity, downstream tasks, task inhomogeneity, and generalization capability. Conclusion: The BDEC model effectively captures intrinsic functional properties of the brain, supporting reliable and generalizable parcellation outcomes. Significance: BDEC provides a useful parcellation for brain network analysis and dimensionality reduction of rs-fMRI data, while also contributing to a deeper understanding of the brain's functional organization.

Embodied reference understanding is crucial for intelligent agents to predict referents based on human intention through gesture signals and language descriptions. This paper introduces the Attention-Dynamic DINO, a novel framework designed to mitigate misinterpretations of pointing gestures across various interaction contexts. Our approach integrates visual and textual features to simultaneously predict the target object’s bounding box and the attention source in pointing gestures. Leveraging the distance-aware nature of nonverbal communication in visual perspective taking, we extend the virtual touch line mechanism and propose an attention-dynamic touch line to represent referring gesture based on interactive distances. The combination of this distance-aware approach and independent prediction of the attention source, enhances the alignment between objects and the gesture represented line. Extensive experiments on the YouRefit dataset demonstrate the efficacy of our gesture information understanding method in significantly improving task performance. Our model achieves 76.3% accuracy at the 0.25 IoU threshold and, notably, surpasses human performance at the 0.75 IoU threshold, marking a first in this domain. Comparative experiments with distance-unaware understanding methods from previous research further validate the superiority of the Attention-Dynamic Touch Line across diverse contexts.

Brain decoding that classifies cognitive states using the functional fluctuations of the brain can provide insightful information for understanding the brain mechanisms of cognitive functions. Among the common procedures of decoding the brain cognitive states with functional magnetic resonance imaging (fMRI), extracting the time series of each brain region after brain parcellation traditionally averages across the voxels within a brain region. This neglects the spatial information among the voxels and the requirement of extracting information for the downstream tasks. In this study, we propose to use a fully connected neural network that is jointly trained with the brain decoder to perform an adaptively weighted average across the voxels within each brain region. We perform extensive evaluations by cognitive state decoding, manifold learning, and interpretability analysis on the Human Connectome Project (HCP) dataset. The performance comparison of the cognitive state decoding presents an accuracy increase of up to 5% and stable accuracy improvement under different time window sizes, resampling sizes, and training data sizes. The results of manifold learning show that our method presents a considerable separability among cognitive states and basically excludes subject-specific information. The interpretability analysis shows that our method can identify reasonable brain regions corresponding to each cognitive state. Our study would aid the improvement of the basic pipeline of fMRI processing.