Wearable inertial motion capture, a new type of motion capture technology, mainly estimates the human posture in 3-D space through multisensor data fusion. The available method for sensor fusion is usually aided by magnetometers to remove the drift error in yaw angle estimation, which in turn limits their application in the presence of a complex magnetic field environment. In this article, an extended Kalman filter (EKF) data fusion method is proposed to fuse the 9-axis sensor data. Meanwhile, the heuristic drift reduction (HDR) method is used to calibrate the accumulated error of a heading angle. In addition, the position in 3-D space is estimated by the foot-mounted zero-velocity-update (ZUPT) technique. Combining 3-D attitude and position, a biomechanical model of the human body is established to track the motion of a real human body. The EKF algorithm and position estimation methods are benchmarked against the golden standard, optical motion capture system, for various indoor experiments. In addition, various outdoor experiments are also conducted to verify the reliability of the proposed method. The results show that the proposed algorithm outperforms the available attitude estimation model in motion tracking and is feasible for 3-D human motion capture.
Cerebral palsy is one of the main factors leading to children’s disability. A large number of such children have hand motor dysfunction, such as limited range of motion, abnormal gestures, etc. Our goal is to design a prototype of wearable gesture training equipment for such children. For this purpose, this paper presents the development of a wireless smart glove to facilitate the accurate measurement of finger movement through the integration of multiple IMU sensors. Commercially available gaming gloves are not fitted with sufficient sensors for this particular application and most of them can only provide one degree of freedom for finger flexion and extension. In this paper, a data glove integrated with nine degree of freedom inertial sensors in conjunction with complex multi-sensor data fusion is developed. In our method, a single/dual state measurement update switching extended Kalman filter is proposed to estimate the spatial attitude of each finger segment. Through traversing the kinematic chain, a tree type hand dynamic model based on joint constraints is established to realize the real-time gesture tracking. Furthermore, a visual interface is developed to display the tracking effect of gestures, and the reliability of our algorithm is verified by optical contrast experiments, the repeatability of joint angle is also evaluated by kinematic analysis. In general, all the experimental results demonstrated that our proposed framework can accurately track the 3-D hand motion. This glove will help quantify joint stiffness and monitor patient progression during the rehabilitation training process.
9 February 2022 in IEEE Transactions on Cognitive and Developmental Systems
Facial expressions are generally recognized based on hand-crafted and deep-learning-based features extracted from RGB facial images. However, such recognition methods suffer from illumination/pose variations. In particular, they fail to recognize these expressions with weak emotion intensities. In this work, we propose a cross-modality attention-based convolutional neural network (CM-CNN) for FER. We extract expression-related features from complementary facial images (gray-scale, LBP, and depth images) to handle the illumination/pose variations and to capture appearance details that describe expressions with weak emotion intensities. Rather than directly concatenating the complementary features, we propose a novel cross-modality attention fusion network to enhance spatial correlations between any two types of facial images. Finally, the CM-CNN is optimized with an improved focal loss, which pays more attention to facial expressions with weak emotion intensities. The average classification accuracies on VT-KFER, BU-3DFE(P1), BU-3DFE(P2), and Bosphorus are 93.86%, 88.91%, 87.28%, and 85.16%, respectively. Evaluations on these databases demonstrate our approach is competitive to state-of-the-art algorithms.
14 March 2024 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
The lower extremity exoskeleton, which can sense the neural motion state of the human body and then provide motion assistance, is gradually replacing the traditional wheelchairs and assistive devices, making many patients with disabilities or movement disorders able to regain the walking function. This survey provides a comprehensive review on recent technological advances in lower extremity neurorehabilitation exoskeleton from the perspectives of sensing, gait dynamics, and human–robot collaboration. For each technology category, a detailed comparison among state-of-the-art solutions is provided. The results show that the exoskeleton has been greatly improved in mechanical and learning ability. However, some issues, such as adaptability, safety, and efficiency still restrict the development of exoskeleton technology. To address these problems, the remaining open challenges and future directions to improve intelligence, sensing, gait analysis, trust, efficiency, generalization, and power consumption of exoskeleton are also presented and discussed.
5 September 2022 in IEEE Transactions on Instrumentation and Measurement
Gait phase contains rich kinematic information of lower limbs, which has great reference significance for rehabilitation medicine, assistive design, and identity recognition. This research presents a wearable gait phase segmentation method based on lower limb motion capture (MoCap) technique. In our method, a body sensor network (BSN) covering the whole lower limbs was established to capture the motion data of human gait, and a 3-D lower limb dynamic model is created to reconstruct lower limb movements through multisensor data fusion. Six gait events are labeled by the lower limb dynamic model. Then, a deep classification network combining temporal convolutional network (TCN) and long-short-term memory (LSTM) is proposed to segment the six gait phases as pattern classification. In addition, different sensor combinations for gait phase segmentation were also evaluated to select optimal sensor layouts. Detection performance is evaluated using metrics of accuracy, specificity, recall, and F1 score, and the averaged performance values are 98.9%, 98.9%, 98.8%, and 98.9%, respectively. The overall experimental results demonstrate that our proposed method can well address the issue of gait phase segmentation and provide spatial–temporal parameters for further gait analysis.