Human kinetics, specifically joint moments and ground reaction forces (GRFs) can provide important clinical information and can be used to control assistive devices. Traditionally, collection of kinetics is mostly limited to the lab environment because it relies on data that are measured from a motion capture system and floor-embedded force plates to calculate the dynamics via musculoskeletal models. This spatially limited method makes it extremely challenging to measure kinetics outside the laboratory in a variety of walking conditions due to the expensive device setup and large space required. Recently, employing machine learning with IMU sensors are suggested as an alternative method for biomechanical analyses. Although these methods enable estimating human kinetic data outside the laboratory by linking IMU sensor data with kinetics dataset, they are limited to show inaccurate kinetic estimates even in highly repeatable single walking conditions due to the employment of generic deep learning algorithms. Thus, this paper proposes a novel deep learning model, Kinetics-FM-DLR-Ensemble-Net for single limb prediction of hip, knee, and ankle joint moments and 3 dimensional Ground Reaction Forces (GRFs) using three IMU sensors on the thigh, shank, and foot under several representatives walking conditions in daily living, such as treadmill, level-ground, stair, and ramp. This is the first study that implements both joint moments and GRFs in multiple walking conditions using IMU sensors via deep learning. Our deep learning model is versatile and accurate for identifying human kinetics across diverse subjects and walking conditions and our model outperforms state-of-the-art deep learning model for kinetics estimation by a large margin.
6 April 2022 in IEEE Journal of Biomedical and Health Informatics
Measurement of human body movement is an essential step in biomechanical analysis. The current standard for human motion capture systems uses infrared cameras to track reflective markers placed on the subject. While these systems can accurately track joint kinematics, the analyses are spatially limited to the lab environment. Though Inertial Measurement Unit (IMU) can eliminate the spatial limitations of the motion capture system, those systems are impractical for use in daily living due to the need for many sensors, typically one per body segment. Due to the need for practical and accurate estimation of joint kinematics, this study implements a reduced number of IMU sensors and employs machine learning algorithm to map sensor data to joint angles. Our developed algorithm estimates hip, knee, and ankle angles in the sagittal plane using two shoe-mounted IMU sensors in different practical walking conditions: treadmill, level overground, stair, and slope conditions. Specifically, we proposed five deep learning networks that use combinations of Convolutional Neural Networks (CNN) and Gated Recurrent Unit (GRU) based Recurrent Neural Networks (RNN) as base learners for our framework. Using those five baseline models, we proposed a novel framework, DeepBBWAE-Net, that implements ensemble techniques such as bagging, boosting, and weighted averaging to improve kinematic predictions. DeepBBWAE-Net predicts joint kinematics for the three joint angles under all the walking conditions with a Root Mean Square Error (RMSE) 6.93-29.0% lower than base models individually. This is the first study that uses a reduced number of IMU sensors to estimate kinematics in multiple walking environments.
1 November 2023 in IEEE Journal of Biomedical and Health Informatics
Reconstructing and predicting 3D human walking poses in unconstrained measurement environments have the potential to use for health monitoring systems for people with movement disabilities by assessing progression after treatments and providing information for assistive device controls. The latest pose estimation algorithms utilize motion capture systems, which capture data from IMU sensors and third-person view cameras. However, third-person views are not always possible for outpatients alone. Thus, we propose the wearable motion capture problem of reconstructing and predicting 3D human poses from the wearable IMU sensors and wearable cameras, which aids clinicians' diagnoses on patients out of clinics. To solve this problem, we introduce a novel Attention-Oriented Recurrent Neural Network (AttRNet) that contains a sensor-wise attention-oriented recurrent encoder, a reconstruction module, and a dynamic temporal attention-oriented recurrent decoder, to reconstruct the 3D human pose over time and predict the 3D human poses at the following time steps. To evaluate our approach, we collected a new WearableMotionCapture dataset using wearable IMUs and wearable video cameras, along with the musculoskeletal joint angle ground truth. The proposed AttRNet shows high accuracy on the new lower-limb WearableMotionCapture dataset, and it also outperforms the state-of-the-art methods on two public full-body pose datasets: DIP-IMU and TotalCaputre.
With the spread of COVID-19, significant emphasis has been placed on mitigation techniques such as mask wearing to slow infectious disease transmission. Widespread use of face coverings has revealed challenges such as mask contamination and waste, presenting an opportunity to improve the current technologies. In response, we have developed the Auto-sanitizing Retractable Mask Optimized for Reusability (ARMOR). ARMOR is a novel, reusable face covering that can be quickly disinfected using an array of ultraviolet C lamps contained within a wearable case. A nanomembrane UVC sensor was used to quantify the intensity of germicidal radiation at 18 different locations on the face covering and determine the necessary exposure time to inactivate SARS-CoV-2 in addition to other viruses and bacteria. After experimentation, it was found that ARMOR successfully provided germicidal radiation to all areas of the mask and will inactivate SARS-CoV-2 in approximately 180 s, H1N1 Influenza in 130 s, and Mycobacterium tuberculosis in 113 s, proving that this design is effective at eliminating a variety of pathogens and can serve as an alternative to traditional waste-producing disposable face masks. The accessibility, ease of use, and speed of sanitization supports the wide application of ARMOR in both clinical and public settings.
Human upper limb movement prediction from multi-modal wearable sensors has applications in rehabilitation, sports analysis, assistive device control, and human-computer interaction, offering less intrusive alternatives to camera-based motion capture (MoCap) systems. Yet, current datasets often lack comprehensive multi-modal data integrating inertial measurement unit (IMU) and surface electromyography (sEMG) sensors and have limited coverage of upper limb kinematics with insufficient fidelity to capture isolated single-joint elbow motions and coordinated multi-joint shoulder movements. To address these gaps, we introduce the Upper Limb Tracking with Multi-modal Capture (ULTRA-MoCap) dataset, which integrates multi-modal sensing with IMU, sEMG, and high-fidelity upper limb kinematics modeling from a multi-degree-of-freedom musculoskeletal model. Data from IMUs on the hand, wrist, and forearm, along with combined sEMG/IMU sensors on the biceps brachii, triceps brachii, and deltoid, were collected from thirteen subjects performing five upper limb movements at different speeds. This paper describes the data collection, sensor configuration, and processing pipeline and highlights ULTRA-MoCap’s potential to support a wide range of research across biomechanics, wearable sensing, and motion analysis.