Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model’s recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%.

Classification of foot–ground contact phases, as well as the swing phase is essential in biomechanics domains where lower-limb motion analysis is required; this analysis is used for lower-limb rehabilitation, walking gait analysis and improvement, and exoskeleton motion capture. In this study, sliding-window label overlapping of time-series wearable motion data in training dataset acquisition is proposed to accurately detect foot–ground contact phases, which are composed of 3 sub-phases as well as the swing phase, at a frequency of 100 Hz with a convolutional neural network (CNN) architecture. We not only succeeded in developing a real-time CNN model for learning and obtaining a test accuracy of 99.8% or higher, but also confirmed that its validation accuracy was close to 85%.

This paper proposed a multi-layered 3D NDT (normal distribution transform) scan-matching approach for robust localization even in the highly dynamic environment of warehouse logistics. Our approach partitioned a given 3D point-cloud map and the scan measurements into several layers regarding the degree of environmental changes in the height direction and computed the covariance estimates for each layer using 3D NDT scan-matching. Because the covariance determinant is the estimate’s uncertainty, we can determine which layers are better to use in the localization in the warehouse. If the layer gets close to the warehouse’s floor, the degree of environmental changes, such as the cluttered warehouse layout and position of boxes, would be significantly large, while it has many good features for scan-matching. If the observation at a specific layer is not explained well enough, then the layer for localization can be switched to other layers with lower uncertainties. Thus, the main novelty of this approach is that localization robustness can be improved even in very cluttered and dynamic environments. This study also provides the simulation-based validation using Nvidia’s Omniverse Isaac sim and detailed mathematical descriptions for the proposed method. Moreover, the evaluated results of this study can be a good starting point for further mitigating the effects of occlusion in warehouse navigation of mobile robots.

The accuracy of the wearable inertia-measurement-unit (IMU)-sensor-based gesture recognition may be significantly affected by undesired changes in the body-fixed frame and the sensor-fixed frame according to the change in the subject and the sensor attachment. In this study, we proposed a novel wearable IMU-sensor-based hand-guiding gesture recognition method robust to significant changes in the subject’s body alignment based on the floating body-fixed frame method and the bi-directional long short-term memory (bi-LSTM). Through comparative experimental studies with the other two methods, it was confirmed that aligning the sensor-fixed frame with the reference frame of the human body and updating the reference frame according to the change in the subject’s body-heading direction helped improve the generalization performance of the gesture recognition model. As a result, the proposed floating body-fixed frame method showed a 91.7% test accuracy, confirming that it was appropriate for gesture recognition under significant changes in the subject’s body alignment during gestures.

This paper describes a new image augmentation method based on a DCGAN considering mode switching in terms of transient learning accuracy threshold on the problem of human gesture recognition through imaging of wearable sensor time-series data and a deep CNN structure. Because the discriminator in GANs learns faster than the generator, it is known that mode collapse occurs, in which only image modes biased to a specific image type are augmented among various image forms. In this study, to solve the mode collapse caused by the learning difficulty mismatch between networks, we add a learning mode switching layer between the generator and discriminator and receive feedback from both networks transient learning accuracies to switch the learning mode if predefined thresholds are exceeded. We confirm that the proposed approach balanced the learning rate between the generator and discriminator networks, resolved the mode collapse problem, and increased the test accuracy of a deep CNN trained with an augmented image set by approximately 20.35% compared to a conventional DCGAN. In addition, it showed better accuracy on a performance comparison with other improved DCGAN methods.

This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.