7 July 2022 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
This article presents a disassembly task planning algorithm considering human–robot collaboration (HRC) and human behavior prediction (HBP). Unlike assembly procedures, the disassembly of end-of-life (EOL) products has been a labor-intensive process with uncertainties difficult to cope with. Meanwhile, it is usually challenging to obtain an optimal sequence efficiently without excessive computational cost. Also, the conventional human-centered task planning, in which the robot has to halt frequently due to unsafe interruptions by human motions, may decrease the efficiency of the disassembly process. In this article, a sequence planner is proposed to assign tasks in real time between a human operator and a robot to overcome the aforementioned challenges. The cost function includes the effort of the human and the robot in terms of both movement distance and time spent on the tasks. The constraints include the disassembly rules and the safety of the human operation. The optimal sequence is generated by solving an optimization problem in a receding-horizon way. In particular, at each step, the proposed disassembly sequence planner locates the workers (a human operator and a robot) and the to-be-disassembled components, predicts human movement for the next several steps, and obtains the optimal disassembly sequence for the next several steps following disassembly rules and safety constraints. Experiments have been extensively conducted on the disassembly of a wooden toybox and a used hard disk drive (HDD) to validate the proposed disassembly sequence planner. The planner has successfully generated the disassembly sequence in an HRC setting explicitly considering real-time human motion prediction and assigned the human operator and the robot to collaboratively complete disassembly tasks without violating disassembly rules and safety constraints.
29 May 2023 in IEEE/ASME Transactions on Mechatronics
While the disassembly of high-precision electronic devices is a predominantly labor-intensive process, collaborative robots provide a promising solution through human–robot collaboration. To ensure efficient yet safe collaboration, this article presents a new way to generate task-constrained and collision-free motion for a collaborative robot operating in a dynamic environment involving human movement, which is traditionally challenging due to the high degree of freedom of the corobot and the uncertainty nature of human motion. We first establish a neural human motion prediction model with quantified uncertainty, and then optimize the configuration of the robot online by taking the human motion and uncertainties into consideration. While such rationale is straightforward in nature, our method explicitly quantified the uncertainty of the neural human prediction model to further enhance the collaboration safety, and integrated the quantified uncertainty into the task-satisfied motion planning in real time to efficiently conduct tasks. Extensive experimental tests and comparison studies have been conducted to validate the efficiency and effectiveness of the proposed planning method.
8 June 2022 in IEEE/ASME Transactions on Mechatronics
Human motion prediction is the foundation stone of human–robot collaboration in intelligent manufacturing. The nonlinear and stochastic nature of human motion has made it challenging to predict the motion accurately. Many recent deep-learning-based approaches, e.g., convolutional neural networks or recurrent neural networks (RNNs), have been applied to address this challenge. On the other hand, existing works tend to ignore the importance of human dynamics in motion prediction, especially the effect of muscle force on the motion. This article proposes a novel dynamic model informed motion prediction method. It utilizes an unscented Kalman filter (UKF) to predict the state of the future arm dynamic model such that the future motion of the human arm can be obtained. In particular, the arm dynamic model is developed based on Lagrangian mechanics and represented by differential equations. Embracing the future muscle force predicted by RNN into the differential equations, such a dynamic model is capable of explicitly establishing the intrinsic relation between the future muscle force and the corresponding future arm motion. UKF is leveraged to predict the future joint position and velocity of the human arm based on the dynamic model. Experiments on three motion datasets validate that the proposed prediction method, compared with the traditional RNN-based prediction using skeleton vectors, significantly improves the prediction accuracy regarding elbow and wrist positions.
19 January 2024 in IEEE Robotics and Automation Letters
Ensuring the safety of human workers in a collaborative environment with robots is of utmost importance. Although accurate pose prediction models can help prevent collisions between human workers and robots, they are still susceptible to critical errors. In this study, we propose a novel approach called deep ensembles of temporal graph neural networks (DE-TGN) that not only accurately forecast human motion but also provide a measure of prediction uncertainty. By leveraging deep ensembles and employing stochastic Monte-Carlo dropout sampling, we construct a volumetric field representing a range of potential future human poses based on covariance ellipsoids. To validate our framework, we conducted experiments using three motion capture datasets including Human3.6M, and two human-robot interaction scenarios, achieving state-of-the-art prediction error. Moreover, we discovered that deep ensembles not only enable us to quantify uncertainty but also improve the accuracy of our predictions.
5 June 2024 in IEEE/ASME Transactions on Mechatronics
There has been a growing utilization of industrial robots as complementary collaborators for human workers in remanufacturing sites. Such a human–robot collaboration (HRC) aims to assist human workers in improving the flexibility and efficiency of labor-intensive tasks. In this article, we propose a human-aware motion planning framework for HRC to effectively compute collision-free motions for manipulators when conducting collaborative tasks with humans. We employ a neural human motion prediction model to enable proactive planning for manipulators. Particularly, rather than blindly trusting and utilizing predicted human trajectories in the manipulator planning, we quantify uncertainties of the neural prediction model to further ensure human safety. Moreover, we integrate the uncertainty-aware prediction into a graph that captures key workspace elements and illustrates their interconnections. Then, a graph neural network (GNN) is leveraged to operate on the constructed graph. Consequently, robot motion planning considers both the dependencies among all the elements in the workspace and the potential influence of future movements of human workers. We experimentally validate the proposed planning framework using a 6-degree-of-freedom manipulator in a shared workspace where a human is performing disassembling tasks. The results demonstrate the benefits of our approach in terms of improving the smoothness and safety of HRC. A brief video introduction of this work is available as the supplemental materials.
5 December 2022 in Journal of Vibration and Control
This work presents a framework aimed at mitigating adverse effects of high-amplitude drone noise ranging from hearing loss to reduced productivity in human–robot collaborative environments by infusing acoustic awareness in a path planning algorithm without imposing any additional design layers or hardware to an operational drone. Following a detailed outline of the proposed approach, it is shown that a significant reduction of noise levels perceived by human workers at noise-sensitive locations is realized via a path planner which generates optimal paths ranging from quietest to shortest paths. The approach is then augmented with a path-correction mechanism which accounts for noise exposure duration to ensure the aforementioned optimal paths are compliant with a given industrial/environmental standard. The correction mechanism enforces an adjustment of subsets of the planned paths inside quiet zones designated around noise-sensitive locations. The presented concepts were verified using numerical simulations conducted for a 2-dimensional rasterized obstacle field followed by a statistical design of experiments. The proposed framework is highly versatile and integrable with widely used industrial path planners, rendering it a highly valuable tool for noisy collaborative workplaces.