This letter illustrates how a challenging problem be solved by assuring the adequacy of the raw information and using an appropriate deep learning network to extract the relevant information. The problem concerned is accurate monitoring of the penetration, in a fully penetrated weld pool, as quantified by the width of the weld on the back-side of the workpiece. This is challenging as the penetration occurs below the workpiece surface and is not visible. A popular method is to use a weld pool image to derive it. Analysis of the physical process suggests that a single weld pool does not contain adequate information but most recent serial weld pools may. As such, although a deep learning model may extract information that is already there, the raw information may not be sufficient. Hence, a model that is capable of extracting information from dynamic serial weld pool images is needed. To this end, a CNN-LSTM (convolutional neural network combined with long-short term memory one) model is proposed. Dynamic weld pools are experimentally generated by changing the welding current and welding speed randomly. The weld pools are imaged using an HDR camera during experiments. Images are also captured from the back-side surface of the workpiece to provide the ground truth for training, validation, and testing. It is found that the highly dynamically changing weld pool can be accurately predicted using serial weld pool images at 0.3 mm for its back-side bead width. Comparison has been made with results from comparative studies to verify the effectiveness of and the contribution from the information adequacy (by using serial images) and the feature extracting capability (by using deep learning).

This paper presents a virtual reality (VR) human-robot interaction welding system which allows human welders to manipulate a welding robot and undertake welding tasks naturally and intuitively via consumer-grade VR hardware (HTC Vive). In this system, human welders' operations are captured by motion-tracked handle controllers and used as commands to teleoperate a 6-DoF industrial robot (UR-5) and to request welding current from a controllable welding power supply (Liburdi Pulsweld P200). The 3D working scene is rendered in real time based on feedback information and shown to the human welder by head-mounted display (HMD) via a motion-tracked headset. To compensate for the time delay between command motion and real motion of the robot, a hidden Markov model (HMM) is proposed to model and predict human welders' operations. The K-means clustering algorithm is applied to cluster human welders' operation data (traveling speed) into latent states. Based on the developed prediction algorithm, the motion of human welders is predicted with an RMSE accuracy of between 2.1 and 4.6mm/s. The position data used as final commands to teleoperate a robot is predicted with an RMSE accuracy of between 1.1 and 2.3mm. This paper presents a general cyber-physical model for human-robot interactive welding based on VR, building a foundation for welding robot teleoperation.