The wide application of industrial robots has greatly improved assembly efficiency and reliability. However, determining how to efficiently teach a robot to perform assembly manipulation trajectories from demonstration videos is a challenging issue. This paper proposes a method integrating deep learning, image processing, and an iteration model to predict the real assembly manipulation trajectory of a human hand from a video without specific depth information. First, a pose estimation network, Keypoint-RCNN, is used to accurately estimate hand pose in the two-dimensional (2-D) image of each frame in a video. Second, image processing is applied to map the 2-D hand pose estimated by the neural network with the real 3-D assembly space. An iteration model based on the trust region algorithm is proposed to solve for the quaternions and translation vectors of two frames. All the quaternions and translation vectors form the predicted assembly manipulation trajectories. Finally, a UR3 robot is used to imitate the assembly operation based on the predicted manipulation trajectories. The results show that the robot could successfully imitate various operations based on the predicted manipulation trajectories.

Bolted joints are widely used in industries to connect and fasten two separated components. During the assembly of bolted joints, torque wrench is commonly employed to control preload indirectly. Much scatter in preload will occur and decrease the reliability of mechanical product. This study invented a novel tightening tool, which could directly control preload with high accuracy in the process of tightening bolts. The novel tightening tool is composed of an ameliorative intelligent wrench and signal processing system. There are two characteristics or innovations in the intelligent wrench. First, an electric tripping structure with multi-level small-size lever is designed to ensure that the target preload can be controlled effectively when the threshold value is reached. Second, an intelligent socket integrated with PZT sensor is designed, which can avoid many PZT sensors being sticked on bolt heads and greatly reduce cost. In the signal processing system, the waveform tracking strategy based on phase-locked loop is proposed to obtain the stable phase difference, which improves the robustness of the calculation of echo time. Three types of bolts with different sizes were tested. The average preload error was approximatively 2%, validating the accuracy and effectiveness of our tightening tool.

Threaded fasteners are widely applied in mechanical systems, providing the functions of connection, fastening, and sealing. However, loosening is vulnerable to occurring in harsh environment. The importance of loosening detection cannot be emphasized. Percussion-based loosening detection method has attracted much attention due to the convenience and low cost. However, the simultaneous loosening detection of multiple-threaded fasteners based on percussion method is still a challenging issue that needs to be addressed. This study proposes a novel multi-bolt loosening detection method combining percussion method, and deep learning. The method consists of three integrated modules, that is, signal preprocessing, loosening information enhancement, and loosening detection modules. In the first module, variational mode decomposition is used to decompose the original signal into a series of intrinsic mode function to eliminate the interference of noise. In the second module, compressive sampling matching pursuit is applied to represent the denoised signal sparsely, and the sparse signal is fused with the denoised signal to enhance loosening information in the signal. Last, DenseNet-CBAM network structure combining attention mechanism is proposed for multiple classification task. Experimental results showed that the proposed method achieved the detection accuracy of more than 97% in three different types of mechanical structures with multiple-threaded fasteners, indicating its great potentials in engineering applications.

As the demands for assembly quality and efficiency increase, robot-assisted assembly applications are becoming more widespread. Peg-in-hole assembly, as a typical form of assembly, has been widely researched by scholars. Currently, robotic peg-in-hole assembly faces challenges such as complex analysis of part contact forces, difficulties in task modeling, and the failure of traditional strategies. Simply controlling the position of the robot's end effector cannot achieve high precision, high efficiency peg-in-hole assembly. Flexible assembly, especially intelligent flexible assembly, is becoming the future development trend. So there is a lack of comprehensive reviews on robotic flexible peg-in-hole assembly. This paper first outlines the basic components of peg-in-hole assembly and summarizes the two basic operational processes of peg-in-hole assembly, along with their related theoretical foundations. We then review and analyze the research on passive compliant assembly, active compliant assembly, and intelligent flexible assembly. Finally, it presents an outlook on the future development directions of robotic peg-in-hole assembly.