A deep learning line laser 3D measurement method based on feature fusion and attention mechanism is proposed to address the impact of reflective workpieces on the extraction of laser stripe centers. Firstly, a UNet segmentation model based on feature fusion and attention mechanism is established. The deep learning model can effectively solve the interference caused by reflection and can segment the overall distribution and bending characteristics of laser stripes. Secondly, the Steger algorithm is used to roughly extract the center of the laser stripe, and the contour polygon segmentation method is used to adaptively obtain the segmentation points of the laser stripe. Finally, polynomial fitting is performed based on segmented points to obtain a smoother laser stripe centerline. The experimental results show that the proposed method can effectively overcome the interference caused by reflective workpieces, and generate a smoother 3D model, and the measurement repeatability error is less than 0.37mm, and the relative error is less than 0.03mm.

Robot plane weld seam tracking boasts stable quality and high efficiency. Nonetheless, extracting three-dimensional features of the seams proves challenging due to factors such as the high reflectivity of welding materials and uneven illumination. To tackle these challenges, this paper proposes a method for feature extraction and tracking of weld seams utilizing deep learning and structured light three-dimensional scanning imaging. Firstly, a precise segmentation model for laser stripes based on deep learning is established to address interference caused by workpiece reflection. Deep learning models effectively mitigate reflection interference and enhance segmentation of laser stripe distribution characteristics. Secondly, the gray center of gravity algorithm is utilized to roughly extract the center of the laser stripe, followed by smoothing the centerline of the stripe to attain a smoother center. Lastly, the least squares fitting method is utilized to generate the welding path, with coordinate transformation applied to the robot system for weld tracking. Experimental results demonstrate the effectiveness of the proposed method in overcoming reflection interference, with a root mean square error of the welding fitting path less than 0.17mm and an average tracking error less than 0.37mm.

Gesture control is often used for the control of robotic manipulators. However conventional methods always focus on position control of robotic manipulators and use a fixed position to place gesture detection devices, which limits the flexibility to control and interact. This paper presents an interactive gesture control system based on Leap Motion Controller that can overcome these shortcomings. A coordinate transformation is performed between the position of the left palm detected by Leap Motion Controller and the tool center point (TCP) of the collaborative manipulator to obtain gesture data first. This gesture data is used to control the posture of a six-joint collaborative manipulator-Lite6. The position of gripper is controlled by grip level of the right palm. A controller is designed to manipulate a lift table on which Leap Motion Controller is placed to achieve adaptive ascending and descending movements to meet the needs of operators of different heights and arm spans. A modified Kalman filter is used to filter noise and smooth the signal captured by Leap Motion Controller. The experiments demonstrate that the system can operate stably and accurately control the position, posture, and gripper position of a collaborative manipulator in real time using human palms’ gestures.

To address the challenges of uneven illumination, background noise, and low contrast in laser stripe center extraction in industrial environments such as measurement, this paper proposes a line laser 3D measurement method based on feature fusion. First, a multi-feature fusion detection model is constructed to obtain the brightness feature map and region-enhanced feature map of the laser stripe. Then, wavelet transform is used to fuse the brightness feature map and the region-enhanced feature map, followed by segmentation of the fused feature map using an adaptive maximum entropy method. Next, a gray-gravity method optimized by gradient direction and curvature is employed to determine the initial center points of the laser stripe. Finally, adaptive segmentation based on neighborhood differences is performed, and polynomial fitting is applied to the segmented stripes to obtain the final laser stripe centers. Experimental results show that, under a noise variance of 0.3, the maximum error in laser stripe extraction is less than 1.07 pixels, the width error between different rows is less than 0.24 mm, and the mean error of repeatability extraction is less than 0.13 mm.