As the basic visual morphological characteristics of molten pool, contour extraction plays an important role in on-line monitoring of welding quality. The limitations of traditional edge detection algorithms make deep learning play a more important role in the task of target segmentation. In this paper, a molten pool visual sensing system in a tungsten inert gas welding (TIG) process environment is established and the corresponding molten pool image data set is made. Based on a residual network, a multi-scale feature fusion semantic segmentation network Res-Seg is designed. In order to further improve the generalization ability of the network model, this paper uses deep convolutional generative adversarial networks (DCGAN) to supplement the molten pool data set, then performs color and morphological data enhancement before network training. By comparing with other traditional edge detection algorithms and semantic segmentation network, it is verified that the scheme has high accuracy and robustness in the actual welding environment. Moreover, a back propagation (BP) neural network is used to predict the weld width, and a fitting test is carried out for the pixel width of the molten pool and its corresponding actual weld width. The average testing error is less than 0.2 mm, which meets the welding accuracy requirements.

Vision-based welding seam tracking is an important and unique branch of welding automation. Active vision seam tracking systems achieve accurate feature extraction by using an auxiliary light source, but this will introduce extra costs and the real-time performance will be affected. In contrast, passive vision systems achieve better real-time performance and their structure is relatively simple. This paper proposes a passive vision welding seam tracking system in Plasma Arc Welding (PAW) based on semantic segmentation. The BiseNetV2 network is adopted in this paper and online hard example mining (OHEM) is used to improve the segmentation effect. This network structure is a lightweight structure allowing effective image feature extraction. According to the segmentation results, the offset between the welding seam and the welding torch can be calculated. The results of the experiments show that the proposed method can achieve 57 FPS and the average error of the offset calculation is within 0.07 mm, meaning it can be used for real-time seam tracking.

Infrared-visible fusion has great potential in night-vision enhancement for intelligent vehicles. The fusion performance depends on fusion rules that balance target saliency and visual perception. However, most existing methods do not have explicit and effective rules, which leads to the poor contrast and saliency of the target. In this paper, we propose the SGVPGAN, an adversarial framework for high-quality infrared-visible image fusion, which consists of an infrared-visible image fusion network based on Adversarial Semantic Guidance (ASG) and Adversarial Visual Perception (AVP) modules. Specifically, the ASG module transfers the semantics of the target and background to the fusion process for target highlighting. The AVP module analyzes the visual features from the global structure and local details of the visible and fusion images and then guides the fusion network to adaptively generate a weight map of signal completion so that the resulting fusion images possess a natural and visible appearance. We construct a joint distribution function between the fusion images and the corresponding semantics and use the discriminator to improve the fusion performance in terms of natural appearance and target saliency. Experimental results demonstrate that our proposed ASG and AVP modules can effectively guide the image-fusion process by selectively preserving the details in visible images and the salient information of targets in infrared images. The SGVPGAN exhibits significant improvements over other fusion methods.

Butt welding is an important link to ensure welding quality, and the penetration state of the weld is the main criterion to achieve this. Online monitoring and control of the penetration state of welded joints is an important measure to ensure welding quality. The molten pool image is monitored by a visual sensor in the gas metal arc welding (GMAW) process, and the bottom molten pool width is predicted by the regression network model. Combined with the real-time control method, the welding current is changed to monitor and control the bottom weld width in real time. Butt-welding experiments with different groove angles verified that the proposed method could achieve satisfactory control accuracy and generalization ability. For butt-welding experiments with constant groove angles of 30° and 45°, the MAE of the controlled backside melt width to the target values was 0.2603 mm and 0.2620 mm. Therefore, it provides a feasible method for the online control of weld penetration.