In this paper, we address the semantic segmentation problem using single-layer networks. This network is used for unsupervised feature learning for the available RGB image and the depth image. A significant contribution of the proposed approach is that the dictionary is selected from the existing samples using the L _{2, 1} optimization. Such a dictionary can capture more meaningful representative samples and exploit intrinsic correlation between features from different modalities. The experimental results on the public NYU dataset show that this strategy dramatically improves the classification performance, compared with existing dictionary learning approach. In addition, we perform experimental verification using the practical robot platforms and show promising results.

In order to adapt to the high standard of defect detection in industrial production for electronic component soldering, and to address the issues of unbalanced integrated circuit (IC) defect samples and the difficulty of defect detection and segmentation, this paper proposes a model based on the Swin Transformer architecture for automatic detection and segmentation of solder defects in electronic components. The approach involves feeding real defect samples collected from industrial production into the Cycle-Generative Adversarial Network, which generates a balanced and sufficient sample dataset through adversarial generation. The Swin Transformer framework is then applied to the current mainstream Mask R-CNN, resulting in an improved network model named Swin Transformer Based Mask R-CNN (ST-mask-rcnn). This model effectively detects IC pin soldering defects and solder joints of surface mounted devices (SMD). The experimental results demonstrate that the proposed method successfully addresses the sample imbalance problem in electronic components. It achieves a 6% and 18% improvement in mean average precision for IC pin soldering defects and SMD resistive solder joints detection, respectively. Additionally, it enhances the segmentation accuracy to 99.81% and 99.92%, with improved detection speed.

In order to improve the efficiency and accuracy of multitarget detection of soldering defects on surface-mounted components in Printed Circuit Board (PCB) fabrication, we propose a sample generation method using Stable Diffusion Model and ControlNet, as well as a defect detection method based on the Swin Transformer. The method consists of two stages: First, high-definition original images collected in industrial production and the corresponding prompts are input to Stable Diffusion Model and ControlNet for automatic generation of nonindependent samples. Subsequently, we integrate Swin Transformer as the backbone into the Cascade Mask R-CNN to improve the quality of defect features extracted from the samples for accurate detection box localization and segmentation. Instead of segmenting individual components on the PCB, the method inspects all components in the field of view simultaneously over a larger area. The experimental results demonstrate the effectiveness of our method in scaling up nonindependent sample datasets, thereby enabling the generation of high-quality datasets. The method accurately recognizes targets and detects defect types when performing multitarget inspection on printed circuit boards. The analysis against other models shows that our improved defect detection and segmentation method improves the Average Recall (AR) by 2.8% and the mean Average Precision (mAP) by 1.9%.

Printed circuit board (PCB) assembly, on the basis of surface mount electronic component welding, is one of the most important electronic assembly processes, and its defect detection is also an important part of industrial generation. The traditional two-stage target detection algorithm model has a large number of parameters and the runtime is too long. The single-stage target detection algorithm has a faster running time, but the detection accuracy needs to be improved. To solve this problem, we innovated and modified the YOLO11n model. Firstly, we used the Retention Block (RetBlock) to improve the C3K2 module in the backbone, creating the RetC3K2 module, which makes up for the limitation of the original module’s limited, purely convolutional local receptive field. Secondly, the neck structure of the original model network is fused with a Multi-Branch Auxiliary Feature Pyramid Network (MAFPN) structure and turned into a multi-branch auxiliary neck network, which enhances the model’s ability to fuse multiple scaled characteristics and conveys diverse information about the gradient for the output layer. The improved YOLO11n model improves its mAP50 by 0.023 (2.5%) and mAP75 by 0.026 (2.8%) in comparison with the primitive model network, and detection precision is significantly improved, proving the superiority of our proposed approach.

Soldering of printed circuit board (PCB)-based surface-mounted assemblies is a critical process, and to enhance the accuracy of detecting their multi-targeted soldering defects, we propose an automated sample generation method that combines ControlNet and a Stable Diffusion Model. This method can expand the dataset by quickly obtaining sample images with high quality containing both defects and normal detection targets. Meanwhile, we propose the Cascade Mask R-CNN model with ConvNeXt as the backbone, which performs well in dealing with multi-target defect detection tasks. Unlike previous detection methods that can only detect a single component, it can detect all components in the region. The results of the experiment demonstrate that the detection accuracy of our proposed approach is significantly enhanced over the previous convolutional neural network model, with an increase of more than 10.5% in the mean accuracy precision (mAP) and 9.5% in the average recall (AR).

When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to imbalances in sample datasets and the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network (GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to automatically generate masks for the defect images. Experimental results demonstrate the effectiveness of our proposed method, as it can simultaneously generate liquid crystal image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.