6 July 2020 in IEEE Transactions on Instrumentation and Measurement
Semiconductor wafer is widely used in welding robot, spray robot, unmanned material delivery vehicle and detection station sensor. The defects of semiconductor wafer, such as stains, burrs, scratches and holes generated in the manufacturing process, severely affect the quality of downstream products. Therefore, the inspection of wafer defect could not be neglected. Traditional semiconductor wafer defect inspection methods based on hand-craft features heavily rely on the expertise and are limited in some application scenario. In this paper, a novel method based on deep convolutional neural networks for semiconductor wafer surface defect inspection is proposed. Firstly, a new structure of feature pyramid networks with atrous convolution (FPNAC) is developed to extract the features and to generate feature maps. Secondly, the feature maps are fed into region proposal network (RPN) to generate region proposals. Finally, the region proposals are aligned to corresponding size as the inputs of deep multi-branches neural network (DMBNN) consisting of three branches, to classify and segment the defects precisely. Experimental results demonstrate that the proposed method yields good comprehensive performance with Mean Pixel Accuracy (MPA) 93.97% and Mean Intersection over Union (MIoU) 83.58%.
16 September 2022 in Measurement Science and Technology
Accurate and efficient fault diagnosis in rotating machinery has long been important and challenging, as it strongly affects the system reliability and safety of industrial applications. In recent years, deep-learning-based methods are developing rapidly and have been widely used in different areas. However, most of them are data-driven and focus on the architecture and design of convolutional neural network (CNN) models, while neglecting the representation of information itself. The intrinsic characteristics of the signal can not fully explored. Moreover, rich multidirectional information hidden inside the signal, which is the key to improving the predictive performance of the entire fault diagnosis model, has usually been ignored. In this work, we propose a multimodal neural-network-based model to pursue feature representation more efficiently and effectively and further improve the diagnostic performance. This method innovatively combines continuous wavelet transform and symmetrized dot pattern graphs through the channel information fusion mechanism after the two-dimensional domain modal transformation of the time-domain signal. The integration of one- and two-dimensional convolutions could fully utilize the feature extraction capability of CNN for multimodal signals, thus forming a multimodal CNN architecture under two-level information fusion. The experiment results prove that the designed model can achieve better performance than the traditional single-modal CNN structure.
31 October 2023 in IEEE Transactions on Automation Science and Engineering
The decline of point cloud registration efficiency caused by bad initial position and disordered registration direction has not been effectively solved. Herein, we propose a robust registration algorithm to tackle these drawbacks. First, a novel automatic point cloud alignment strategy considering the normal vector of feature points is demonstrated. This strategy ensures fast convergence in the case of bad initial position. Second, we introduce a cross iterative optimization strategy, which combines the alignment algorithm with an improved ICP (Point-Surface ICP) version based on surface constraints to complete faster and more orderly registration. In order to reduce the computational complexity, we present a linearization for the Point-Surface ICP based on Rodrigues rotation parameterization with the small incremental rotation assumption. In the elimination of outliers, we use the normal distribution of multiple errors to automatically select the threshold interval. Eventually, a large number of experiments are conducted on some public data-sets for performance evaluation of the as-proposed algorithm. Compared with other optimal methods, our method achieves a 17.1 $\%$ and 58.98 $\%$ increase in registration accuracy in Dragon dataset and Armadillo dataset, respectively, indicating the higher superiority of our algorithm. Note to Practitioners —This paper was motivated by solving the problem of registering two PCs. Most existing approaches generally can’t solve the decline of point cloud registration efficiency caused by bad initial position and disordered registration direction. In this paper, the position information and normal vector information of feature points are considered as the constraint conditions of pose alignment, and the improved ICP is used for further registration. In order to reduce the influence of outliers, an adaptive comprehensive elimination condition is proposed. We have demonstrated through extensive experiments that the proposed registration algorithm achieves improved accuracy, robustness to point clouds of different scales, and faster convergence speed.
19 February 2021 in IEEE Transactions on Instrumentation and Measurement
Surface defects on optical fiber end-face, such as scratches, cause heavy loss and low data-transfer rate. Therefore, the inspection of optical fiber end-face defect cannot be neglected. Most optical fiber defect detection methods ignore the striped pattern of scratches and easily cause incomplete segmentation. In this article, a novel method based on deep convolutional neural networks (DCNNs) for weak microscratches inspection in the optical fiber end-face is proposed. First, a new feature pyramid structure with criss-cross max-pooling (CCMP) block is developed to extract precise boundary features in all directions and generate feature maps. Second, these feature maps are fed into the spatial attention upsample (SAU) module to recover a full-resolution feature map and localize the defects. Finally, multiple scratches with various types can be detected in the images. This article uses many real optical fiber images as experimental images. The experimental results demonstrate that the proposed method yields good comprehensive performance with a mean pixel accuracy (mPA) of 92.70% and a mean intersection over union (mIoU) of 85.35%.
30 November 2021 in IEEE Sensors Journal
Automatic anomaly detection has always been a very important and challenging subject, which strongly affects product quality. This paper, proposes a novel weakly supervised ensemble learning based framework for automated pixel-wise anomaly detection, which enables the model to be trained using only a small quantity of labelled samples, instead of hundreds or thousands or more, meanwhile it maintains excellent performance in practical applications. Specifically, first, synthesized defects with random shapes, texture and brightness are injected into fault-free samples for preliminary training. Then homographic adaptation is introduced, and Siamese homography method is employed to further improve the training performance by limiting the repeatability and consistency of results in different Siamese network channels. While in model inference module, a multi-modal boosting method with adaptive weights is utilized instead of prediction on purely original sample images. Experiments are performed on several typical open industry anomaly detection datasets, and experimental results show that our method achieves better performance when compared with other classic superior methods. Typically, the segmentation metric mIOU can reach 85.79%, 78.38% and 76.65% on these datasets, specifically.
28 November 2023 in IEEE Transactions on Instrumentation and Measurement
Recently, deep-learning-based methods have achieved excellent performance in semiconductor wafer defect inspection. However, these trained networks are only effective for a certain background pattern and fail in defect transfer detection between multiple background patterns. Once the background pattern has changed, images are needed to be recollected and relabeled, which is very time-consuming. To achieve defect transfer detection in semiconductor wafers, a novel prototype learning (PL)-based network is proposed in this article. First, a multiscale residual fusion module (MSRFM) is elaborately designed to generate feature maps that minimize the interference of background patterns. Second, some labeled defect images are chosen as a defect library, which is further extracted into a defect prototype. Then, a distance map will be calculated between feature maps and the defect prototype, resulting in a precise prediction. Finally, an extra self-constraint loss ( $L_{\text {sc}}$ ) is designed to regulate the defect prototype calculation process. Different from previous methods, only defect-free images are required for the proposed method to achieve defect transfer detection. Experiments on real-world semiconductor wafer production lines show that the proposed method achieves mean intersection over union (mIoU) of 83.49% and 80.12% in defect transfer detection between two background pattern wafers. Furthermore, the excellent performance on other classical industrial datasets demonstrates that the proposed network has great robustness to various defects and industrial scenarios.