1 August 2022 in IEEE Transactions on Instrumentation and Measurement
In electronic manufacturing, anomaly detection of surface mount devices (SMDs) through computer vision (CV) is an important task to control the production quality of SMDs. The difficulty of the detection is that some anomalous regions on the surfaces of SMDs are very minor and with variable shapes, which leads to poor detection efficiency. To solve this problem, based on the assumption that normal samples can be reconstructed more accurately than anomalous samples, a self-supervised image anomaly detection framework with a multiscale two-branch feature fusion strategy is proposed. Specifically, it adopts autoencoder (AE) as the basic framework, and to enhance the reconstruction error between input anomalous samples and the reconstructed ones, a self-supervised learning task of reconstructing images is introduced to have the model neglect the encoding of the suspected anomalous regions found by a contextual attention mask (CAM) module. Meanwhile, a multiscale feature fusion strategy is developed to fuse texture and structure features in the decoder to reconstruct samples. Moreover, a multilevel anomalous score criterion is proposed to enlarge the scores for the samples with very minor anomalies. At last, an SMD-capacitor anomaly detection dataset (SMDC-DET) is built to evaluate the proposed method. The experiments show that the proposed method achieves an average area under the curve (AUC) accuracy of 98.82%, much better when compared to the start-of-the-art existing anomaly detection methods.
3 April 2025 in IEEE Transactions on Industrial Informatics
Constructing comprehensive multimodal feature representations from RGB images (RGB) and point clouds (PT) in 2D–3D multimodal anomaly detection (MAD) methods is very important to reveal various types of industrial anomalies. For multimodal representations, most of the existing MAD methods often consider the explicit spatial correspondence between the modality-specific features extracted from RGB and PT through space-aligned fusion, while overlook the implicit interaction relationships between them. In this study, we propose a uni-modal and cross-modal fusion (UCF) method, which comprehensively incorporates the implicit relationships within and between modalities in multimodal representations. Specifically, UCF first establishes uni-modal and cross-modal embeddings to capture intramodal and intermodal relationships through uni-modal reconstruction and cross-modal mapping. Then, an adaptive nonequal fusion method is proposed to develop fusion embeddings, with the aim of preserving the primary features and reducing interference of the uni-modal and cross-modal embeddings. Finally, uni-modal, cross-modal, and fusion embeddings are all collaborated to reveal anomalies existing in different modalities. Experiments conducted on the MVTec 3D-AD benchmark and the real-world surface mount inspection demonstrate that the proposed UCF outperforms existing approaches, particularly in precise anomaly localization.