13 February 2025 in IEEE Transactions on Instrumentation and Measurement
Surface defect recognition (SDR) with limited data is a common challenge in industrial production. Recent methods generally utilize generative adversarial networks (GANs) to generate defect samples as training data for improving the performance of SDR. However, the instability of GAN training often results in uncontrollable and low-quality samples under severe data constraints, making it difficult for the existing methods to effectively handle SDR tasks with different granularities. To address the issue, this article proposes a human-guided data augmentation method under extremely limited data. Its core idea is to introduce human feedback into a diffusion model for synthesizing controllable and high-quality defect samples by reinforcement learning (RL), aiming to improve various-granularity SDR tasks such as defect classification and segmentation. First, a conditional diffusion model (CDM) is constructed to generate controllable defect samples using semantic labels, which learn defect distribution from a small number of annotated defect samples. Then, a reward model is designed to evaluate the outcome of the CDM by human feedback. Next, based on the trained reward model, the CDM is further optimized by proximal policy optimization (PPO). Finally, the refined CDM is used to generate high-quality defect samples as training data for enhancing defect classification and segmentation. Extensive experiments on NEU-Seg, magnetic-tile (MT), and the collected Tire datasets demonstrate that our method outperforms the state-of-the-art generative methods in terms of generated image quality. Furthermore, the performance of defect classification and segmentation has also shown significant enhancements based on the generated samples, with a maximum improvement of 16.90% in accuracy and 12.85% in mean intersection over union (mIoU) compared to results obtained without data augmentation.
25 February 2026 in IEEE Transactions on Industrial Informatics
Few-shot industrial defect segmentation (FIDS) is an extremely challenging task in industrial inspection, which focuses on segmenting unseen defect categories with only a few samples. Existing few-shot segmentation methods are commonly developed with constrained feature extraction capabilities, making it difficult to accurately segment diverse unseen defects from distinctive industrial scenarios. To this end, we aim to leverage strong generalization strengths of the large-scale visual foundation model, segment anything model (SAM), to handle FIDS task. However, simply incorporating SAM into FIDS is insufficient for automatically categorizing defects, as the SAM heavily relies on manual prompts for segmentation. To address this, we propose a novel reference-defect prompt learning method RDPrompter, which learns adaptive prompts from defect similarities based on SAM for guiding automated FIDS. Specifically, to obtain adaptive prompts, we introduce a multiscale feature extraction strategy to mine rich defect features by the image encoder of SAM. Then, self-correcting probability prototype and cosine similarity attention are proposed to form a mixed similarity aggregation module, which obtains multiscale feature similarities for defect localization. Based on the similarities, a prompt embedding construction module is designed to further extract fine defect location information and generate prompt embeddings for FIDS. Sufficient experiments on MVTec-Unseen, SDD, FSSD-12 and the collected CID datasets demonstrate the superiority of our method.
28 October 2025 in Expert Systems with Applications
1 January 2026 in IEEE Transactions on Automation Science and Engineering