Intelligent defect detection technology combined with deep learning has gained widespread attention in recent years. However, the small number, and diverse and random nature, of defects on industrial surfaces pose a significant challenge to deep learning-based methods. Generating defect images can effectively solve this problem. This paper investigates and summarises traditional defect generation and deep learning-based methods. It analyses the various advantages and disadvantages of these methods and establishes a benchmark through classical adversarial networks and diffusion models. The performance of these methods in generating defect images is analysed through various indices. This paper discusses the existing methods, highlights the shortcomings and challenges in the field of defect image generation, and proposes future research directions. Finally, the paper concludes with a summary.