In order to improve the accuracy and detection effect of fabric defect detection, a fabric defect detection method based on local similarity comparison is proposed in this paper. This method first takes each pixel in the image as the central pixel, selects a specific window as the region size, and then uses the similarity between the central region and the surrounding neighborhood to find the neighborhood most similar to the central region to complete the estimation of the central pixel. Finally, the target image is obtained by the principle of background difference, so as to detect fabric defects. The results show that this method is superior to the traditional detection method, which can not only detect the defect image under the complex background, but also have good detection results for the fabric defect image under the influence of different organization and lighting factors. The detection accuracy rate under factory conditions can reach 98.45%, which has a high applicability and detection rate, and also demonstrates certain anti-interference performance.