With increasing interest in smart factories, considerable attention has been paid to the development of deep-learning-based quality inspection systems. Deep-learning-based quality inspection helps productivity improvements by solving the limitations of existing quality inspection methods (e.g., an inspector’s human errors, various defects, and so on). In this study, we propose an optimized YOLO (You Only Look Once) v5-based model for inspecting small coils. Performance improvement techniques (model structure modification, model scaling, pruning) are applied for model optimization. Furthermore, the model is prepared by adding data applied with histogram equalization to improve model performance. Compared with the base model, the proposed YOLOv5 model takes nearly half the time for coil inspection and improves the accuracy of inspection by up to approximately 1.6%, thereby enhancing the reliability and productivity of the final products.