In recent years, steel surface defect detection has emerged as a significant area of focus within intelligent manufacturing research. Existing approaches often exhibit insufficient accuracy and limited generalization capability, constraining their practical implementation in industrial environments. To overcome these shortcomings, this study presents IMTS-YOLO, an enhanced detection model based on the YOLOv11n architecture, incorporating several technical innovations designed to improve detection performance. The proposed framework introduces four key enhancements. First, an Intelligent Guidance Mechanism (IGM) refines the feature extraction process to address semantic ambiguity and enhance cross-scenario adaptability, particularly for detecting complex defect patterns. Second, a multi-scale convolution module (MulBk) captures and integrates defect features across varying receptive fields, thereby improving the characterization of intricate surface textures. Third, a triple-head adaptive feature fusion (TASFF) structure enables more effective detection of irregularly shaped defects while maintaining computational efficiency. Finally, a specialized bounding box regression loss function (Shape-IoU) optimizes localization precision and training stability. The model achieved a 5.0% improvement in mAP50 and a 3.2% improvement in mAP50-95 on the NEU-DET dataset, while also achieving a 4.4% improvement in mAP50 and a 3.1% improvement in mAP50-95 in the cross-dataset GC10-DET validation. These results confirm the model’s practical value for real-time industrial defect inspection applications.