Aiming at the problems of low efficiency and poor accuracy in the product surface defect detection. In this paper, an online surface defects detection method based on YOLOV3 is proposed. Firstly, using lightweight network MobileNetV2 to replace the original backbone as the feature extractor to improve network speed. Then, we propose an extended feature pyramid network (EFPN) to extend the detection layer for multi-size object detection and design a novel feature fusing module (FFM) embedded in the extend layer to super-resolve features and capture more regional details. In addition, we add an IoU loss function to solve the mismatch between classification and bounding box regression. The proposed method is used to train and test on the hot rolled steel open dataset NEU-DET, which contains six typical defects of a steel surface, namely rolled-in scale, patches, crazing, pitted surface, inclusion and scratches. The experimental results show that our method achieves a satisfactory balance between performance and consumption and reaches 86.96% mAP with a speed of 80.96 FPS, which is more accurate and faster than many other algorithms and can realize real-time and high-precision inspection of product surface defects.