The surface defects of continuous casting slab have an extremely negative impact on the quality and productivity of steel. Detection of surface defects poses significant challenges in real-time production line, such as excessive reliance on human intervention, low efficiency, and limited measurement accuracy. To tackle these challenges, this paper proposes a lightweight deep learning model called CCSNet, which can achieve efficient feature fusion while reducing computational costs. The new model solves the balance between accuracy and real-time performance of slab surface defect detection. Firstly, real-time photos of the slab surface were obtained using the visual inspection platform installed on the continuous casting production line. From these photos, a dataset containing five distinct kinds of defects was produced. Next, the model composition is introduced, and the feature information required for detection is generated by training the dataset. Finally, the suggested model's validity is demonstrated by model testing and comparison with other models. The CCSNet achieves an average accuracy of 90.1% on the self-made dataset, with the model's weight size being only 6 MB. The detection performance surpasses other traditional models, which can work for real-time defect detection under actual conditions.