Aiming at the fault states of split pins in the catenary fastening devices of high-speed railway, such as missing, loosening and improper installation, a method for detecting defects of split pins is proposed. Firstly, split pins are localized by a two-stage positioning method based on the improved YOLOV3 algorithm. The first stage is used to localize five joint components on catenary support devices and the second stage is applied to locate the split pins in the joint component images. Then, the deeplabv3+ algorithm is implemented for semantic segmentation on split pin images. Finally, split pins are classified based on semantic segmentation images in accordance with split pins’ semantic information of head, body and tail. In order to verify the adaptability and accuracy of the proposed method, the catenary support devices images in multi high-speed railway lines and multi environments were tested. Meanwhile, our algorithm is compared with other deep learning algorithm. The results show that the proposed method has a higher accuracy in detecting defects of split pins, which can guarantee the stable operation of catenary support devices.