Foreign object intrusion detection is vital to ensure the safety of railway transportation. Recently, object detection algorithms based on deep learning have been applied in a wide range of fields. However, in complex and volatile railway environments, high false detection, missed detection, and poor timeliness still exist in traditional object detection methods. To address these problems, an efficient railway foreign object intrusion detection approach SDRC-YOLO is proposed. First, a hybrid attention mechanism that fuses local representation ability is proposed to improve the identification accuracy of small targets. Second, DW-Decoupled Head is proposed to construct a mixed feature channel to improve localization and classification ability. Third, a large convolution kernel is applied to build a larger receptive field and improve the feature extraction capability of the network. In addition, the lightweight universal upsampling operator CARAFE is employed to sample the size and proportion of the intruding foreign body features in order to accelerate the convergence speed of the network. Experimental results show that, compared with the baseline YOLOv5s algorithm, SDRC-YOLO improved the mean average precision (mAP) by 2.8% and 1.8% on datasets RS and Pascal VOC 2012, respectively.