Transmission lines are often located in complex environments and are susceptible to the presence of foreign objects. Failure to promptly address these objects can result in accidents, including short circuits and fires. Existing foreign object detection networks face several challenges, such as high levels of memory consumption, slow detection speeds, and susceptibility to background interference. To address these issues, this paper proposes a lightweight detection network based on deep learning, namely YOLOv5 with an improved version of CSPDarknet and a Swin Transformer (YOLOv5-IC-ST). YOLOv5-IC-ST was developed by incorporating the Swin Transformer into YOLOv5, thereby reducing the impact of background information on the model. Furthermore, the improved CSPDarknet (IC) enhances the model’s feature-extraction capability while reducing the number of parameters. To evaluate the model’s performance, a dataset specific to foreign objects on transmission lines was constructed. The experimental results demonstrate that compared to other single-stage networks such as YOLOv4, YOLOv5, and YOLOv7, YOLOv5-IC-ST achieves superior detection results, with a mean average precision (mAP) of 98.4%, a detection speed of 92.8 frames per second (FPS), and a compact model size of 10.3 MB. These findings highlight that the proposed network is well suited for deployment on embedded devices such as UAVs.