In the area of transmission line inspection, one of the prominent areas of research has been to unite Unmanned Aerial Vehicles (UAVs) with neural network object detection algorithms. This area of research is challenging because of high computational resource consumption and poor infrared detection capabilities. In this study we propose an infrared image detection algorithm, named IISD-YOLO, using a modified version of the YOLOv11 network, to detect infrared transmission line insulator strings. Firstly, the original object detection layer was removed and replaced with the ShuffleNetv2 network to achieve the goal of a lightweight model; subsequently, based on the original feature extraction module C3k2, the Manhattan Self-Attention (MaSA) mechanism was introduced to design a new feature extraction module, C3k2-MaSA, which enhances the feature extraction capability for infrared objects; finally, the bidirectional feature pyramid network (Bi-FPN) is used to replace the original feature fusion module, enhancing the network’s ability to process and fuse information at different scales. The comparative experiments show that compared with the mainstream YOLO models, IISD-YOLO has improved by 4.5, 6.1, and 4.8 percentage points respectively on mAP@50 over YOLOv5, YOLOv8, and YOLOv10; furthermore, this model outperforms advanced models including YOLO-CIR, FA-YOLO, YOFIR, and RT-DETR, with improvements of 2.9, 9.1, 5.0, and 1.1 percentage points respectively on mAP@50. The ablation study shows that each improvement effectively enhances the overall performance. Compared with the original YOLOv11, the IISD-YOLO has increased its mAP@50 by 3.5 percentage points, while reducing the number of Params by 1.1 million and the computational GFLOPs by 2 G. These results confirm the superior performance of IISD-YOLO in infrared insulator string detection.