To address the challenges of high missed detection rates for minute transmission line defects, strong complex background interference, and limited computational power on edge devices in UAV-assisted power line inspection, this paper proposes a lightweight improved YOLOv12 real-time detection model. First, a Bidirectional Weighted Feature Fusion Network (BiFPN) is introduced to enhance bidirectional interaction between shallow localization information and deep semantic features through learnable feature layer weighting, thereby improving detection sensitivity for line defects. Second, a Cross-stage Channel-Position Collaborative Attention (CPCA) module is embedded in the BiFPN’s cross-stage connections, jointly modeling channel feature significance and spatial contextual relationships to effectively suppress complex background noise from vegetation occlusion and metal reflections while enhancing defect feature representation. Furthermore, the backbone network is reconstructed using ShuffleNetV2’s channel rearrangement and grouped convolution strategies to reduce model complexity. Experimental results demonstrate that the improved model achieved 98.7% mAP@0.5 on our custom transmission line defect dataset, representing a 3.0% improvement over the baseline YOLOv12, with parameters compressed to 2.31M (8.3% reduction) and real-time detection speed reaching 142.7 FPS. This method effectively balances detection accuracy and inference efficiency, providing reliable technical support for unmanned intelligent inspection of transmission lines.