Timely detection and handling of substation defects plays a foundational role in ensuring the stable operation of power systems. Existing substation defect detection methods fail to make full use of the temporal information contained in substation inspection samples, resulting in problems such as weak generalization ability and susceptibility to background interference. To address these issues, a change attention guided substation defect detection algorithm (CAG-Net) based on a dual-temporal encoder–decoder framework is proposed. The encoder module employs a Siamese backbone network composed of efficient local-global context aggregation modules to extract multi-scale features, balancing local details and global semantics, and designs a change attention guidance module that takes feature differences as attention weights to dynamically enhance the saliency of defect regions and suppress background interference. The decoder module adopts an improved FPN structure to fuse high-level and low-level features, supplement defect details, and improve the model’s ability to detect small targets and multi-scale defects. Experimental results on the self-built substation multi-phase defect dataset (SMDD) show that the proposed method achieves 81.76% in terms of mAP, which is 3.79% higher than that of Faster R-CNN and outperforms mainstream detection models such as GoldYOLO and YOLOv10. Ablation experiments and visualization analysis demonstrate that the method can effectively focus on defect regions in complex environments, improving the positioning accuracy of multi-scale targets.