Defects on large-scale structural surfaces can compromise integrity and pose safety hazards, highlighting the need for efficient automated inspection. UAVs provide a flexible and effective platform for such inspections, yet traditional vision-based methods often require high computational resources and show limited sensitivity to small defects, restricting practical UAV deployment. To address these challenges, we propose ESS-DETR, a lightweight and high-precision detection model designed for UAV-based surface inspection, built upon core modules: EMO-inspired lightweight backbone that integrates convolution and efficient attention mechanisms to reduce parameters; Scale-Decoupled Loss that adaptively balances targets of various sizes to enhance accuracy and robustness for small and irregular defect patterns frequently encountered in UAV imagery; and SPPELAN multi-scale fusion module that improves feature discrimination under complex reflections, shadows, and lighting variations typical of aerial inspection environments. Experimental results demonstrate that ESS-DETR reduces computational complexity from 103.4 to 60.5 GFLOPs and achieves a Precision of 0.837, Recall of 0.738, and mAP of 79, outperforming Faster R-CNN, RT-DETR, and YOLOv11, particularly for small-scale defects, confirming that ESS-DETR effectively balances accuracy, efficiency, and onboard deployability, providing a practical solution for intelligent UAV-based surface inspection.