Missing bolts compromise the structural integrity of bolted connections in steel bridges and industrial infrastructure. Manual visual inspection remains labour-intensive, subjective, and hazardous in hard-to-reach locations. This study presents a comparative benchmarking framework for unmanned aerial vehicles (UAVs) missing-bolt patch classification using convolutional neural network (CNN), focusing on balancing accuracy and computational efficiency. A UAV-acquired dataset of bolt-centric image patches was developed to evaluate four systematic experimental schemes: (i) a custom lightweight CNN trained from scratch, (ii) the lightweight CNN integrated with Squeeze-and-Excitation (SE) attention blocks across multiple positions, (iii) nine fine-tuned state-of-the-art (SOTA) pretrained CNN backbones, and (iv) SE-enhanced versions of these pretrained models. All architectures were evaluated under a standardised experimental protocol. Results show that the proposed lightweight CNN achieves classification performance comparable to heavyweight pretrained models while requiring significantly lower computational resources. Integrating SE blocks did not improve classification performance for this localised task and, in several configurations, reduced accuracy and training stability. Pretrained transfer learning models achieved high accuracy overall, but their computational complexity limits direct deployment on edge devices and UAV platforms. Grad-CAM visual explanations confirmed that the lightweight CNN consistently focuses on relevant bolt and hole regions. The findings demonstrate that a task-specific lightweight CNN offers a practical balance between inspection reliability and deployment efficiency for automated structural monitoring.