Military target detection technology is the basis and key for reconnaissance and command decision-making, as well as the premise of target tracking. Current military target detection algorithms involve many parameters and calculations, prohibiting deployment on the weapon equipment platform with limited hardware resources. Given the above problems, this paper proposes a lightweight military target detection method entitled SMCA-α-YOLOv5. Specifically, first, the Focus module is replaced with the Stem block to improve the feature expression ability of the shallow network. Next, we redesign the backbone network of YOLOv5 by embedding the coordinate attention module based on the MobileNetV3 block, reducing the network parameter cardinality and computations, thus improving the model’s average detection accuracy. Finally, we propose a power parameter loss that combines the optimizations of the EIOU loss and Focal loss, improving further the detection accuracy and convergence speed. According to the experimental findings, when applied to the self-created military target data set, the developed method achieves an average precision of 98.4% and a detection speed of 47.6 Frames Per Second (FPS). Compared with the SSD, Faster-RCNN, YOLOv3, YOLOv4, and YOLOv5 algorithms, the mAP values of the improved algorithm surpass the competitor methods by 8.3%, 9.9%, 2.1%, 1.6%, and 1.9%, respectively. Compared with the YOLOv5 algorithm, the parameter cardinality and computational burden are decreased by 85.7% and 95.6%, respectively, meeting mobile devices’ military target detection requirements.