Ground cracks resulting from coal mining subsidence can pose significant risks to coal mine safety, production, and the surrounding ecological environment. The use of drone technology coupled with deep learning holds practical value for accurately identifying ground cracks in mining areas. This study introduces an enhanced target detection algorithm based on YOLOv8n for analyzing ground cracks in aerial images captured by unmanned aerial vehicles (UAVs) dealing with small targets and widely distributed in shrubland. The C2f module of the backbone network is improved by incorporating deformable convolution (DCN), enhancing the model’s ability to adapt to detected object shapes, focus on small targets, and reduce irrelevant background information. Subsequently, the global attention mechanism (GAM) is integrated into the neck network to minimize feature information loss during training. Finally, the wise IOU (WIOU) loss function replaces the complete IOU (CIOU) loss function to improve the model’s generalizability. Experimental results demonstrate that the algorithm enhanced in this study exhibits improvements in precision (P), recall (R), and mean average precision (mAP@0.5) of 3.5%, 4.8%, and 5.1%, respectively, compared to the original YOLOv8n model. Furthermore, compared to other prevalent algorithms, the enhanced algorithm notably reduces model parameters, floating point operations per second (FLOPs), and model size while maintaining high detection accuracy. In conclusion, the enhanced YOLOv8n-based algorithm offers practical benefits for detecting cracks in mining subsidence areas.