In this work, an attention-mechanism-enhanced method based on a single-stage object detection model was proposed and implemented for the problem of rice pest detection. A multi-scale feature fusion network was first constructed to improve the model’s predictive accuracy when dealing with pests of different scales. Attention mechanisms were then introduced to enable the model to focus more on the pest areas in the images, significantly enhancing the model’s performance. Additionally, a small knowledge distillation network was designed for edge computing scenarios, achieving a high inference speed while maintaining a high accuracy. Experimental verification on the IDADP dataset shows that the model outperforms current state-of-the-art object detection models in terms of precision, recall, accuracy, mAP, and FPS. Specifically, a mAP of 87.5% and an FPS value of 56 were achieved, significantly outperforming other comparative models. These results sufficiently demonstrate the effectiveness and superiority of the proposed method.