Aiming at difficult image acquisition and low recognition accuracy of two rice canopy pests, rice stem borer and rice leaf roller, we constructed a GA-Mask R-CNN (Generative Adversarial Based Mask Region Convolutional Neural Network) intelligent recognition model for rice stem borer and rice leaf roller, and we combined it with field monitoring equipment for them. Firstly, based on the biological habits of rice canopy pests, a variety of rice pest collection methods were used to obtain the images of rice stem borer and rice leaf roller pests. Based on different segmentation algorithms, the rice pest images were segmented to extract single pest samples. Secondly, the bug generator based on a generative adversarial network strategy improves the sensitivity of the classification network to the bug information, generates the pest information images in the real environment, and obtains the sample dataset for deep learning through multi-way augmentation. Then, through adding channel attention ECA module in Mask R-CNN and improving the connection of residual blocks in the backbone network ResNet101, the recognition accuracy of the model is improved. Finally, the GA-Mask R-CNN model was tested on a multi-source dataset with an average precision (AP) of 92.71%, recall (R) of 89.28% and a balanced score F1 of 90.96%. The average precision, recall, and balanced score F1 are improved by 7.07, 7.65, and 8.83%, respectively, compared to the original Mask R-CNN. The results show that the GA-Mask R-CNN model performance indexes are all better than the Mask R-CNN, the Faster R-CNN, the SSD, the YOLOv5, and other network models, which can provide technical support for remote intelligent monitoring of rice pests.