Rice yield is closely related to the number and proportional area of rice panicles. Currently, rice panicle information is acquired with manual observation, which is inefficient and subjective. To solve this problem, we propose an improved Mask R-CNN combined with Otsu preprocessing for rice detection and segmentation. This method first constructs a rice dataset for rice images in a large field environment, expands the dataset using data augmentation, and then uses LabelMe to label the rice panicles. The optimized Mask R-CNN is used as a rice detection and segmentation model. Actual rice panicle images are preprocessed by the Otsu algorithm and input into the model, which yields accurate rice panicle detection and segmentation results using the structural similarity and perceptual hash value as the measurement criteria. The results show that the proposed method has the highest detection and segmentation accuracy for rice panicles among the compared algorithms. When further calculating the number and relative proportional area of the rice panicles, the average error of the number of rice panicles is 16.73% with a minimum error of 5.39%, and the error of the relative proportional of rice panicles does not exceed 5%, with a minimum error of 1.97% and an average error of 3.90%. The improved Mask R-CNN combined with Otsu preprocessing for rice panicle detection and segmentation proposed in this paper can operate well in a large field environment, making it highly suitable for rice growth monitoring and yield estimation.