This study aims to develop an instance segmentation model for the joint segmentation of pomelo fruits and stems in complex natural orchard environments, with particular emphasis on slender, small-scale, and easily occluded stem targets. To this end, YOLO-MOD, an improved instance segmentation algorithm based on YOLOv11-seg, is proposed. Specifically, Omni-Dimensional Dynamic Convolution (ODConv) is introduced into the C3k2 module to enhance complex feature representation; a Multi-Scale Dilated Attention (MSDA) module is embedded to improve the multi-scale semantic perception of slender stem regions; and the original upsampling operator is replaced with DySample to strengthen fine-grained boundary recovery. Experimental results show that, compared with the original YOLOv11-seg, YOLO-MOD improves the Box mAP@50 and Mask mAP@50 by 2.9% and 3.9%, respectively. For the Stem class, the Box mAP@50 and Mask mAP@50 increase from 71.9% to 77.8% and from 68.4% to 76.2%, respectively. These results indicate that YOLO-MOD can achieve fine-grained segmentation of pomelo fruits and stems on the dataset used in this study. However, its generalization capability across different orchards, seasons, pomelo varieties, and fruit types still requires further evaluation, and its practical effectiveness in an integrated robotic harvesting system remains to be further validated.