Accurate detection of tea shoots in natural environments is crucial for facilitating intelligent tea picking, field management, and automated harvesting. However, the detection performance of existing methods in complex scenes remains limited due to factors such as the small size, high density, severe overlap, and the similarity in color between tea shoots and the background. Consequently, this paper proposes an improved target detection algorithm, RFA-YOLOv8, based on YOLOv8, which aims to enhance the detection accuracy and robustness of tea shoots in natural environments. First, a self-constructed dataset containing images of tea shoots under various lighting conditions is created for model training and evaluation. Second, the multi-scale feature extraction capability of the model is enhanced by introducing RFCAConv along with the optimized SPPFCSPC module, while the spatial perception ability is improved by integrating the RFAConv module. Finally, the EIoU loss function is employed instead of CIoU to optimize the accuracy of the bounding box positioning. The experimental results demonstrate that the improved model achieves 84.1% and 58.7% in mAP@0.5 and mAP@0.5:0.95, respectively, which represent increases of 3.6% and 5.5% over the original YOLOv8. Robustness is evaluated under strong, moderate, and dim lighting conditions, yielding improvements of 6.3% and 7.1%. In dim lighting, mAP@0.5 and mAP@0.5:0.95 improve by 6.3% and 7.1%, respectively. The findings of this research provide an effective solution for the high-precision detection of tea shoots in complex lighting environments and offer theoretical and technical support for the development of smart tea gardens and automated picking.

With the continuous development of artificial intelligence technology, the transformation of traditional agriculture into intelligent agriculture is quickly accelerating. However, due to the diverse growth postures of tender shoots and complex growth environments in tea plants, traditional tea picking machines are unable to precisely select the tender shoots, and the picking of high-end and premium tea still relies on manual labor, resulting in low efficiency and high costs. To address these issues, an instance segmentation algorithm named YOLOv8-TEA is proposed. Firstly, this algorithm is based on the single-stage instance segmentation algorithm YOLOv8-seg, replacing some C2f modules in the original feature extraction network with MVB, combining the advantages of convolutional neural networks (CNN) and Transformers, and adding a C2PSA module following spatial pyramid pooling (SPPF) to integrate convolution and attention mechanisms. Secondly, a learnable dynamic upsampling method is used to replace the traditional upsampling, and the CoTAttention module is added, along with the fusion of dilated convolutions in the segmentation head to enhance the learning ability of the feature fusion network. Finally, through ablation experiments and comparative experiments, the improved algorithm significantly improves the segmentation accuracy while effectively reducing the model parameters, with mAP (Box) and mAP (Mask) reaching 86.9% and 86.8%, respectively, and GFLOPs reduced to 52.7.