At present, the primary technical deterrent to the use of strawberry harvesting robots is the low harvest rate, and there is a need to improve the accuracy and real-time performance of the localization algorithms to detect the picking point on the strawberry stem. The pose estimation of the fruit target (the direction of the fruit axis) can improve the accuracy of the localization algorithm. This study proposes a novel harvesting robot for the ridge-planted strawberries as well as a fruit pose estimator called rotated YOLO (R-YOLO), which significantly improves the localization precision of the picking points. First, the lightweight network Mobilenet-V1 was used to replace the convolution neural network as the backbone network for feature extraction. The simplified network structure substantially increased the operating speed. Second, the rotation angle parameter $\alpha $ was used to label the training set and set the anchors; the rotation of the bounding boxes of the target fruits was predicted using logistic regression with the rotated anchors. The test results of a set of 100 strawberry images showed that the proposed model’s average recognition rate to be 94.43% and the recall rate to be 93.46%. Eighteen frames per second (FPS) were processed on the embedded controller of the robot, demonstrating good real-time performance. Compared with several other target detection methods used for the fruit harvesting robots, the proposed model exhibited better performance in terms of real-time detection and localization accuracy of the picking points. Field test results showed that the harvesting success rate reached 84.35% in modified situations. The results of this study provide technical support for improving the target detection of the embedded controller of harvesting robots.
Due to the complex unstructured environmental factors in ridge-planting strawberry cultivation, automated harvesting remains a significant challenge. This paper presents an oriented-ridge double-arm cooperative harvesting robot designed for this cultivation. The robot is equipped with a novel non-destructive harvesting end-effector and two self-developed specialized manipulators, integrated with the strawberry picking point visual perception system based on the lightweight Mask R-CNN and a CAN bus-based machine control system. The greenhouse harvesting experiments show that the robot achieved an average harvesting success rate of 49.30% in natural environments after flower and fruit thinning, while only a 30.23% success rate was achieved in untrimmed natural environments. This indicates that the agronomic practice of flower and fruit thinning can significantly simplify the automated harvesting environment and improve harvesting performance. Automated harvesting efficiency test results show that the single-arm average harvesting speed is 7 s per fruit, while double-arm cooperative harvesting can achieve 4 s per fruit. Future expansion by increasing the number of robotic arms could significantly improve harvesting efficiency. However, the study conducted for this paper was poor for those strawberries whose body or stem was severely blocked, which should be further improved upon in follow-up studies.
26 June 2024 in Journal of the Science of Food and Agriculture