The task of gripping has been identified as the rate-limiting step in the development of tree-fruit harvesting systems. There is, however, no set of universally adopted ‘specifications’ with standardized measurement procedures for the characterization of gripper performance in the harvest of soft tree fruit. A set of metrics were defined for evaluation of the performance of end effectors used in soft tree-fruit harvesting based on (i) laboratory-based trials using metrics termed ‘picking area’, which was the cross-sectional area in a plane normal to the direction of approach of the gripper to the fruit in which a fruit was successfully harvested by the gripper; ‘picking volume’, which was the volume of space in which fruit was successfully harvested by the gripper; and ‘grasp force’, which was the peak force involved in removing a fruit from the grasp of a gripper; (ii) orchard-based trials using metrics termed ‘detachment success’ and ‘harvest success’, i.e., the % of harvest attempts of fruit on tree (of a given canopy architecture) that resulted in stalk breakage and return of fruit to a receiving area, respectively; and (iii) postharvest damage in terms of a score based on the percentage of fruit and severity of the damage. Evaluations were made of external (skin) damage visible 1 h after gripping and of internal (flesh) damage after ripening of the fruit. The use of the metrics was illustrated in an empirical evaluation of nine gripper designs in the harvest of mango fruit in the context of fruit weight and orientation to the gripper. A design using six flexible fingers achieved a picking area of ~150 cm² and a picking volume of 467 cm³ in laboratory trials involving a 636 g phantom fruit as well as detachment and harvest efficiency rates of 74 and 65%, respectively, in orchard trials with no postharvest damage associated with the harvest of unripe fruit. Additional metrics are also proposed. Use of these metrics in future studies of fruit harvesting is recommended for literature–performance comparisons.

Recent developments in affordable depth imaging hardware and the use of 2D Convolutional Neural Networks (CNN) in object detection and segmentation have accelerated the adoption of machine vision in a range of applications, with mainstream models often out-performing previous application-specific architectures. The need for the release of training and test datasets with any work reporting model development is emphasized to enable the re-evaluation of published work. An additional reporting need is the documentation of the performance of the re-training of a given model, quantifying the impact of stochastic processes in training. Three mango orchard applications were considered: the (i) fruit count, (ii) fruit size and (iii) branch avoidance in automated harvesting. All training and test datasets used in this work are available publicly. The mAP ‘coefficient of variation’ (Standard Deviation, SD, divided by mean of predictions using models of repeated trainings × 100) was approximately 0.2% for the fruit detection model and 1 and 2% for the fruit and branch segmentation models, respectively. A YOLOv8m model achieved a mAP50 of 99.3%, outperforming the previous benchmark, the purpose-designed ‘MangoYOLO’, for the application of the real-time detection of mango fruit on images of tree canopies using an edge computing device as a viable use case. YOLOv8 and v9 models outperformed the benchmark MaskR-CNN model in terms of their accuracy and inference time, achieving up to a 98.8% mAP50 on fruit predictions and 66.2% on branches in a leafy canopy. For fruit sizing, the accuracy of YOLOv8m-seg was like that achieved using Mask R-CNN, but the inference time was much shorter, again an enabler for the field adoption of this technology. A branch avoidance algorithm was proposed, where the implementation of this algorithm in real-time on an edge computing device was enabled by the short inference time of a YOLOv8-seg model for branches and fruit. This capability contributes to the development of automated fruit harvesting.

Experimental trials on the performance of end-effectors for the automated harvest of soft fruit are constrained by seasonal limitations on fruit availability and fruit perishability, necessitating the use of different sets of fruit across time. Consequently, the use of fruit and stalk phantoms, rather than real fruit, is an attractive proposition. In this paper, a process for the cost-effective production of stable fruit phantoms using silicone (polydimethylsiloxane, PDMS) and starch was presented. A preliminary consideration was also presented for the creation of a phantom fruit stalk, involving a wooden dowel or a magnetic latching. Mango fruit phantoms were benchmarked to mango fruit in terms of density, firmness, brittleness, etc.