Automated fruit-picking equipment has the potential to significantly enhance the efficiency of picking. Accurate detection and localization of fruits are particularly crucial in this regard. However, current methods rely on expensive tools such as depth cameras and LiDAR. This study proposes a low-cost method based on monocular images to achieve target detection and depth estimation. To improve the detection accuracy of targets, especially small targets, an advanced YOLOv8s detection algorithm is introduced. This approach utilizes the BiFormer block, an attention mechanism for dynamic query-aware sparsity, as the backbone feature extractor. It also adds a small-target-detection layer in the Neck and employs EIoU Loss as the loss function. Furthermore, a fused depth estimation method is proposed, which incorporates high-resolution, low-resolution, and local high-frequency depth estimation to obtain depth information with both high-frequency details and low-frequency structure. Finally, the spatial 3D coordinates of the fruit are obtained by fusing the planar coordinates and depth information. The experimental results with citrus as the target result in an improved YOLOv8s network mAP of 88.45% and a recognition accuracy of 94.7%. The recognition of citrus in a natural environment was improved by 2.7% compared to the original model. In the detection range of 30 cm~60 cm, the depth-estimation results (MAE, RSME) are 0.53 and 0.53. In the illumination intensity range of 1000 lx to 5000 lx, the average depth estimation results (MAE, RSME) are 0.49 and 0.64. In the simulated fruit-picking scenario, the success rates of grasping at 30 cm and 45 cm were 80.6% and 85.1%, respectively. The method has the advantage of high-resolution depth estimation without constraints of camera parameters and fruit size that monocular geometric and binocular localization do not have, providing a feasible and low-cost localization method for fruit automation equipment.

Pecans have rich nutritional value and high economic value. Fast and accurate shell and kernel sorting will improve the efficiency of its automated production. Therefore, we propose a FastQAFPN-YOLOv8s target detection network to achieve fast and accurate detection of unseparated materials. The method uses lightweight Pconv operators to build the FasterNextBlock structure, which serve as the backbone feature extractor for the Fasternet feature extraction network. The ECIoU loss function combining EIoU and CIoU speeds up the adjustment of the prediction frame and the network regression. In the Neck part of the network, the QAFPN feature fusion extraction network is proposed to replace the PAN-FPN in YOLOv8s with a Rep-PAN structure based on the QARepNext reparameterization structure for feature fusion extraction to achieve a trade-off between network performance and inference speed. To validate the method, we built a three-axis mobile sorting device and created a dataset of 3,000 images of walnuts after breaking their shells for experiments. The results show that the improved network has a number of 6071008 parameters, a training time of 2.49 h, a model size of 12.3 MB, an mAP of 94.5%, and a frame rate of 52.1 FPS. Compared with the original model, the number of parameters decreases by 45.5%, the training time decreases by 32.7%, the model size decreases by 45.3%, and the frame rate improves by 40.8%. However, some accuracy is lost along with the lightweight, with a 1.2% decrease in mAP. The network reduces the model size by 59.7MB and 23.9MB compared to YOLOv7 and YOLOv6, respectively, and improves the frame rate by 15.67fps and 22.55fps, respectively. the average confidence and mAP are little changed compared to YOLOv7 and improved by 4.2% and 2.4% compared to YOLOv6, respectively. The FastQAFPN-YOLOv8s detection method can effectively reduce the model size while ensuring the recognition accuracy.