Intelligent robots for fruit harvesting have been actively developed over the past decades to bridge the increasing gap between feeding a rapidly growing population and limited labour resources. Despite significant advancements in this field, widespread use of harvesting robots in orchards is yet to be seen. To identify the challenges and formulate future research and development directions, this work reviews the state-of-the-art of intelligent fruit harvesting robots by comparing their system architectures, visual perception approaches, fruit detachment methods and system performances. The potential reasons behind the inadequate performance of existing harvesting robots are analysed and a novel map of challenges and potential research directions is created, considering both environmental factors and user requirements.

Robotic harvesting shows a promising aspect in future development of agricultural industry. However, there are many challenges which are still presented in the development of a fully functional robotic harvesting system. Vision is one of the most important keys among these challenges. Traditional vision methods always suffer from defects in accuracy, robustness, and efficiency in real implementation environments. In this work, a fully deep learning-based vision method for autonomous apple harvesting is developed and evaluated. The developed method includes a light-weight one-stage detection and segmentation network for fruit recognition and a PointNet to process the point clouds and estimate a proper approach pose for each fruit before grasping. Fruit recognition network takes raw inputs from RGB-D camera and performs fruit detection and instance segmentation on RGB images. The PointNet grasping network combines depth information and results from the fruit recognition as input and outputs the approach pose of each fruit for robotic arm execution. The developed vision method is evaluated on RGB-D image data which are collected from both laboratory and orchard environments. Robotic harvesting experiments in both indoor and outdoor conditions are also included to validate the performance of the developed harvesting system. Experimental results show that the developed vision method can perform highly efficient and accurate to guide robotic harvesting. Overall, the developed robotic harvesting system achieves 0.8 on harvesting success rate and cycle time is 6.5 s.

Autonomous harvesting shows a promising prospect in the future development of the agriculture industry, while the vision system is one of the most challenging components in the autonomous harvesting technologies. This work proposes a multi-function network to perform the real-time detection and semantic segmentation of apples and branches in orchard environments by using the visual sensor. The developed detection and segmentation network utilises the atrous spatial pyramid pooling and the gate feature pyramid network to enhance feature extraction ability of the network. To improve the real-time computation performance of the network model, a lightweight backbone network based on the residual network architecture is developed. From the experimental results, the detection and segmentation network with ResNet-101 backbone outperformed on the detection and segmentation tasks, achieving an $F_1$ score of 0.832 on the detection of apples and 87.6% and 77.2% on the semantic segmentation of apples and branches, respectively. The network model with lightweight backbone showed the best computation efficiency in the results. It achieved an $F_1$ score of 0.827 on the detection of apples and 86.5% and 75.7% on the segmentation of apples and branches, respectively. The weights size and computation time of the network model with lightweight backbone were 12.8 M and 32 ms, respectively. The experimental results show that the detection and segmentation network can effectively perform the real-time detection and segmentation of apples and branches in orchards.

Soft actuators, as an important part of soft robotics, have attracted significant attention due to their inherent compliance, flexibility and safety. However, low capacity in force and load limits their applications. Prestored elastic energy can improve the capacity in output force and load of soft actuators. This work introduces a soft pneumatic bistable reinforced actuator inspired by the Venus Flytrap's bistable mechanism that allows for the storage of elastic energy. The proposed actuator consists of two separated 3D-printed actuation chambers that are attached to a central prestressed steel shell to achieve bistability. The pressure triggering the pneumatic bistable reinforced actuator from one stable state to the other is derived and validated by experiments. Further experimental comparisons between the proposed actuator and a dual chamber actuator show that the proposed design significantly improves the block tip force, load capacity and stiffness. The pneumatic bistable reinforced actuator also demonstrates superior performance in the actuation speed and bending angle under the same input pressure. Finally, a two-finger gripper is developed using the proposed actuator, which is demonstrated to grasp and hold various objects.