Robotic harvesting shows a promising aspect in future development of agricultural industry. However, there are many challenges which are still presented in the development of a fully functional robotic harvesting system. Vision is one of the most important keys among these challenges. Traditional vision methods always suffer from defects in accuracy, robustness, and efficiency in real implementation environments. In this work, a fully deep learning-based vision method for autonomous apple harvesting is developed and evaluated. The developed method includes a light-weight one-stage detection and segmentation network for fruit recognition and a PointNet to process the point clouds and estimate a proper approach pose for each fruit before grasping. Fruit recognition network takes raw inputs from RGB-D camera and performs fruit detection and instance segmentation on RGB images. The PointNet grasping network combines depth information and results from the fruit recognition as input and outputs the approach pose of each fruit for robotic arm execution. The developed vision method is evaluated on RGB-D image data which are collected from both laboratory and orchard environments. Robotic harvesting experiments in both indoor and outdoor conditions are also included to validate the performance of the developed harvesting system. Experimental results show that the developed vision method can perform highly efficient and accurate to guide robotic harvesting. Overall, the developed robotic harvesting system achieves 0.8 on harvesting success rate and cycle time is 6.5 s.

Soft actuators, as an important part of soft robotics, have attracted significant attention due to their inherent compliance, flexibility and safety. However, low capacity in force and load limits their applications. Prestored elastic energy can improve the capacity in output force and load of soft actuators. This work introduces a soft pneumatic bistable reinforced actuator inspired by the Venus Flytrap's bistable mechanism that allows for the storage of elastic energy. The proposed actuator consists of two separated 3D-printed actuation chambers that are attached to a central prestressed steel shell to achieve bistability. The pressure triggering the pneumatic bistable reinforced actuator from one stable state to the other is derived and validated by experiments. Further experimental comparisons between the proposed actuator and a dual chamber actuator show that the proposed design significantly improves the block tip force, load capacity and stiffness. The pneumatic bistable reinforced actuator also demonstrates superior performance in the actuation speed and bending angle under the same input pressure. Finally, a two-finger gripper is developed using the proposed actuator, which is demonstrated to grasp and hold various objects.

The peg-in-hole assembly is an essential robotic task that is widely involved in the industry and daily life. Hole search is a crucial step in peg-in-hole since it aligns the peg and hole where the positional uncertainty is presented. This study introduces a hole search strategy by computing an exploration distribution based on uncertainty analysis. Our method firstly utilizes vision to estimate the hole’s pose. It then performs an uncertainty analysis based on vision estimation and systematic configures to compute the exploration distribution of the hole search. The computed exploration distribution calculates a search trajectory through the distribution and finds the hole. The effectiveness of the search strategy is experimentally verified using a robot manipulator in the peg-in-hole of the optical transceiver, which is a challenging task that has not yet been addressed. The results confirm that the proposed search strategy can achieve a 90 $\%$ success rate on the hole search. Through the rational design of the recovery strategy, our method can secure a 100 $\%$ success rate on the robotic peg-in-hole of apparatus with a small clearance (0.1 mm).