Over the past decade, there have been increasing attempts to integrate robotic harvesting technology into agricultural scenarios to reduce growing labour costs and increase crop yields. In this paper, we demonstrate a prototype harvesting robot for picking watermelons in greenhouses. For robotic harvesting, we design a dedicated end-effector for grasping fruits and shearing pedicels, which mainly consists of a flexible gripper and a cutting device. The improved YOLOv5s–CBAM is employed to locate the watermelon fruits with 89.8% accuracy on the test dataset, while the K-means method is used to further refine the segmentation of the watermelon point cloud in the region of interest. Then, the ellipsoid is fitted with the segmented fruit point cloud to obtain the lowest point of the ellipsoid as the grasping point. A series of tests conducted in a laboratory simulation scenario proved that the overall harvesting success rate was 93.3% with a positioning error of 8.7 mm when the watermelon was unobstructed. The overall harvesting success rate was 85.0% with a positioning error of 14.6 mm when the watermelon was partially obscured by leaves.

In the process of vegetable plug seedling cultivation, packaging, and transportation, there may be missing, unhealthy or injured seedlings in the tray, which results in a missed planting or a low seedling survival rate after automatic transplanting. In this study, a seedling selection system with the function of seedlings identification, week seedlings elimination, and missing seedlings supplement was developed for an automatic transplanter. A plug seedling identification system based on a machine vision was used to detect vegetable plug seedlings based on the area characteristics of plug seedlings, stem leaves and plug bodies. The identification results were transmitted to a programmable logic controller (PLC), which controlled a nozzle to eliminate the unqualified seedlings from the conveyor belt lattice. When the empty conveyor belt lattice reaches the seedling throwing funnel, the rear conveyor belt lattice with the plug seedling is accelerated to ensure the continuity of seedlings supply. The adaptive fuzzy PID control algorithm was used to control the stepper motor of the conveyor belt to realize accurate seedling conveying and a seedling supplement. Using 30 days pepper plug seedlings as experimental seedlings, a comparative field experiment was carried out to evaluate the performance of the seedling selection system. The results showed that when the seedling selection system was turned on and the seedling extracting frequencies were 60, 80, and 100 plants/min, the success rates of plug seedling identification were 98.84%, 98.38%, and 96.99%, and the robust seedling rates were 98.05%, 97.78%, and 95.83%. The robust seedling rates were increased by 15.64%, 16.07%, and 13.89%, respectively, in contrast to turning off the seedling selection system.

Illumination in the natural environment is uncontrollable, and the field background is complex and changeable which all leads to the poor quality of broccoli seedling images. The colors of weeds and broccoli seedlings are close, especially under weedy conditions. The factors above have a large influence on the stability, velocity and accuracy of broccoli seedling recognition based on traditional 2D image processing technologies. The broccoli seedlings are higher than the soil background and weeds in height due to the growth advantage of transplanted crops. A method of broccoli seedling recognition in natural environments based on Binocular Stereo Vision and a Gaussian Mixture Model is proposed in this paper. Firstly, binocular images of broccoli seedlings were obtained by an integrated, portable and low-cost binocular camera. Then left and right images were rectified, and a disparity map of the rectified images was obtained by the Semi-Global Matching (SGM) algorithm. The original 3D dense point cloud was reconstructed using the disparity map and left camera internal parameters. To reduce the operation time, a non-uniform grid sample method was used for the sparse point cloud. After that, the Gaussian Mixture Model (GMM) cluster was exploited and the broccoli seedling points were recognized from the sparse point cloud. An outlier filtering algorithm based on k-nearest neighbors (KNN) was applied to remove the discrete points along with the recognized broccoli seedling points. Finally, an ideal point cloud of broccoli seedlings can be obtained, and the broccoli seedlings recognized. The experimental results show that the Semi-Global Matching (SGM) algorithm can meet the matching requirements of broccoli images in the natural environment, and the average operation time of SGM is 138 ms. The SGM algorithm is superior to the Sum of Absolute Differences (SAD) algorithm and Sum of Squared Differences (SSD) algorithms. The recognition results of Gaussian Mixture Model (GMM) outperforms K-means and Fuzzy c-means with the average running time of 51 ms. To process a pair of images with the resolution of 640×480, the total running time of the proposed method is 578 ms, and the correct recognition rate is 97.98% of 247 pairs of images. The average value of sensitivity is 85.91%. The average percentage of the theoretical envelope box volume to the measured envelope box volume is 95.66%. The method can provide a low-cost, real-time and high-accuracy solution for crop recognition in natural environment.

The automatic identification of seedling defects is an important technology of an intelligent automatic transplanting machine, which effectively improves the quality of the transplanting machine’s operation. The accurate segmentation of seedling substrate and seedling region is the key to the success of the seedling defect recognition algorithm. This paper proposes the maxIOU algorithm to calculate the image segmentation threshold: The image G channel and excess green color space were selected as the color space for the segmentation of the substrate region and seedling region by analyzing the color histogram. Several images were randomly selected from the dataset to generate a training set and were labeled manually as the ground truth. The training set images were segmented using a threshold of zero to 255, and the intersection over union (IOU) were calculated using the algorithm segmented result and the ground truth. The threshold corresponding to the average IOU maximum was used as the segmentation threshold. After image segmentation, three features (area of the substrate, area of the seedling, and filling ratio of the lower part of the substrate) were obtained by the algorithm, and the image was identified for whether there was an empty conveyor belt, seedling deficiency, multiple seedlings, skew, and damaged substrate. The algorithm was tested on the automatic transplanter test platform. The experiment results were as follows: Firstly, the image segmentation threshold was calculated by the maxIOU method. The color component interval corresponding to the segmented substrate region was [0, 24] in the G channel, and the color component interval corresponding to the segmented seedling region was [21, 255] in the excess green channel. The average IOU of the substrate area was 0.854, and the average IOU of the seedling area was 0.820 in the verification experiment. Secondly, a dataset including 431 normal seedling images and 69 defective seedling images (empty conveyor belt, seedling deficiency, multiple seedlings, skew, and damaged substrate) was identified for defects. The accuracy, precision, and recall were 97.6%, 97.4%, and 99.8%. The processing time was 71.4 ms. The conclusion of the experiment was as follows: the maxIOU algorithm had high accuracy in the segmentation of the substrate and seedling region. The defect identification algorithm had high accuracy for defect identification of cabbage seedlings, and the algorithm had good real-time performance, which can be applied to high speed field transplanters.