Recently, farmers of sweet pepper suffer from the increase of its unit production costs due to the rise of labor costs. The rise of unit production costs of sweet pepper, on the other hand, decreases its productivity and causes the lack of its farming expertise, thus resulting in the quality degradation of products. In this regard, it is necessary to introduce an automated robot harvest system into the farming of sweet pepper. In this study, the authors developed an image-based closed-loop control system (a vision servo system) and an automated sweet pepper harvesting robot system and then carried out experiments to verify its efficiency. The working area of the manipulator that detects products through an imaging sensor in the farming environment of sweet pepper, decides whether to harvest it or not, and then informs the location of the product to the control center, which is set up at the distance scope of 350~600 mm from the center of the system and 1000 mm vertically. In order to confirm the performance of the sweet pepper recognition in this study, 269 sweet pepper images were used to extract fruits. Of 269 sweet pepper images, 82.16% were recognized successfully. The harvesting experiment of the system developed in this study was carried out with 100 sweet peppers. The result of experiment with 100 sweet peppers presents the fact that its approach rate to peduncle is about 86.7%, and via four sessions of repetitive harvest experiment it achieves a maximal 70% harvest rate, and its average time of harvest is 51.1 s.

For harvest automation of sweet pepper, image recognition algorithms for differentiating each part of a sweet pepper plant were developed and performances of these algorithms were compared. An imaging system consisting of two cameras and six halogen lamps was built for sweet pepper image acquisition. For image analysis using the normalized difference vegetation index (NDVI), a band-pass filter in the range of 435 to 950 nm with a broad spectrum from visible light to infrared was used. K-means clustering and morphological skeletonization were used to classify sweet pepper parts to which the NDVI was applied. Scale-invariant feature transform (SIFT) and speeded-up robust features (SURFs) were used to figure out local features. Classification performances of a support vector machine (SVM) using the radial basis function kernel and backpropagation (BP) algorithm were compared to classify local SURFs of fruits, nodes, leaves, and suckers. Accuracies of the BP algorithm and the SVM for classifying local features were 95.96 and 63.75%, respectively. When the BP algorithm was used for classification of plant parts, the recognition success rate was 94.44% for fruits, 84.73% for nodes, 69.97% for leaves, and 84.34% for suckers. When CNN was used for classifying plant parts, the recognition success rate was 99.50% for fruits, 87.75% for nodes, 90.50% for leaves, and 87.25% for suckers.