Crop monitoring is highly important in terms of the efficient and stable performance of tasks such as planting, spraying, and harvesting, and for this reason, several studies are being conducted to develop and improve crop monitoring robots. In addition, the applications of deep learning algorithms are increasing in the development of agricultural robots since deep learning algorithms that use convolutional neural networks have been proven to show outstanding performance in image classification, segmentation, and object detection. However, most of these applications are focused on the development of harvesting robots, and thus, there are only a few studies that improve and develop monitoring robots through the use of deep learning. For this reason, we aimed to develop a real-time robot monitoring system for the generative growth of tomatoes. The presented method detects tomato fruits grown in hydroponic greenhouses using the Faster R-CNN (region-based convolutional neural network). In addition, we sought to select a color model that was robust to external light, and we used hue values to develop an image-based maturity standard for tomato fruits; furthermore, the developed maturity standard was verified through comparison with expert classification. Finally, the number of tomatoes was counted using a centroid-based tracking algorithm. We trained the detection model using an open dataset and tested the whole system in real-time in a hydroponic greenhouse. A total of 53 tomato fruits were used to verify the developed system, and the developed system achieved 88.6% detection accuracy when completely obscured fruits not captured by the camera were included. When excluding obscured fruits, the system’s accuracy was 90.2%. For the maturity classification, we conducted qualitative evaluations with the assistance of experts.

Protected cultivation systems such as greenhouses are becoming increasingly popular globally and have been adopted because of unpredictable climatic conditions and their ability to easily control micro- and macroenvironments. However, limitations such as hazardous work environments and shortages in labor are major concerns for agricultural production using these structures. This has led to the development and adoption of robotic systems. For the efficient use of robots in protected cultivation systems, we formulate the work efficiency problem and model a three-dimensional standard strawberry greenhouse to analyze the effectiveness of a strawberry-harvesting robot compared to different levels of human workforce (experienced, average, and beginner). Simulations are conducted using Quest software to compare the efficiency of different scenarios of robotics to humans. Different methods of improvement from battery capacity and charge rate to harvesting speed are investigated and optimal conditions are recommended. The average hourly production of the robot is about five times lower than that of skilled workers. However, robots are more productive due to their ability to work around the clock. Comparative analyses show that a reduction in harvesting time per strawberry from 3 to 1 s would result in an increase in daily production from 347.93 to 1021.30 kg. This would lead to a five-fold increase in comparison to present daily production. A 10% improvement in battery charge time would result in the battery capacity gaining two extra hours from the current 10 h and would cut the current 2 h needed for charge to 1 h. This paper proposes an operation process and suggestions for changes needed for improving the work efficiency of robots in a greenhouse. This could be extended to other crops and greenhouses.

In this study, we aimed to develop a prediction model of the solid solutions concentration (SSC) and moisture content (MC) in oriental melon with snapshot-type hyperspectral imagery (Visible (VIS): 460–600 nm, 16 bands; Red-Near infrared (Red-NIR): 600–860 nm, 15 bands) using a machine learning model. The oriental melons were cultivated in a hydroponic greenhouse, Republic of Korea, and a total of 91 oriental melons that were harvested from March to April of 2022 were used as samples. The SSC and MC of the oriental melons were measured using destructive methods after taking hyperspectral imagery of the oriental melons. The reflectance spectrum obtained from the hyperspectral imagery was processed by the standard normal variate (SNV) method. Variable importance in projection (VIP) scores were used to select the bands related to SSC and MC. As a result, ten (609, 736, 561, 849, 818, 489, 754, 526, 683, and 597 nm) and six (609, 736, 561, 818, 849, and 489 nm) bands were selected for the SSC and MC, respectively. Four machine learning models, support vector regression (SVR), ridge regression (RR), K-nearest neighbors regression (K-NNR), and random forest regression (RFR), were used to develop models to predict SSC and MC, and their performances were compared. The SVR showed the best performance for predicting both the SSC and MC of the oriental melons. The SVR model achieved a relatively high accuracy with R² values of 0.86 and 0.74 and RMSE values of 1.06 and 1.05 for SSC and MC, respectively. However, it will be necessary to carry out more experiments under various conditions, such as differing maturities of fruits and varying light sources and environments, to achieve more comprehensive predictions and apply them to monitoring robots in the future. Nevertheless, it is considered that the snapshot-type hyperspectral imagery aided by SVR would be a useful tool to predict the SSC and MC of oriental melon. In addition, if the maturity classification model for the oriental melon can be applied to fields, it could lead to less labor and result in high-quality oriental melon production.

In this study, we propose a tomato maturity estimation approach based on a deep neural network. Tomato images were obtained using an RGB camera installed on a monitoring robot and samples were cropped to generate a dataset with which to train the classification model. The classification model is trained using cross-entropy loss and mean–variance loss, which can implicitly provide label distribution knowledge. For continuous maturity estimation in the test stage, the output probability distribution of four maturity classes is calculated as an expected (normalized) value. Our results demonstrate that the F1 score was approximately 0.91 on average, with a range of 0.85–0.97. Furthermore, comparison with the hue value—which is correlated with tomato growth—showed no significant differences between estimated maturity and hue values, except in the pink stage. From the overall results, we found that our approach can not only classify the discrete maturation stages of tomatoes but can also continuously estimate their maturity. Furthermore, it is expected that with higher accuracy data labeling, more precise classification and higher accuracy may be achieved.

It is necessary to convert to automation in a tomato hydroponic greenhouse because of the aging of farmers, the reduction in agricultural workers as a proportion of the population, COVID-19, and so on. In particular, agricultural robots are attractive as one of the ways for automation conversion in a hydroponic greenhouse. However, to develop agricultural robots, crop monitoring techniques will be necessary. In this study, therefore, we aimed to develop a maturity classification model for tomatoes using both support vector classifier (SVC) and snapshot-type hyperspectral imaging (VIS: 460–600 nm (16 bands) and Red-NIR: 600–860 nm (15 bands)). The spectral data, a total of 258 tomatoes harvested in January and February 2022, was obtained from the tomatoes’ surfaces. Spectral data that has a relationship with the maturity stages of tomatoes was selected by correlation analysis. In addition, the four different spectral data were prepared, such as VIS data (16 bands), Red-NIR data (15 bands), combination data of VIS and Red-NIR (31 bands), and selected spectral data (6 bands). These data were trained by SVC, respectively, and we evaluated the performance of trained classification models. As a result, the SVC based on VIS data achieved a classification accuracy of 79% and an F1-score of 88% to classify the tomato maturity into six stages (Green, Breaker, Turning, Pink, Light-red, and Red). In addition, the developed model was tested in a hydroponic greenhouse and was able to classify the maturity stages with a classification accuracy of 75% and an F1-score of 86%.

Honeybees pollinate over 75% of the total food resources produced annually, and they produce valuable hive products, such as bee pollen, propolis, and royal jelly. However, species such as the Asian hornet (Vespa velutina) feed on more than 85% of honeybees, causing a decline in their population and considerable damage to beekeepers in Korea. To prevent damage to honeybees, a portable real-time monitoring system was developed that detects V. velutina individuals and notifies users of their presence. This system was designed with a focus on portability and ease of installation, as V. velutina can be found in various areas of apiary sites. To detect V. velutina, an improved convolutional neural network YOLOv5s was trained on 1960 high-resolution ($3840\times 2160$) image data. At the confidence threshold of ≥0.600 and intersection over the union of ≥0.500, the performance of the system in terms of detection accuracy, precision, recall, F1 score, and mean average precision was high. A distance-based performance comparison showed that the system was able to detect V. velutina individuals while monitoring three beehives. During a field test of monitoring three beehives, the system could detect 83.3% of V. velutina during their hunting activities and send alarms to registered mobile application users.