Crop monitoring is highly important in terms of the efficient and stable performance of tasks such as planting, spraying, and harvesting, and for this reason, several studies are being conducted to develop and improve crop monitoring robots. In addition, the applications of deep learning algorithms are increasing in the development of agricultural robots since deep learning algorithms that use convolutional neural networks have been proven to show outstanding performance in image classification, segmentation, and object detection. However, most of these applications are focused on the development of harvesting robots, and thus, there are only a few studies that improve and develop monitoring robots through the use of deep learning. For this reason, we aimed to develop a real-time robot monitoring system for the generative growth of tomatoes. The presented method detects tomato fruits grown in hydroponic greenhouses using the Faster R-CNN (region-based convolutional neural network). In addition, we sought to select a color model that was robust to external light, and we used hue values to develop an image-based maturity standard for tomato fruits; furthermore, the developed maturity standard was verified through comparison with expert classification. Finally, the number of tomatoes was counted using a centroid-based tracking algorithm. We trained the detection model using an open dataset and tested the whole system in real-time in a hydroponic greenhouse. A total of 53 tomato fruits were used to verify the developed system, and the developed system achieved 88.6% detection accuracy when completely obscured fruits not captured by the camera were included. When excluding obscured fruits, the system’s accuracy was 90.2%. For the maturity classification, we conducted qualitative evaluations with the assistance of experts.

Recently, many methods to interpret and visualize deep neural network predictions have been proposed, and significant progress has been made. However, a more class-discriminative and visually pleasing explanation is required. Thus, this paper proposes a region-based approach that estimates feature importance in terms of appropriately segmented regions. By fusing the saliency maps generated from multi-scale segmentations, a more class-discriminative and visually pleasing map is obtained. This paper incorporates this regional multi-scale concept into a prediction difference method that is model-agnostic. An input image is segmented in several scales using the superpixel method, and exclusion of a region is simulated by sampling a normal distribution constructed via the boundary prior. The experimental results demonstrate that the regional multi-scale method produces much more class-discriminative and visually pleasing saliency maps.

In the realm of agricultural automation, the efficient management of tasks like yield estimation, harvesting, and monitoring is crucial. While fruits are typically detected using bounding boxes, pixel-level segmentation is essential for extracting detailed information such as color, maturity, and shape. Furthermore, while previous studies have typically focused on controlled environments and scenes, achieving robust performance in real orchard conditions is also imperative. To prioritize these aspects, we propose the following two considerations: first, a novel peach image dataset designed for rough orchard environments, focusing on pixel-level segmentation for detailed insights; and second, utilizing a transformer-based instance segmentation model, specifically the Swin Transformer as a backbone of Mask R-CNN. We achieve superior results compared to CNN-based models, reaching 60.2 AP on the proposed peach image dataset. The proposed transformer-based approach specially excels in detecting small or obscured peaches, making it highly suitable for practical field applications. The proposed model achieved 40.4 AP for small objects, nearly doubling that of CNN-based models. This advancement significantly enhances automated agricultural systems, especially in yield estimation, harvesting, and crop monitoring.

Fruit trees in orchards are typically placed at equal distances in rows; therefore, their branches are intertwined. The precise segmentation of a target tree in this situation is very important for many agricultural tasks, such as yield estimation, phenotyping, spraying, and pruning. However, our survey on tree segmentation revealed that no study has explicitly addressed this intertwining situation. This paper presents a novel dataset in which a precise tree region is labeled carefully by a human annotator by delineating the branches and trunk of a target apple tree. Because traditional rule-based image segmentation methods neglect semantic considerations, we employed cutting-edge deep learning models. Five recently pre-trained deep learning models for segmentation were modified to suit tree segmentation and were fine-tuned using our dataset. The experimental results show that YOLOv8 produces the best average precision (AP), 93.7 box AP@0.5:0.95 and 84.2 mask AP@0.5:0.95. We believe that our model can be successfully applied to various agricultural tasks.

Background and objective: Conventional anthropometric studies using Kinect depth sensors have concen-trated on estimating the distances between two points such as height. This paper deals with a novel waist measurement method using SVM regression, further widening spectrum of Kinect's potential ap-plications. Waist circumference is a key index for the diagnosis of abdominal obesity, which has been linked to metabolic syndromes and other related diseases. Yet, the existing measuring method, tape mea-sure, requires a trained personnel and is therefore costly and time-consuming. Methods: A dataset was constructed by recording both 30 frames of Kinect depth image and careful tape measurement of 19 volunteers by a clinical investigator. This paper proposes a new SVM regressor-based approach for estimating waist circumference. A waist curve vector is extracted from a raw depth image using joint information provided by Kinect SDK. To avoid overfitting, a data augmentation tech-nique is devised. The 30 frontal vectors and 30 backside vectors, each sampled for 1 s per person, are combined to form 900 waist curve vectors and a total of 17,100 samples were collected from 19 indi-viduals. On an individual basis, we performed leave-one-out validation using the SVM regressor with the tape measurement-gold standard of waist circumference measurement-values labeled as ground-truth. On an individual basis, we performed leave-one-out validation using the SVM regressor with the tape measurement-gold standard of waist circumference measurement-values labeled as ground-truth. Results: The mean error of the SVM regressor was 4.62 cm, which was smaller than that of the geometric estimation method. Potential uses are discussed. Conclusions: A possible method for measuring waist circumference using a depth sensor is demonstrated through experimentation. Methods for improving accuracy in the future are presented. Combined with other potential applications of Kinect in healthcare setting, the proposed method will pave the way for patient-centric approach of delivering care without laying burdens on patients. (C) 2020 The Authors. Published by Elsevier B.V.

Recently, computer vision methods have been widely applied to agricultural tasks, such as robotic harvesting. In particular, fruit harvesting robots often rely on object detection or segmentation to identify and localize target fruits. During the model selection process for object detection, the average precision (AP) score typically provides the de facto standard. However, AP is not intuitive for determining which model is most efficient for robotic harvesting. It is based on the intersection-over-union (IoU) of bounding boxes, which reflects only regional overlap. IoU alone cannot reliably predict the success of robotic gripping, as identical IoU scores may yield different results depending on the overlapping shape of the boxes. In this paper, we propose a novel evaluation metric for robotic harvesting. To assess gripping success, our metric uses the center coordinates of bounding boxes and a margin hyperparameter that accounts for the gripper’s specifications. We conducted evaluation about popular object detection models on peach and apple datasets. The experimental results showed that the proposed gripping success metric is much more intuitive and helpful in interpreting the performance data.