Smart farming (SF) applications rely on robust and accurate computer vision systems. An important computer vision task in agriculture is semantic segmentation, which aims to classify each pixel of an image and can be used for selective weed removal. State-of-the-art implementations use convolutional neural networks (CNN) that are trained on large image datasets. In agriculture, publicly available RGB image datasets are scarce and often lack detailed ground-truth information. In contrast to agriculture, other research areas feature RGB-D datasets that combine color (RGB) with additional distance (D) information. Such results show that including distance as an additional modality can improve model performance further. Therefore, we introduce WE3DS as the first RGB-D image dataset for multi-class plant species semantic segmentation in crop farming. It contains 2568 RGB-D images (color image and distance map) and corresponding hand-annotated ground-truth masks. Images were taken under natural light conditions using an RGB-D sensor consisting of two RGB cameras in a stereo setup. Further, we provide a benchmark for RGB-D semantic segmentation on the WE3DS dataset and compare it with a solely RGB-based model. Our trained models achieve up to 70.7% mean Intersection over Union (mIoU) for discriminating between soil, seven crop species, and ten weed species. Finally, our work confirms the finding that additional distance information improves segmentation quality.

Modern precision agriculture applications increasingly rely on stable computer vision outputs. An important computer vision task is to discriminate between soil and plant pixels, which is called plant segmentation. For this task, supervised learning techniques, such as decision tree classifiers (DTC), support vector machines (SVM), or artificial neural networks (ANN) are increasing in popularity. The selection of training data is of utmost importance in these approaches as it influences the quality of the resulting models. We investigated the influence of three modeling parameters, namely proportion of plant pixels (plant cover), criteria on what pixel to choose (pixel selection), and number/type of features (input features) on the segmentation quality using DTCs. Our findings show that plant cover and, to a minor degree, input features have a significant impact on segmentation quality. We can state that the overperformance of multi-feature input decision tree classifiers over threshold-based color index methods can be explained to a high degree by the more balanced training data. Single-feature input decision tree classifiers can compete with state-of-the-art models when the same training data are provided. This study is the first step in a systematic analysis of influence parameters of such plant segmentation models.