The production of food, feed, fiber, and fuel is a key task of agriculture, which has to cope with many challenges in the upcoming decades, e.g., a higher demand, climate change, lack of workers, and the availability of arable land. Vision systems can support making better and more sustainable field management decisions, but also support the breeding of new crop varieties by allowing temporally dense and reproducible measurements. Recently, agricultural robotics got an increasing interest in the vision and robotics communities since it is a promising avenue for coping with the aforementioned lack of workers and enabling more sustainable production. While large datasets and benchmarks in other domains are readily available and enable significant progress, agricultural datasets and benchmarks are comparably rare. We present an annotated dataset and benchmarks for the semantic interpretation of real agricultural fields. Our dataset recorded with a UAV provides high-quality, pixel-wise annotations of crops and weeds, but also crop leaf instances at the same time. Furthermore, we provide benchmarks for various tasks on a hidden test set comprised of different fields: known fields covered by the training data and a completely unseen field.

Plant diseases can impact crop yield. Thus, the detection of plant diseases using sensors that can be mounted on aerial vehicles is in the interest of farmers to support decision-making in integrated pest management and to breeders for selecting tolerant or resistant genotypes. This paper investigated the detection of Cercospora leaf spot (CLS), caused by Cercospora beticola in sugar beet using RGB imagery. We proposed an approach to tackle the CLS detection problem using fully convolutional neural networks, which operate directly on RGB images captured by a UAV. This efficient approach does not require complex multi- or hyper-spectral sensors, but provides reliable results and high sensitivity. We provided a detection pipeline for pixel-wise semantic segmentation of CLS symptoms, healthy vegetation, and background so that our approach can automatically quantify the grade of infestation. We thoroughly evaluated our system using multiple UAV datasets recorded from different sugar beet trial fields. The dataset consisted of a training and a test dataset and originated from different fields. We used it to evaluate our approach under realistic conditions and analyzed its generalization capabilities to unseen environments. The obtained results correlated to visual estimation by human experts significantly. The presented study underlined the potential of high-resolution RGB imaging and convolutional neural networks for plant disease detection under field conditions. The demonstrated procedure is particularly interesting for applications under practical conditions, as no complex and cost-intensive measuring system is required.

Monitoring plants and fruits is important in modern agriculture, with applications ranging from high-throughput phenotyping to autonomous harvesting. Obtaining highly accurate 3D measurements under real agricultural conditions is a challenging task. In this letter, we address the problem of estimating the 3D shape of fruits when only a partial view is available. We propose a pipeline that exploits high-resolution 3D data in the learning phase but only requires a single RGB-D frame to predict the 3D shape of a complete fruit during operation. To achieve this, we first learn a latent space of potential fruit appearances that we can decode into an SDF volume. With the pretrained, frozen decoder, we subsequently learn an encoder that can produce meaningful latent vectors from a single RGB-D frame. The experiments presented in this letter suggest that our approach can predict the 3D shape of whole fruits online, needing only 4 ms for inference. We evaluate our approach in controlled environments and illustrate its deployment in greenhouses without modifications.

Measuring plant traits with high throughput allows breeders to monitor and select the best cultivars for subsequent breeding generations. This can enable farmers to improve yield to produce more food, feed, and fiber. Current breeding practices involve extracting leaf parameters on a small subset of the leaves present in the breeding plots, while still requiring substantial manual labor. To automate this process, an important step is the precise distinction between separate leaves, which is the problem we address in this letter. We exploit recent advancements in 3D deep learning to build a convolutional neural network that learns to segment individual leaves. As done in current breeding practices, we select a subset of leaves to be used for phenotypic trait evaluation as this allows us to alleviate the influence of segmentation errors on the phenotypic trait estimation. To this extent we propose to use an additional neural network to predict the quality of each segmented leaf and discard inaccurate leaf instances. The experiments show that our network yields higher segmentation accuracy on sugar beet breeding plots planted under the supervision of the German Federal Office for Plant Varieties. Furthermore, we show that our neural network helps in filtering out leaves with lower segmentation accuracy.

Scene understanding is crucial for autonomous systems to reliably navigate in the real world. Panoptic segmentation of 3D LiDAR scans allows us to semantically describe a vehicle's environment by predicting semantic classes for each 3D point and to identify individual instances through different instance IDs. To describe the dynamics of the surroundings, 4D panoptic segmentation further extends this information with temporarily consistent instance IDs to identify the different instances in the scans consistently over whole sequences. Previous approaches for 4D panoptic segmentation rely on post-processing steps and are often not end-to-end trainable. In this paper, we propose a novel approach that can be trained end-to-end and directly predicts a set of non-overlapping masks along with their semantic classes and instance IDs that are consistent over time without any post-processing like clustering or associations between predictions. We extend a mask-based 3D panoptic segmentation model to 4D by reusing queries that decoded instances in previous scans. This way, each query decodes the same instance over time, carries its ID and the tracking is performed implicitly. This enables us to jointly optimize segmentation and tracking and directly supervise for 4D panoptic segmentation.

Background: This study addresses the importance of precise referencing in 3-dimensional (3D) plant phenotyping, which is crucial for advancing plant breeding and improving crop production. Traditionally, reference data in plant phenotyping rely on invasive methods. Recent advancements in 3D sensing technologies offer the possibility to collect parameters that cannot be referenced by manual measurements. This work focuses on evaluating a 3D printed sugar beet plant model as a referencing tool. Results: Fused deposition modeling has turned out to be a suitable 3D printing technique for creating reference objects in 3D plant phenotyping. Production deviations of the created reference model were in a low and acceptable range. We were able to achieve deviations ranging from −10 mm to +5 mm. In parallel, we demonstrated a high-dimensional stability of the reference model, reaching only ±4 mm deformation over the course of 1 year. Detailed print files, assembly descriptions, and benchmark parameters are provided, facilitating replication and benefiting the research community. Conclusion: Consumer-grade 3D printing was utilized to create a stable and reproducible 3D reference model of a sugar beet plant, addressing challenges in referencing morphological parameters in 3D plant phenotyping. The reference model is applicable in 3 demonstrated use cases: evaluating and comparing 3D sensor systems, investigating the potential accuracy of parameter extraction algorithms, and continuously monitoring these algorithms in practical experiments in greenhouse and field experiments. Using this approach, it is possible to monitor the extraction of a nonverifiable parameter and create reference data. The process serves as a model for developing reference models for other agricultural crops.