Species-level crop and weed semantic segmentation in agricultural field images enables plant identification and enhanced precision weed management. However, the scarcity of labeled data poses significant challenges for model development. Here, we report a patch-level synthetic data generation pipeline that improves semantic segmentation performance in natural agriculture scenes by creating realistic training samples, achieved by pasting patches of segmented plants onto soil backgrounds. This pipeline effectively preserves foreground context and ensures diverse and accurate samples, thereby enhancing model generalization. The semantic segmentation performance of the baseline model was higher when trained solely on data synthesized by our proposed method compared to training solely on real data, with an approximate increase in the mean intersection over union (mIoU) by approximately 1.1% (from 0.626 to 0.633). Building on this, we created hybrid datasets by combining synthetic and real data and investigated the impact of synthetic data volume. By increasing the number of synthetic images in these hybrid datasets from 1× to 20×, we observed a substantially performance improvement, with mIoU increasing by 15% at 15×. However, the gains diminish beyond this point, with the optimal balance between accuracy and efficiency achieved at 10×. These findings highlight synthetic data as a scalable and effective augmentation strategy for addressing the challenges of limited labeled data in agriculture.

The increase in the global population is leading to a doubling of the demand for protein. Soybean (Glycine max), a key contributor to global plant-based protein supplies, requires ongoing yield enhancements to keep pace with increasing demand. Precise, on-plant seed counting and localization may catalyze breeding selection of shoot architectures and seed localization patterns related to superior performance in high planting density and contribute to increased yield. Traditional manual counting and localization methods are labor-intensive and prone to error, necessitating more efficient approaches for yield prediction and seed distribution analysis. To solve this, we propose MSANet: a novel deep learning framework tailored for counting and localization of soybean seeds on mature field-grown soy plants. A multi-scale attention map mechanism was applied to maximize model performance in seed counting and localization in soybean breeding fields. We compared our model with a previous state-of-the-art model using the benchmark dataset and an enlarged dataset, including various soybean genotypes. Our model outperforms previous state-of-the-art methods on all datasets across various soybean genotypes on both counting and localization tasks. Furthermore, our model also performed well on in-canopy 360° video, dramatically increasing data collection efficiency. We also propose a technique that enables previously inaccessible insights into the phenotypic and genetic diversity of single plant vertical seed distribution, which may accelerate the breeding process. To accelerate further research in this domain, we have made our dataset and software publicly available: https://github.com/UTokyo-FieldPhenomics-Lab/MSANet.

Paddy is a critical global food crop, and its productivity is directly dependent on the number of panicles. Accurate panicle counting is essential for effective crop management. Existing localization-based methods rely on intermediate representations like density maps, which are often counter-intuitive and prone to errors. To quantify and locate panicles directly rather than sequentially, we propose P2PNet-Paddy, a novel approach based on the Point-to-Point Network (P2PNet) architecture, which was initially developed for crowd counting and localization. While the original P2PNet uses the Hungarian algorithm for point matching, it is found to result in uncertain matching results without context, reducing the counting efficiency. It is found that P2PNet encounters significant performance limitations when applied to agricultural datasets characterized by highly imbalanced panicle density distributions. We suggest integrating a density-based sampler to mitigate the impact of panicle density imbalance in the dataset and the k-nearest neighbors Matching Objective (KMO) Hungarian matcher to enhance the matching process by incorporating additional contextual information. Our proposed approach led to notable improvements, reducing MAE by 36% and MAPE by 48.3% compared to the baseline P2PNet model. The proposed method achieved an MAE of 7.31, MAPE of 39.18, and R² score of 0.87 on the test data. The dataset and code will be made publicly available at https://github.com/tejasri19/P2PNet-Paddy.