Crop mapping using satellite imagery is crucial for agriculture applications. However, a fundamental challenge that hinders crop mapping progress is the scarcity of samples. The latest foundation model, Segment Anything Model (SAM), provides an opportunity to address this issue, yet few studies have been conducted in this area. This study investigated the parcel segmentation performance of SAM on commonly used medium-resolution satellite imagery (i.e., Sentinel-2 and Landsat-8) and proposed a novel automated sample generation framework based on SAM. The framework comprises three steps. First, an image optimization automatically selects high-quality images as the inputs for SAM. Then, potential samples are generated based on the masks produced by SAM. Finally, the potential samples are subsequently subjected to a sample cleaning procedure to acquire the most reliable samples. Experiments were conducted in Henan Province, China, and southern Ontario, Canada, using six proven effective classifiers. The effectiveness of our method is demonstrated through the combination of field-survey-collected samples and differently proportioned generated samples. Our results indicated that directly using SAM for parcel segmentation remains challenging, unless the parcels are large, regular in shape, and have distinct color differences from surroundings. Additionally, the proposed approach significantly improved the performance of classifiers and alleviated the sample scarcity problem. Compared to classifiers trained only by field-survey-collected samples, our method resulted in an average improvement of 16% and 78.5% in Henan and Ontario, respectively. The random forest achieved relatively good performance, with weighted-average F1 of 0.97 and 0.996 obtained using Sentinel-2 imagery in the two study areas, respectively. Our study contributes insights into solutions for sample scarcity in crop mapping and highlights the promising application of foundation models like SAM.
16 July 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Large-scale crop mapping not only requires high-quality remote sensing data, but also has high requirements for multi-source data organization and interpretation. How to make full use of multi-source remote sensing data and combine it with artificial intelligence algorithms has become a key solution for current crop mapping. To solve the above problems, this study proposed a spatiotemporal data cubes approach, which could organize multi-source remote sensing data in a unified manner and inherits the deep learning methods. Based on the spatiotemporal data cubes, an innovative sample expansion mechanism was designed, which will effectively solve the problem of sample scarcity in ground surveys. The Attention-based Bidirectional Long Short-Term Memory (AtBiLSTM) network was used with a late-fusion strategy as a classifier on each cube cell to further enhance the incorporation of multi-source information. In this study, experiments towards winter wheat and garlic were conducted in Henan Province of China (approximately 167,000 km2). The approach was verified via comparing with random forest and early-fusion strategy, which demonstrated competitive performance, achieving a weighted-average F1 score of 0.888, with individual F1 scores of 0.919, 0.854, and 0.846 for winter wheat, garlic, and others classes, respectively. The analysis also examined the impact of data availability and found that the temporal distribution of observations was more important than the absolute quantity of available data. Overall, the spatiotemporal data cubes approach proposed in this paper has the potential to be extended to a national, intercontinental, or even global scale, and can provide a reference for large-scale crop mapping.