European satellite missions Sentinel-1 (S1) and Sentinel-2 (S2) provide at high spatial resolution and high revisit time, respectively, radar and optical images that support a wide range of Earth surface monitoring tasks, such as Land Use/Land Cover mapping. A long-standing challenge in the remote sensing community is about how to efficiently exploit multiple sources of information and leverage their complementarity, in order to obtain the most out of radar and optical data. In this work, we propose to deal with land cover mapping in an object-based image analysis (OBIA) setting via a deep learning framework designed to leverage the multi-source complementarity provided by radar and optical satellite image time series (SITS). The proposed architecture is based on an extension of Recurrent Neural Network (RNN) enriched via a modified attention mechanism capable to fit the specificity of SITS data. Our framework also integrates a pretraining strategy that allows to exploit specific domain knowledge, shaped as hierarchy over the set of land cover classes, to guide the model training. Thorough experimental evaluations, involving several competitive approaches were conducted on two study sites, namely the Reunion island and a part of the Senegalese groundnut basin. Classification results, 79% of global accuracy on the Reunion island and 90% on the Senegalese site, respectively, have demonstrated the suitability of the proposal.
Satellite image time series (SITS) collected by modern Earth Observation (EO) systems represent a valuable source of information that supports several tasks related to the monitoring of the Earth surface dynamics over large areas. A main challenge is then to design methods able to leverage the complementarity between the temporal dynamics and the spatial patterns that characterize these data structures. Focusing on land cover classification (or mapping) tasks, the majority of approaches dealing with SITS data only considers the temporal dimension, while the integration of the spatial context is frequently neglected. In this work, we propose an attentive spatial temporal graph convolutional neural network that exploits both spatial and temporal dimensions in SITS. Despite the fact that this neural network model is well suited to deal with spatio-temporal information, this is the first work that considers it for the analysis of SITS data. Experiments are conducted on two study areas characterized by different land cover landscapes and real-world operational constraints (i.e., limited labeled data due to acquisition costs). The results show that our model consistently outperforms all the competing methods obtaining a performance gain, in terms of F-Measure, of at least 5 points with respect to the best competing approaches on both benchmarks.
11 October 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Extensive research studies have been conductedin recent years to exploit the complementarity among multi-sensor (or multi-modal) remote sensing data for prominentapplications such as land cover mapping. In order to make a step further with respect to previous studies which investigate multi-temporal SAR and optical data or multi-temporal/multi-scale optical combinations, here we propose a deep learning framework that simultaneously combine all these input sources, specifically multi-temporal SAR/optical data and fine scale optical information. Our proposal relies on a patch-based multi-branch convolutional neural network (CNN) that exploits different per source CNN encoders to deal with the specificity of the input signals. Additionally, our framework is equipped with a self-distillation strategy to boost the per source analyses. This new strategy leverages the final prediction of the multi-source framework to guide the learning of the per source CNN encoders supporting the network to learn from itself.Experiments are carried out on two real world benchmarks, namely the Reunion island (a french overseas department) and the Dordogne study site (a southwest department in France)where the annotated reference data was collected under operational constraints (sparsely annotated ground truth data). Obtained results, providing an overall classification accuracyof about 94% (resp. 88%) on theReunion island(resp. theDordogne) study site. These findings highlight the effectivenessof our framework based on Convolutional Neural Networks andself-distillation to combine heterogeneous multi-sensor remotesensing data and confirm the benefit of multi-modal analysisfor downstream tasks such as land cover mapping.
6 June 2019 in IEEE Geoscience and Remote Sensing Letters
Modern earth observation (EO) systems produce huge volumes of images with the objective to monitor the earth surface. Due to the high revisit time of EO systems, such as Sentinel-2 constellation, satellite image time series (SITS) is continuously produced allowing to improve the monitoring of spatiotemporal phenomena. How to efficiently analyze SITS considering both spectral and spatial information is still an open question in the remote sensing field. To deal with SITS classification, in this letter, we propose a spatio-spectral classification framework that leverages the mathematical morphology to extract spatial characteristics from SITS data and combines them with the already available spectral and temporal information. Experiments carried out on two study sites characterized by different heterogeneous land covers have demonstrated the significance of our proposal and the value to combine spatial as well as spectral information in the context of SITS land cover classification.
9 November 2021 in International Journal of Remote Sensing
In Sub-Saharan Africa, smallholder farms play a key role in agriculture, occupying most of the agricultural land. Design policies for increasing smallholder productivity remains a safe way to establish sustainable food systems and boost local economies. However, efforts are still needed in order to achieve accurate and timely monitoring in smallholder farming systems. With the advent of modern Earth Observation programmes such as the Sentinel satellites, which provide quasi-synchronous and high-resolution multi-source information over any area of the continental surfaces, new opportunities are opened up to accurately map crop yields in smallholder farming systems. This study intends to estimate and forecast millet yields in central Senegal, making the use of multi-source (synthetic-aperture radar (SAR) and optical) image time series and state-of-the-art machine learning models. A Random Forest (RF) model explained up to 50% of the millet yield variability, while deep learning models such as Convolutional Neural Network (CNN) showed promise results but performed lower. We also found that the concatenation of SAR polarizations and vegetation indices improved our crop yield modelling, but such improvement was tightly related to the modelling approach, namely RF and CNN. Using RF to forecast millet yields, we achieved stable and satisfactory accuracy 2 weeks before the harvest period.