In recent years, Earth observation (EO) satellites have generated big amounts of geospatial data that are freely available for society and researchers. This scenario brings challenges for traditional spatial data infrastructures (SDI) to properly store, process, disseminate and analyze these big data sets. To meet these demands, novel technologies have been proposed and developed, based on cloud computing and distributed systems, such as array database systems, MapReduce systems and web services to access and process big Earth observation data. Currently, these technologies have been integrated into cutting edge platforms in order to support a new generation of SDI for big Earth observation data. This paper presents an overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine (GEE), Sentinel Hub, Open Data Cube (ODC), System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud. We also provide a comparison of these platforms according to criteria that represent capabilities of the EO community interest.
The physical phenomena derived from an analysis of remotely sensed imagery provide a clearer understanding of the spectral variations of a large number of land use and cover (LUC) classes. The creation of LUC maps have corroborated this view by enabling the scientific community to estimate the parameter heterogeneity of the Earth’s surface. Along with descriptions of features and statistics for aggregating spatio-temporal information, the government programs have disseminated thematic maps to further the implementation of effective public policies and foster sustainable development. In Brazil, PRODES and DETER have shown that they are committed to monitoring the mapping areas of large-scale deforestation systematically and by means of data quality assurance. However, these programs are so complex that they require the designing, implementation and deployment of a spatial data infrastructure based on extensive data analytics features so that users who lack a necessary understanding of standard spatial interfaces can still carry out research on them. With this in mind, the Brazilian National Institute for Space Research (INPE) has designed TerraBrasilis, a spatial data analytics infrastructure that provides interfaces that are not only found within traditional geographic information systems but also in data analytics environments with complex algorithms. To ensure it achieved its best performance, we leveraged a micro-service architecture with virtualized computer resources to enable high availability, lower size, simplicity to produce an increment, reliable to change and fault tolerance in unstable computer network scenarios. In addition, we tuned and optimized our databases both to adjust to the input format of complex algorithms and speed up the loading of the web application so that it was faster than other systems.
The development of analytical software for big Earth observation data faces several challenges. Designers need to balance between conflicting factors. Solutions that are efficient for specific hardware architectures can not be used in other environments. Packages that work on generic hardware and open standards will not have the same performance as dedicated solutions. Software that assumes that its users are computer programmers are flexible but may be difficult to learn for a wide audience. This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. To allow experts to use satellite imagery to the fullest extent, sits adopts a time-first, space-later approach. It supports the complete cycle of data analysis for land classification. Its API provides a simple but powerful set of functions. The software works in different cloud computing environments. Satellite image time series are input to machine learning classifiers, and the results are post-processed using spatial smoothing. Since machine learning methods need accurate training data, sits includes methods for quality assessment of training samples. The software also provides methods for validation and accuracy measurement. The package thus comprises a production environment for big EO data analysis. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome, one of the world’s fast moving agricultural frontiers for the year 2018.
Recently, remote sensing image time series analysis has being widely used to investigate the dynamics of environments over time. Many studies have combined image time series analysis with machine learning methods to improve land use and cover change mapping. In order to support image time series analysis, analysis-ready data (ARD) image collections have been modeled and organized as multidimensional data cubes. Data cubes can be defined as sets of time series associated with spatially aligned pixels. Based on lessons learned in the research project e-Sensing, related to national demands for land use and cover monitoring and related to state-of-the-art studies on relevant topics, we define the requirements to build Earth observation data cubes for Brazil. This paper presents the methodology to generate ARD and multidimensional data cubes from remote sensing images for Brazil. We describe the computational infrastructure that we are developing in the Brazil Data Cube project, composed of software applications and Web services to create, integrate, discover, access, and process the data sets. We also present how we are producing land use and cover maps from data cubes using image time series analysis and machine learning techniques.
Tropical forests regulate the global water and carbon cycles and also host most of the world’s biodiversity. Despite their importance, they are hard to survey due to their location, extent, and particularly, their cloud coverage. Clouds hinder the spatial and radiometric correction of satellite imagery and also diminishing the useful area on each image, making it difficult to monitor land change. For this reason, our purpose is to identify the cloud detection algorithm best suited for the Amazon rainforest on Sentinel–2 images. To achieve this, we tested four cloud detection algorithms on Sentinel–2 images spread in five areas of the Amazonia. Using more than eight thousand validation points, we compared four cloud detection methods: Fmask 4, MAJA, Sen2Cor, and s2cloudless. Our results point out that FMask 4 has the best overall accuracy on images of the Amazon region (90%), followed by Sen2Cor’s (79%), MAJA (69%), and S2cloudless (52%). We note the choice of method depends on the intended use. Since MAJA reduces the number of false positives by design, users that aim to improve the producer’s accuracy should consider its use.
Earth observation satellites produce petabytes of geospatial data. To manage large data sets, researchers need stable and efficient solutions that support their analytical tasks. Since the technology for big data handling is evolving rapidly, researchers find it hard to keep up with the new developments. To lower this burden, we argue that researchers should not have to convert their algorithms to specialised environments. Imposing a new API to researchers is counterproductive and slows down progress on big data analytics. This paper assesses the cost of research-friendliness, in a case where the researcher has developed an algorithm in the R language and wants to use the same code for big data analytics. We take an algorithm for remote sensing time series analysis on compare it use on map/reduce and on array database architectures. While the performance of the algorithm for big data sets is similar, organising image data for processing in Hadoop is more complicated and time-consuming than handling images in SciDB. Therefore, the combination of the array database SciDB and the R language offers an adequate support for researchers working on big Earth observation data analytics.