Citizens are increasingly becoming involved in data collection, whether for scientific purposes, to carry out micro-tasks, or as part of a gamified, competitive application. In some cases, volunteered data collection overlaps with that of mapping agencies, e.g., the citizen-based mapping of features in OpenStreetMap. LUCAS (Land Use Cover Area frame Sample) is one source of authoritative in-situ data that are collected every three years across EU member countries by trained personnel at a considerable cost to taxpayers. This paper presents a mobile application called FotoQuest Austria, which involves citizens in the crowdsourcing of in-situ land cover and land use data, including at locations of LUCAS sample points in Austria. The results from a campaign run during the summer of 2015 suggest that land cover and land use can be crowdsourced using a simple protocol based on LUCAS. This has implications for remote sensing as this data stream represents a new source of potentially valuable information for the training and validation of land cover maps as well as for area estimation purposes. Although the most detailed and challenging classes were more difficult for untrained citizens to recognize, the agreement between the crowdsourced data and the LUCAS data for basic high level land cover and land use classes in homogeneous areas (ca. 80%) shows clear potential. Recommendations for how to further improve the quality of the crowdsourced data in the context of LUCAS are provided so that this source of data might one day be accurate enough for land cover mapping purposes.

Accuracy assessment, also referred to as validation, is a key process in the workflow of developing a land cover map. To make this process open and transparent, we have developed a new online tool called LACO-Wiki, which encapsulates this process into a set of four simple steps including uploading a land cover map, creating a sample from the map, interpreting the sample with very high resolution satellite imagery and generating a report with accuracy measures. The aim of this paper is to present the main features of this new tool followed by an example of how it can be used for accuracy assessment of a land cover map. For the purpose of illustration, we have chosen GlobeLand30 for Kenya. Two different samples were interpreted by three individuals: one sample was provided by the GlobeLand30 team as part of their international efforts in validating GlobeLand30 with GEO (Group on Earth Observation) member states while a second sample was generated using LACO-Wiki. Using satellite imagery from Google Maps, Bing and Google Earth, the results show overall accuracies between 53% to 61%, which is lower than the global accuracy assessment of GlobeLand30 but may be reasonable given the complex landscapes found in Kenya. Statistical models were then fit to the data to determine what factors affect the agreement between the three interpreters such as the land cover class, the presence of very high resolution satellite imagery and the age of the image in relation to the baseline year for GlobeLand30 (2010). The results showed that all factors had a significant effect on the agreement.

Mapping the extent and location of field boundaries is critical to food security analysis but remains problematic in the Global South where such information is needed the most. The difficulty is due primarily to fragmentation in the landscape, small farm sizes, and irregular farm boundaries. Very high-resolution satellite imagery affords an opportunity to delineate such fields, but the challenge remains of determining such boundaries in a systematic and accurate way. In this paper, we compare a new crowd-driven manual digitization tool (Crop Land Extent) with two semi-automated methods (contour detection and multi-resolution segmentation) to determine farm boundaries from WorldView imagery in highly fragmented agricultural landscapes of Ethiopia. More than 7000 one square-kilometer image tiles were used for the analysis. The three methods were assessed using quantitative completeness and spatial correctness. Contour detection tended to under-segment when compared to manual digitization, resulting in better performance for larger (approaching 1 ha) sized fields. Multi-resolution segmentation on the other hand, tended to over-segment, resulting in better performance for small fields. Neither semi-automated method in their current realizations however are suitable for field boundary mapping in highly fragmented landscapes. Crowd-driven manual digitization is promising, but requires more oversight, quality control, and training than the current workflow could allow.

Simple consensus methods are often used in crowdsourcing studies to label cases when data are provided by multiple contributors. A basic majority vote rule is often used. This approach weights the contributions from each contributor equally but the contributors may vary in the accuracy with which they can label cases. Here, the potential to increase the accuracy of crowdsourced data on land cover identified from satellite remote sensor images through the use of weighted voting strategies is explored. Critically, the information used to weight contributions based on the accuracy with which a contributor labels cases of a class and the relative abundance of class are inferred entirely from the contributed data only via a latent class analysis. The results show that consensus approaches do yield a classification that is more accurate than that achieved by any individual contributor. Here, the most accurate individual could classify the data with an accuracy of 73.91% while a basic consensus label derived from the data provided by all seven volunteers contributing data was 76.58%. More importantly, the results show that weighting contributions can lead to a statistically significant increase in the overall accuracy to 80.60% by ignoring the contributions from the volunteer adjudged to be the least accurate in labelling.

There is a need to validate existing global cropland maps since they are used for different purposes including agricultural monitoring and assessment. In this paper we validate three recent global products (ESA-CCI, GlobeLand30, FROM-GC) and one regional product (Tanzania Land Cover 2010 Scheme II) using a validation data set that was collected by students through the Geo-Wiki tool. The ultimate aim was to understand the usefulness of these products for agricultural monitoring. Data were collected wall-to-wall for Kilosa district and for a sample across Tanzania. The results show that the amount of and spatial extent of cropland in the different products differs considerably from 8% to 42% for Tanzania, with similar values for Kilosa district. The agreement of the validation data with the four different products varied between 36% and 54% and highlighted that cropland is overestimated by the ESA-CCI and underestimated by FROM-GC. The validation data were also analyzed for consistency between the student interpreters and also compared with a sample interpreted by five experts for quality assurance. Regarding consistency between the students, there was more than 80% agreement if one difference in cropland category was considered (e.g., between low and medium cropland) while most of the confusion with the experts was also within one category difference. In addition to the validation of current cropland products, the data set collected by the students also has potential value as a training set for improving future cropland products.

Volunteered geographic data are being used increasingly to support land cover mapping and validation, yet the reliability of the volunteered data still requires further research. This study proposes data-based guidelines to help design the data collection by assessing the reliability of volunteered data collected using the Geo-Wiki tool. We summarized the interpretation difficulties of the volunteers at a global scale, including those areas and land cover types that generate the most confusion. We also examined the factors affecting the reliability of majority opinion and individual classification. The results showed that the highest interpretation inconsistency of the volunteers occurred in the ecoregions of tropical and boreal forests (areas with relatively poor coverage of very high resolution images), the tundra (a unique region that the volunteers are unacquainted with), and savannas (transitional zones). The volunteers are good at identifying forests, snow/ice and croplands, but not grasslands and wetlands. The most confusing pairs of land cover types are also captured in this study and they vary greatly with different biomes. The reliability can be improved by providing more high resolution ancillary data, more interpretation keys in tutorials, and tools that assist in coverage estimation for those areas and land cover types that are most prone to confusion. We found that the reliability of the majority opinion was positively correlated with the percentage of volunteers selecting this choice and negatively related to their self-evaluated uncertainty when very high resolution images were available. Factors influencing the reliability of individual classifications were also compared and the results indicated that the interpretation difficulty of the target sample played a more important role than the knowledge base of the volunteers. The professional background and local knowledge had an influence on the interpretation performance, especially in identifying vegetation land cover types other than croplands. These findings can help in building a better filtering system to improve the reliability of volunteered data used in land cover validation and other applications.