Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

The hemlock woolly adelgid (HWA; Adelges tsugae) is an invasive insect infestation that is spreading into the forests of the northeastern United States, driven by the warmer winter temperatures associated with climate change. The initial stages of this disturbance are difficult to detect with passive optical remote sensing, since the insect often causes its host species, eastern hemlock trees (Tsuga canadensis), to defoliate in the midstory and understory before showing impacts in the overstory. New active remote sensing technologies—such as the recently launched NASA Global Ecosystem Dynamics Investigation (GEDI) spaceborne lidar—can address this limitation by penetrating canopy gaps and recording lower canopy structural changes. This study explores new opportunities for monitoring the HWA infestation with airborne lidar scanning (ALS) and GEDI spaceborne lidar data. GEDI waveforms were simulated using airborne lidar datasets from an HWA-infested forest plot at the Harvard Forest ForestGEO site in central Massachusetts. Two airborne lidar instruments, the NASA G-LiHT and the NEON AOP, overflew the site in 2012 and 2016. GEDI waveforms were simulated from each airborne lidar dataset, and the change in waveform metrics from 2012 to 2016 was compared to field-derived hemlock mortality at the ForestGEO site. Hemlock plots were shown to be undergoing dynamic changes as a result of the HWA infestation, losing substantial plant area in the middle canopy, while still growing in the upper canopy. Changes in midstory plant area (PAI 11–12 m above ground) and overall canopy permeability (indicated by RH10) accounted for 60% of the variation in hemlock mortality in a logistic regression model. The robustness of these structure-condition relationships held even when simulated waveforms were treated as real GEDI data with added noise and sparse spatial coverage. These results show promise for future disturbance monitoring studies with ALS and GEDI lidar data.

Vicarious calibration is the determination of an on-orbit sensor’s radiometric response using measurements over test sites such as Railroad Valley (RRV), Nevada. It has the highest accuracy when a remote sensor’s view angle is aligned with that of the surface measurements, namely at a nadir view. For view angles greater than 10°, the dominant error is the uncertainty in the off-nadir correction factor. The factor is largest in the back-scatter principal plane and can reach 20%. The Orbiting-Carbon Observatory has access to a number of datasets to determine this deviation. These include measurements from field instruments such as the Portable Apparatus for Rapid Acquisition of Bidirectional Observation of the Land and Atmosphere (PARABOLA), as well as satellite measurements from Multi-angle Imaging SpectroRadiometer (MISR) and MODerate resolution Imaging Spectroradiometer (MODIS). The correction factor derived from PARABOLA is consistent in time and space to within 2% for view angles as large as 30°. Field spectrometer data show that the correction term is spectrally invariant. For this reason, a time-invariant model of RRV surface reflectance, along with empirically derived coefficients, is sufficient to use in the calibration of off-nadir sensors, provided there has been no recent rainfall. With this off-nadir correction, calibrations can be expected to have uncertainties within 5%.

Landsat-8 derived Land Surface Temperature (LST) is used to measure Surface Urban Heat Island (SUHI) patterns and intensity in Worcester, MA, USA. Additionally, near-surface air temperature variability is measured using in-situ sensors to further contextualize the urban-to-rural land-cover driven thermal patterns in the study area. Despite the widespread applicability of thermal data, many SUHI studies do not compare LST with in-situ information. Comparisons between satellite-based and in-situ measurements of land surface temperature (LST) are important for establishing confidence in the utility of remotely sensed information to monitor and help improve the lived experience in cities. The objective of this study is to determine the capability of Landsat-8 Thermal Infrared Scanner (TIRS) to measure fine-scale temperature variation in a moderately sized urban area with a mixture of land-cover types. Ground-Level Temperature (GLT) was measured at 13 sites using iButton® Thermochron® temperature sensors from 13 June 2013 to 28 October 2014. Landsat-derived LST was compared to in-situ GLT using 30 Landsat-8 TIRS images for the commensurate time period. Sites with 1) eastern solar exposure; 2) low tree canopy coverage; and 3) proximity to impervious surfaces have higher annual temperature and greater offset from Landsat-derived LST. Sites with more than 47% tree canopy coverage have a more consistent LST and GLT relationship, (e.g. MAE < 3.74°C), show lower annual variability (e.g. r² > 0.85), and also experienced low LST variability over the time series (e.g. coefficient of variation 0.007). Results indicate that site characteristics and land-cover type affect the offset between in-situ air temperature measurements and satellite-derived LST.