In the process of implementing a protocol for the transport of science data, the Open Source Project for a Network Data Access Protocol (OPeNDAP) group has learned a considerable amount about the internal anatomy of what are commonly considered monolithic concepts. In order to communicate among our group, we have adopted a collection of deinitions and observations about data and the metadata that make them useful: differentiating between "semantic" and "syntactic" metadata, and deining categories such as "translational" and "use" metadata. We share the deinitions and categorizations here in the hope that others will ind them as useful as we do

We performed an out-of-distribution (OOD) analysis of ∼12,000,000 semi-independent 128 × 128 pixel${}^2$ sea surface temperature (SST) regions, which we define as cutouts, from all nighttime granules in the MODIS R2019 Level-2 public dataset to discover the most complex or extreme phenomena at the ocean’s surface. Our algorithm (ULMO) is a probabilistic autoencoder (PAE), which combines two deep learning modules: (1) an autoencoder, trained on ∼150,000 random cutouts from 2010, to represent any input cutout with a 512-dimensional latent vector akin to a (non-linear) Empirical Orthogonal Function (EOF) analysis; and (2) a normalizing flow, which maps the autoencoder’s latent space distribution onto an isotropic Gaussian manifold. From the latter, we calculated a log-likelihood (LL) value for each cutout and defined outlier cutouts to be those in the lowest 0.1% of the distribution. These exhibit large gradients and patterns characteristic of a highly dynamic ocean surface, and many are located within larger complexes whose unique dynamics warrant future analysis. Without guidance, ULMO consistently locates the outliers where the major western boundary currents separate from the continental margin. Prompted by these results, we began the process of exploring the fundamental patterns learned by ULMO thereby identifying several compelling examples. Future work may find that algorithms such as ULMO hold significant potential/promise to learn and derive other, not-yet-identified behaviors in the ocean from the many archives of satellite-derived SST fields. We see no impediment to applying them to other large remote-sensing datasets for ocean science (e.g., SSH and ocean color).

The primary measure of the quality of sea surface temperature (SST) fields obtained from satellite-borne infrared sensors has been the bias and variance of matchups with co-located in-situ values. Because such matchups tend to be widely separated, these bias and variance estimates are not necessarily a good measure of small scale (several pixels) gradients in these fields because one of the primary contributors to the uncertainty in satellite retrievals is atmospheric contamination, which tends to have large spatial scales compared with the pixel separation of infrared sensors. Hence, there is not a good measure to use in selecting SST fields appropriate for the study of submesoscale processes and, in particular, of processes associated with near-surface fronts, both of which have recently seen a rapid increase in interest. In this study, two methods are examined to address this problem, one based on spectra of the SST data and the other on their variograms. To evaluate the methods, instrument noise was estimated in Level-2 Visible-Infrared Imager-Radiometer Suite (VIIRS) and Advanced Very High Resolution Radiometer (AVHRR) SST fields of the Sargasso Sea. The two methods provided very nearly identical results for AVHRR: along-scan values of approximately 0.18 K for both day and night and along-track values of 0.21 K for day and night. By contrast, the instrument noise estimated for VIIRS varied by method, scan geometry and day-night. Specifically, daytime, along-scan (along-track), spectral estimates were found to be approximately 0.05 K (0.08 K) and the corresponding nighttime values of 0.02 K (0.03 K). Daytime estimates based on the variogram were found to be 0.08 K (0.10 K) with the corresponding nighttime values of 0.04 K (0.06 K). Taken together, AVHRR instrument noise is significantly larger than VIIRS instrument noise, along-track noise is larger than along-scan noise and daytime levels are higher than nighttime levels. Given the similarity of results and the less stringent preprocessing requirements, the variogram is the preferred method, although there is a suggestion that this approach overestimates the noise for high quality data in dynamically quiet regions. Finally, simulations of the impact of noise on the determination of SST gradients show that on average the gradient magnitude for typical ocean gradients will be accurately estimated with VIIRS but substantially overestimated with AVHRR.

We have assembled 2 851 702 nearly cloud-free cutout images (sized 144 km × 144 km) of sea surface temperature (SST) data from the entire 2012–2020 Level-2 Visible Infrared Imaging Radiometer Suite (VIIRS) dataset to perform a quantitative comparison to the ocean model output from the MIT General Circulation Model (MITgcm). Specifically, we evaluate outputs from the LLC4320 (LLC, latitude–longitude–polar cap) $\frac{\mathrm{1}}{\mathrm{48}}{}^{\circ }$ global-ocean simulation for a 1-year period starting on 17 November 2011 but otherwise matched in geography and the day of the year to the VIIRS observations. In lieu of simple (e.g., mean, standard deviation) or complex (e.g., power spectrum) statistics, we analyze the cutouts of SST anomalies with an unsupervised probabilistic autoencoder (PAE) trained to learn the distribution of structures in SST anomaly (SSTa) on ∼ 10–80 km scales (i.e., submesoscale to mesoscale). A principal finding is that the LLC4320 simulation reproduces, over a large fraction of the ocean, the observed distribution of SSTa patterns well, both globally and regionally. Globally, the medians of the structure distributions match to within 2σ for 65 % of the ocean, despite a modest, latitude-dependent offset. Regionally, the model outputs reproduce mesoscale variations in SSTa patterns revealed by the PAE in the VIIRS data, including subtle features imprinted by variations in bathymetry. We also identify significant differences in the distribution of SSTa patterns in several regions: (1) in an equatorial band equatorward of 15^∘; (2) in the Antarctic Circumpolar Current (ACC), especially in the eastern half of the Indian Ocean; and (3) in the vicinity of the point at which western boundary currents separate from the continental margin. It is clear that region 3 is a result of premature separation in the simulated western boundary currents. The model output in region 2, the southern Indian Ocean, tends to predict more structure than observed, perhaps arising from a misrepresentation of the mixed layer or of energy dissipation and stirring in the simulation. The differences in region 1, the equatorial band, are also likely due to model errors, perhaps arising from the shortness of the simulation or from the lack of high-frequency and/or wavenumber atmospheric forcing. Although we do not yet know the exact causes for these model–data SSTa differences, we expect that this type of comparison will help guide future developments of high-resolution global-ocean simulations.