The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed.

[1] A recent expansion in global seismic anisotropy data provides important new insights about the style of mantle convection. Interpretations of these geophysical measurements rely on complex relationships between mineral physics, seismology, and mantle dynamics. We report on 3‐D laboratory experiments using finite strain markers evolving in time‐dependent, viscous flow fields to quantify the range in expected anisotropy patterns within buoyant plumes surfacing in a variety of tectonic settings. A surprising result is that laboratory proxies for the olivine fast axis overwhelmingly align tangential to radial outflow in plumes well before reaching the surface. These remarkably robust, and ancient, anisotropy patterns evolve differently in stagnant, translational, and divergent plate tectonic settings and are essentially orthogonal to patterns typically referenced when prospecting for plume signals in seismic data. Results suggest a fundamental change in the mineral physics‐seismology‐circulation relationship used in accepting or rejecting a plume model.

To ensure seamless, programmatic access to data for High Performance Computing (HPC) and analysis across multiple research domains, it is vital to have a methodology for standardization of both data and services. At the Australian National Computational Infrastructure (NCI) we have developed a Data Quality Strategy (DQS) that currently provides processes for: (1) Consistency of data structures needed for a High Performance Data (HPD) platform; (2) Quality Control (QC) through compliance with recognized community standards; (3) Benchmarking cases of operational performance tests; and (4) Quality Assurance (QA) of data through demonstrated functionality and performance across common platforms, tools and services. By implementing the NCI DQS, we have seen progressive improvement in the quality and usefulness of the datasets across the different subject domains, and demonstrated the ease by which modern programmatic methods can be used to access the data, either in situ or via web services, and for uses ranging from traditional analysis methods through to emerging machine learning techniques. To help increase data re-usability by broader communities, particularly in high performance environments, the DQS is also used to identify the need for any extensions to the relevant international standards for interoperability and/or programmatic access.

In recent years, magnetotelluric (MT) processing has become computationally intensive as the scale and size of MT surveys being run increases. Consequently, High Performance Computing (HPC) is now becoming a valuable tool for timely processing and modelling of these large MT datasets. As part of the MT component of the 2017-2019 Australian Research Data Commons (ARDC) funded Geoscience Data Enhanced Virtual Laboratory (DeVL) continuity project, The National Computational Infrastructure (NCI) at the Australian National University will enable MT datasets from The University of Adelaide to be added to the NCI HPC platform with the goal of creating a more Findable, Accessible, Interoperable, Reusable (FAIR) and open public resource. A focus will be on making the time series datasets more suitable for use on HPC and more interoperable with other Earth science disciplines, where High Performance Data (HPD) formats will allow for better scalability and performance. Metadata attributes, as defined by the Australian MT research community, will be added directly to the time series data files. Additionally, time series processing and 3D inversion codes are being optimised for HPD/HPC, with the end goal of rapid time series processing and 3D inversion. Making FAIR MT time series available on HPC can lead to a transformative change in the way MT data analysis is routinely conducted and such a change has the capacity to create new ways of doing collaborative and transparent MT analysis.

Large geophysical data has traditionally been difficult to manage in a consistent, open, and efficient manner. The demands of modern, large-scale computing techniques, coupled with the need for sound data and metadata management, mean that established data formats and access methods are no longer adequate. Geoscience Australia (GA) has been working with its partners to leverage and extend existing data standards to represent various geophysical data in modern scientific container formats including netCDF & HDF. The new data encodings support rapid and efficient data subsetting, either directly from a file or remotely via web services. These will underpin GA’s future data delivery pipelines for Australian government-funded geophysical data. NetCDF efficiently handles multi-variate raster, line, and point data, as well as n-dimensional data structures supporting more demanding applications such as AEM and airborne gravity data. Structural and metadata standards deliver interoperability, and existing and emerging data types are supported without loss of precision or other information. This extended abstract will cover:

NetCDF, the widely-used array-oriented data container file format, has previously been extended in an initiative called netCDF-LD, to include Linked Data metadata elements. In this paper, we build on that initiative with demonstrations of a Simple Knowledge Organization System (SKOS)-aware file format and associated tooling. First, we discuss a very simple way to reference SKOS vocabulary data stored online in netCDF files via Linked Data. Second, we describe our prototype ncskos tools, including ‘ncskosdump’, which wraps the well-known ‘ncdump’ tool used to print out netCDF headers and data. Our tools utilize some of the features of Linked Data and SKOS vocabularies to enhance the metadata of netCDF files by allowing: multilingual metadata label retrieval; alternate term name retrieval; and hierarchical vocabulary relationship navigation. In doing this, ncskosdump preserves the ncdump practice of writing output in standard CDL (network Common Data Language). For the demonstration of theses formats and tools, we relate how we have included URI links in netCDF files to SKOS concepts within a demonstration vocabulary and how the ncskos tools can be used to manage these files in ways that are not possible using only regular netCDF metadata. We also discuss problems we perceived in scaling Linked Data functionality when applying it to large numbers of netCDF files or in multiple file management sessions, and how we have catered for these. Finally, we indicate some future work in the area of more comprehensive Linked Data representation in netCDF files.