Improvements in the number and resolution of Earth- and satellite-based sensors coupled with finer-resolution models have resulted in an explosion in the volume of Earth science data. This data-rich environment is changing the practice of Earth science, extending it beyond discovery and applied science to new realms. This Review highlights recent big data applications in three subdisciplines—hydrology, oceanography, and atmospheric science. We illustrate how big data relate to contemporary challenges in science: replicability and reproducibility and the transition from raw data to information products. Big data provide unprecedented opportunities to enhance our understanding of Earth’s complex patterns and interactions. The emergence of digital twins enables us to learn from the past, understand the current state, and improve the accuracy of future predictions.

Advances in ocean observations and models mean increasing flows of data. Integrating observations between disciplines over spatial scales from regional to global presents challenges. Running ocean models and managing the results is computationally demanding. The rise of cloud computing presents an opportunity to rethink traditional approaches. This includes developing shared data processing workflows utilizing common, adaptable software to handle data ingest and storage, and an associated framework to manage and execute downstream modeling. Working in the cloud presents challenges: migration of legacy technologies and processes, cloud-to-cloud interoperability, and the translation of legislative and bureaucratic requirements for “on-premises” systems to the cloud. To respond to the scientific and societal needs of a fit-for-purpose ocean observing system, and to maximize the benefits of more integrated observing, research on utilizing cloud infrastructures for sharing data and models is underway. Cloud platforms and the services/APIs they provide offer new ways for scientists to observe and predict the ocean’s state. High-performance mass storage of observational data, coupled with on-demand computing to run model simulations in close proximity to the data, tools to manage workflows, and a framework to share and collaborate, enables a more flexible and adaptable observation and prediction computing architecture. Model outputs are stored in the cloud and researchers either download subsets for their interest/area or feed them into their own simulations without leaving the cloud. Expanded storage and computing capabilities make it easier to create, analyze, and distribute products derived from long-term datasets. In this paper, we provide an introduction to cloud computing, describe current uses of the cloud for management and analysis of observational data and model results, and describe workflows for running models and streaming observational data. We discuss topics that must be considered when moving to the cloud: costs, security, and organizational limitations on cloud use. Future uses of the cloud via computational sandboxes and the practicalities and considerations of using the cloud to archive data are explored. We also consider the ways in which the human elements of ocean observations are changing – the rise of a generation of researchers whose observations are likely to be made remotely rather than hands on – and how their expectations and needs drive research towards the cloud. In conclusion, visions of a future where cloud computing is ubiquitous are discussed.

Habitat in open water is not simply a 2-D to 2.5-D surface such as the ocean bottom or the air-water interface. Rather, pelagic habitat is a 3-D volume of water that can change over time, leading us to the term habitat space. Visualization and analysis in 2-D is well supported with GIS tools, but a new tool was needed for visualization and analysis in four dimensions. Observational data (cruise profiles (x_o, y_o, z, t_o)), numerical circulation model fields (x,y,z,t), and trajectories (larval fish, 4-D line) need to be merged together in a meaningful way for visualization and analysis. As a first step toward this new framework, UNIDATA's Integrated Data Viewer (IDV) has been used to create a set of tools for habitat analysis in 4-D. IDV was designed for 3-D+time geospatial data in the meteorological community. NetCDF Java^TM libraries allow the tool to read many file formats including remotely located data (e.g. data available via OPeNDAP ). With this project, IDV has been adapted for use in delineating habitat space for multiple fish species in the ocean. The ability to define and visualize boundaries of a water mass, which meets specific biologically relevant criteria (e.g., volume, connectedness, and inter-annual variability) based on model results and observational data, will allow managers to investigate the survival of individual year classes of commercially important fisheries. Better understanding of the survival of these year classes will lead to improved forecasting of fisheries recruitment.

The United States National Harmful Algal Bloom Observing Network (NHABON) brings together local, state, Tribal, regional and federal HAB observing capabilities to enable forecasting and early warning systems for harmful algal blooms that leverage economies of scale to provide sustained and reliable information needed to protect public health and food safety. NHABON is a multi-agency partnership that leverages decades of investment in the regional ocean observing systems forming the backbone of the NOAA Integrated Ocean Observing System.In this presentation, I will discuss the emerging NHABON Community of Practice in the U.S., a regional alliance of the Global Ocean Observing System (GOOS), designed to unify early warning systems into a cohesive national network. Specifically, I will review development of the HAB Data Assembly Center: a centralized location for accessing HAB data products, cyberinfrastructure for processing near real-time plankton imagery and biotoxin data, a common code repository for machine learning models and training data sets, and data science and data management support for research teams issuing critical food safety alerts. I will provide examples of how regional systems in the U.S. can be coordinated through a national network, support technologies and monitoring programs to detect blooms, distribute forecasts and early warnings, and provide data and information products. The ultimate aim of NHABON is global access to sustained service delivery for operators and managers in aquaculture, fisheries, and public health.