Faced with an unprecedented amount of data coming from evermore ubiquitous sensors, the wastewater treatment community has been hard at work to develop new monitoring systems, models and controllers to bridge the gap between current practice and data-driven, smart water systems. For additional sensor data and models to have an appreciable impact, however, they must be relevant enough to be looked at by busy water professionals; be clear enough to be understood; be reliable enough to be believed and be convincing enough to be acted upon. Failure to attain any one of those aspects can be a fatal blow to the adoption of even the most promising new measurement technology. This review paper examines the state-of-the-art in the transformation of raw data into actionable insight, specifically for water resource recovery facility (WRRF) operation. Sources of difficulties found along the way are pinpointed, while also exploring possible paths towards improving the value of collected data for all stakeholders, i.e., all personnel that have a stake in the good and efficient operation of a WRRF.
24 February 2025 in Current Opinion in Biotechnology
This paper highlights the potential of machine learning (ML) for wastewater applications, with a focus on key applications and considerations. It underscores the need for simplicity in ML models to ensure their interpretability and trustworthiness, cautioning against the use of overly complex 'black box' models unless absolutely necessary, especially with limited data. Not all modelling problems should be considered nails for which the ML hammer is the best-available tool. We emphasise the critical role of thorough data collection, including metadata, given its scarcity in some areas. Future research is encouraged to develop benchmark hybrid models to bridge the educational gap for environmental engineers and to establish best practices for managing data and model metadata, thereby improving ML's accessibility and utility in wastewater applications.
The recent SARS-COV-2 pandemic has sparked the adoption of wastewater-based epidemiology (WBE) as a low-cost way to monitor the health of populations. In parallel, the pandemic has encouraged researchers to openly share their data to serve the public better and accelerate science. However, environmental surveillance data are highly dependent on context and are difficult to interpret meaningfully across sites. This paper presents the second iteration of the Public Health Environmental Surveillance Open Data Model (PHES-ODM), an open-source dictionary and set of data tools to enhance the interoperability of environmental surveillance data and enable the storage of contextual (meta)data. The data model describes how to store environmental surveillance program data, metadata about measurements taken on various specimens (water, air, surfaces, sites, populations) and data about measurement protocols. The model provides software tools that support the collection and use of PHES-ODM formatted data, including performing PCR calculations and data validation, recording data into input templates, generating wide tables for analysis, and producing SQL database definitions. Fully open-source and already adopted by institutions in Canada, the European Union, and other countries, the PHES-ODM provides a path forward for creating robust, interoperable, open datasets for environmental public health surveillance for SARS-CoV-2 and beyond.
Wastewater surveillance (WWS) has quickly emerged as an invaluable tool for public health surveillance, particularly in the wake of the COVID-19 pandemic. Its long-term utility is constrained, however, by fragmented data systems, inconsistent metadata practices, and poor interoperability. The Public Health and Environmental Surveillance Open Data Model (PHES-ODM) was developed as an open, collaborative framework to standardize WWS data and support transparent, ethical data use aligned with FAIR principles in response to these challenges. Building on the success and global adoption of earlier versions, this paper introduces version 3 of the model, expanding to address persistent barriers to interoperability and data utility. Key enhancements include improved metadata capture, support for complex relational linkages across sites, samples, measures, and populations, and new tables for public health actions, external data linkages, and analytical workflows. Tools for mapping across existing standards and supporting long and wide data formats are also introduced. Balancing robustness with usability, PHES-ODM v3 provides a scalable, modular infrastructure adaptable to diverse WWS programmes. The model offers comprehensive solutions for improving data quality, accessibility, and integration, supporting more effective public health decision-making in an increasingly complex global surveillance landscape.
Time series data flows through a four-stage pipeline: outlier removal (autoregressive_filter), fault removal (range_filter), gap filling (daily_pattern), and smoothing (gaussian_kernel). Water resource recovery facilities generate tremendous amounts of process data, most of it in the form of time series. This data can be found in supervisory control and data acquisition systems, process historians, or relational databases where metadata is (hopefully) abundant. Removal of data artefacts and sensor faults, as well as smoothing and aggregation, are all essential steps in transforming raw data into a form fit for modelling, analysis, and decision-making. These transformations typically ignore metadata, meaning metadata-rich datasets can quickly lose all context, traceability, and reusability. The context at risk of being lost includes both the original metadata and descriptions of the processing itself. Failure to report this metadata is due to the difficulty of manual reporting and to the sometimes highly iterative nature of data pre-processing. This paper presents metEAUdata, a framework that automates metadata creation and preservation during pre-processing operations. Unlike existing solutions that bundle limited preprocessing algorithms, metEAUdata provides a lightweight interface enabling any algorithm to become metadata-aware through simple wrapper functions, extending metadata recording to existing scientific libraries with minimal effort. By automating the creation of structured, extensive metadata for arbitrary preprocessing pipelines, metEAUdata supports the transition towards FAIR-compliant time series data.