The Ensemble Streamflow Prediction (ESP) system, developed by the National Weather Service (NWS), uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin. Using this approach, simulated historical probabilistic forecasts were generated for 14 forecast points in the Colorado River basin, and the statistical properties of the ensembles were evaluated. The median forecast traces were analyzed using “traditional” verification measures; these forecasts represented “deterministic ESP forecasts.” The minimum-error and historical traces were examined to evaluate the median forecasts and the forecast system. Distribution-oriented verification measures were used to analyze the probabilistic information contained in the entire forecast ensemble. Using a single-trace prediction, for example, the median, resulted in a loss of valuable uncertainty information about predicted seasonal volumes that i... Abstract The Ensemble Streamflow Prediction (ESP) system, developed by the National Weather Service (NWS), uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin. Using this approach, simulated historical probabilistic forecasts were generated for 14 forecast points in the Colorado River basin, and the statistical properties of the ensembles were evaluated. The median forecast traces were analyzed using “traditional” verification measures; these forecasts represented “deterministic ESP forecasts.” The minimum-error and historical traces were examined to evaluate the median forecasts and the forecast system. Distribution-oriented verification measures were used to analyze the probabilistic information contained in the entire forecast ensemble. Using a single-trace prediction, for example, the median, resulted in a loss of valuable uncertainty information about predicted seasonal volumes that i...
Prediction of snowmelt has become a critical issue in much of the western United States given the increasing demand for water supply, changing snow cover patterns, and the subsequent requirement of optimal reservoir operation. The increasing importance of hydrologic predictions necessitates that traditional forecasting systems be re-evaluated periodically to assure continued evolution of the operational systems given scientific advancements in hydrology. The National Weather Service (NWS) SNOW17, a conceptually based model used for operational prediction of snowmelt, has been relatively unchanged for decades. In this study, the Snow–Atmosphere–Soil Transfer (SAST) model, which employs the energy balance method, is evaluated against the SNOW17 for the simulation of seasonal snowpack (both accumulation and melt) and basin discharge. We investigate model performance over a 13-year period using data from two basins within the Reynolds Creek Experimental Watershed located in southwestern Idaho. Both models are coupled to the NWS runoff model [SACramento Soil Moisture Accounting model (SACSMA)] to simulate basin streamflow. Results indicate that while in many years simulated snowpack and streamflow are similar between the two modeling systems, the SAST more often overestimates SWE during the spring due to a lack of mid-winter melt in the model. The SAST also had more rapid spring melt rates than the SNOW17, leading to larger errors in the timing and amount of discharge on average. In general, the simpler SNOW17 performed consistently well, and in several years, better than, the SAST model. Input requirements and related uncertainties, and to a lesser extent calibration, are likely to be primary factors affecting the implementation of an energy balance model in operational streamflow prediction.
The hydrologic community is generally moving towards the use of probabilistic estimates of streamflow, primarily through the implementation of Ensemble Streamflow Prediction (ESP) systems, ensemble data assimilation methods, or multi-modeling platforms. However, evaluation of probabilistic outputs has not necessarily kept pace with ensemble generation. Much of the modeling community is still performing model evaluation using standard deterministic measures, such as error, correlation, or bias, typically applied to the ensemble mean or median. Probabilistic forecast verification methods have been well developed, particularly in the atmospheric sciences, yet few have been adopted for evaluating uncertainty estimates in hydrologic model simulations. In the current paper, we overview existing probabilistic forecast verification methods and apply the methods to evaluate and compare model ensembles produced from two different parameter uncertainty estimation methods: the Generalized Uncertainty Likelihood Estimator (GLUE), and the Shuffle Complex Evolution Metropolis (SCEM). Model ensembles are generated for the National Weather Service SACramento Soil Moisture Accounting (SAC-SMA) model for 12 forecast basins located in the Southeastern United States. We evaluate the model ensembles using relevant metrics in the following categories: distribution, correlation, accuracy, conditional statistics, and categorical statistics. We show that the presented probabilistic metrics are easily adapted to model simulation ensembles and provide a robust analysis of model performance associated with parameter uncertainty. Application of these methods requires no information in addition to what is already available as part of traditional model validation methodology and considers the entire ensemble or uncertainty range in the approach.