Naoki Mizukami

Naoki Mizukami

Auto-generated profile, based on publicly available information

Naoki Mizukami

National Center for Atmospheric Research Boulder CO USA

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 October 2017 in Hydrology and Earth System Sciences
Verified
Journal Article
Open access
PDF file
NANans Addor
Nans Addor
ANAndrew J. Newman
Andrew J. Newman
NMNaoki Mizukami
Naoki Mizukami
MCMartyn P. Clark
Martyn P. Clark
We present a new data set of attributes for 671 catchments in the contiguous United States (CONUS) minimally impacted by human activities. This complements the daily time series of meteorological forcing and streamflow provided by Newman et al. (2015b). To produce this extension, we synthesized diverse and complementary data sets to describe six main classes of attributes at the catchment scale: topography, climate, streamflow, land cover, soil, and geology. The spatial variations among basins over the CONUS are discussed and compared using a series of maps. The large number of catchments, combined with the diversity of the attributes we extracted, makes this new data set well suited for large-sample studies and comparative hydrology. In comparison to the similar Model Parameter Estimation Experiment (MOPEX) data set, this data set relies on more recent data, it covers a wider range of attributes, and its catchments are more evenly distributed across the CONUS. This study also involves assessments of the limitations of the source data sets used to compute catchment attributes, as well as detailed descriptions of how the attributes were computed. The hydrometeorological time series provided by Newman et al. (2015b, https://doi.org/10.5065/D6MW2F4D) together with the catchment attributes introduced in this paper (https://doi.org/10.5065/D6G73C3Q) constitute the freely available CAMELS data set, which stands for Catchment Attributes and MEteorology for Large-sample Studies.

20 September 2021 in Water Resources Research
Verified
Journal Article
Open access
PDF file
MCMartyn P. Clark
Martyn P. Clark
RVRichard M Vogel
Richard M Vogel
JLJonathan R. Lamontagne
Jonathan R. Lamontagne
NMNaoki Mizukami
Naoki Mizukami
WKWouter Knoben
Wouter Knoben
SPSimon Michael Papalexiou
Simon Michael Papalexiou
The goal of this commentary is to critically evaluate the use of popular performance metrics in hydrologic modeling. We focus on the Nash-Sutcliffe Efficiency (NSE) and the Kling-Gupta Efficiency (KGE) metrics, which are both widely used in hydrologic research and practice around the world. Our specific objectives are: (1) to provide tools that quantify the sampling uncertainty in popular performance metrics; (2) to quantify sampling uncertainty in popular performance metrics across a large sample of catchments; and (3) to prescribe the further research that is needed to improve the estimation, interpretation, and use of popular performance metrics in hydrologic modeling. Our large-sample analysis demonstrates that there is substantial sampling uncertainty in the NSE and KGE estimators. This occurs because the probability distribution of squared errors between model simulations and observations has heavy tails, meaning that performance metrics can be heavily influenced by just a few data points. Our results highlight obvious (yet ignored) abuses of performance metrics that contaminate the conclusions of many hydrologic modeling studies: It is essential to quantify the sampling uncertainty in performance metrics when justifying the use of a model for a specific purpose and when comparing the performance of competing models.

17 June 2019 in Hydrology and Earth System Sciences
Verified
Journal Article
Open access
PDF file
NMNaoki Mizukami
Naoki Mizukami
OROldrich Rakovec
Oldrich Rakovec
ANAndrew J. Newman
Andrew J. Newman
MCMartyn P. Clark
Martyn P. Clark
AWAndrew W. Wood
Andrew W. Wood
Rohini KumarRohini Kumar
Rohini Kumar
Calibration is an essential step for improving the accuracy of simulations generated using hydrologic models. A key modeling decision is selecting the performance metric to be optimized. It has been common to use squared error performance metrics, or normalized variants such as Nash–Sutcliffe efficiency (NSE), based on the idea that their squared-error nature will emphasize the estimates of high flows. However, we conclude that NSE-based model calibrations actually result in poor reproduction of high-flow events, such as the annual peak flows that are used for flood frequency estimation. Using three different types of performance metrics, we calibrate two hydrological models at a daily step, the Variable Infiltration Capacity (VIC) model and the mesoscale Hydrologic Model (mHM), and evaluate their ability to simulate high-flow events for 492 basins throughout the contiguous United States. The metrics investigated are (1) NSE, (2) Kling–Gupta efficiency (KGE) and its variants, and (3) annual peak flow bias (APFB), where the latter is an application-specific metric that focuses on annual peak flows. As expected, the APFB metric produces the best annual peak flow estimates; however, performance on other high-flow-related metrics is poor. In contrast, the use of NSE results in annual peak flow estimates that are more than 20 % worse, primarily due to the tendency of NSE to underestimate observed flow variability. On the other hand, the use of KGE results in annual peak flow estimates that are better than from NSE, owing to improved flow time series metrics (mean and variance), with only a slight degradation in performance with respect to other related metrics, particularly when a non-standard weighting of the components of KGE is used. Stochastically generated ensemble simulations based on model residuals show the ability to improve the high-flow metrics, regardless of the deterministic performances. However, we emphasize that improving the fidelity of streamflow dynamics from deterministically calibrated models is still important, as it may improve high-flow metrics (for the right reasons). Overall, this work highlights the need for a deeper understanding of performance metric behavior and design in relation to the desired goals of model calibration.

23 September 2017 in Water Resources Research
Verified
Journal Article
Closed access
PDF file
NMNaoki Mizukami
Naoki Mizukami
MCMartyn P. Clark
Martyn P. Clark
ANAndrew J. Newman
Andrew J. Newman
AWAndrew W. Wood
Andrew W. Wood
EGEthan D. Gutmann
Ethan D. Gutmann
LSLuis Samaniego
Luis Samaniego

1 August 2017 in Journal of Hydrometeorology
Verified
Journal Article
Open access
PDF file
ANAndrew J. Newman
Andrew J. Newman
NMNaoki Mizukami
Naoki Mizukami
MCMartyn P. Clark
Martyn P. Clark
AWAndrew W. Wood
Andrew W. Wood
BNBart Nijssen
Bart Nijssen
GNGrey Nearing
Grey Nearing
The concepts of model benchmarking, model agility, and large-sample hydrology are becoming more prevalent in hydrologic and land surface modeling. As modeling systems become more sophisticated, these concepts have the ability to help improve modeling capabilities and understanding. In this paper, their utility is demonstrated with an application of the physically based Variable Infiltration Capacity model (VIC). The authors implement VIC for a sample of 531 basins across the contiguous United States, incrementally increase model agility, and perform comparisons to a benchmark. The use of a large-sample set allows for statistically robust comparisons and subcategorization across hydroclimate conditions. Our benchmark is a calibrated, time-stepping, conceptual hydrologic model. This model is constrained by physical relationships such as the water balance, and it complements purely statistical benchmarks due to the increased physical realism and permits physically motivated benchmarking using metrics that relate one variable to another (e.g., runoff ratio). The authors find that increasing model agility along the parameter dimension, as measured by the number of model parameters available for calibration, does increase model performance for calibration and validation periods relative to less agile implementations. However, as agility increases, transferability decreases, even for a complex model such as VIC. The benchmark outperforms VIC in even the most agile case when evaluated across the entire basin set. However, VIC meets or exceeds benchmark performance in basins with high runoff ratios (greater than ~0.8), highlighting the ability of large-sample comparative hydrology to identify hydroclimatic performance variations.

17 June 2019 in Earth's Future
Verified
Journal Article
Open access
PDF file
OCOriana S. Chegwidden
Oriana S. Chegwidden
BNBart Nijssen
Bart Nijssen
DRDavid E. Rupp
David E. Rupp
JAJeffrey R. Arnold
Jeffrey R. Arnold
MCMartyn P. Clark
Martyn P. Clark
MXMu Xiao
Mu Xiao
Methodological choices can have strong effects on projections of climate change impacts on hydrology. In this study, we investigate the ways in which four different steps in the modeling chain influence the spread in projected changes of different aspects of hydrology. To form the basis of these analyses, we constructed an ensemble of 160 simulations from permutations of two Representative Concentration Pathways, 10 global climate models, two downscaling methods, and four hydrologic model implementations. The study is situated in the Pacific Northwest of North America, which has relevance to a diverse, multinational cast of stakeholders. We analyze the effects of each modeling decision on changes in gridded hydrologic variables of snow water equivalent and runoff, as well as streamflow at point locations. Results show that the choice of representative concentration pathway or global climate model is the driving contributor to the spread in annual streamflow volume and timing. On the other hand, hydrologic model implementation explains most of the spread in changes in low flows. Finally, by grouping the results by climate region the results have the potential to be generalized beyond the Pacific Northwest. Future hydrologic impact assessments can use these results to better tailor their modeling efforts.