Kalman filter (KF) and its variants and extensions are wildly used for hydrologic prediction in environmental science and engineering. In many data assimilation applications of Kalman filter (KF) and its variants and extensions, accurate estimation of extreme states is often of great importance. When the observations used are uncertain, however, KF suffers from conditional bias (CB) which results in consistent under- and overestimation of extremes in the right and left tails, respectively. Recently, CB-penalized KF, or CBPKF, has been developed to address CB. In this paper, we present an alternative formulation based on variance-inflated KF to reduce computation and algorithmic complexity, and describe adaptive implementation to improve unconditional performance. For theoretical basis and context, we also provide a complete self-contained description of CB-penalized Fisher-like estimation and CBPKF. The results from one-dimensional synthetic experiments for a linear system with varying degrees of nonstationarity show that adaptive CBPKF reduces the root-mean-square error at the extreme tail ends by 20 to 30% over KF while performing comparably to KF in the unconditional sense. The alternative formulation is found to approximate the original formulation very closely while reducing computing time to 1.5 to 3.5 times of that for KF depending on the dimensionality of the problem. Hence, adaptive CBPKF offers a significant addition to the dynamic filtering methods for general application in data assimilation when the accurate estimation of extremes is of importance.
30 September 2023 in Stochastic Environmental Research and Risk Assessment
When there exist catchment-wide biases in the distributed hydrologic model states, state updating based on streamflow assimilation at the catchment outlet tends to over- and under-adjust model states close to and away from the outlet, respectively. This is due to the greater sensitivity of the simulated outlet flow to the model states that are located more closely to the outlet in the hydraulic sense, and the subsequent overcompensation of the states in the more influential grid boxes to make up for the larger scale bias. In this work, we describe Mean Field Bias (MFB)-aware variational (VAR) assimilation, or MVAR, to address the above. MVAR performs bi-scale state updating of the distributed hydrologic model using streamflow observations in which MFB in the model states are first corrected at the catchment scale before the resulting states are adjusted at the grid box scale. We comparatively evaluate MVAR with conventional VAR based on streamflow assimilation into the distributed Sacramento Soil Moisture Accounting model for a headwater catchment. Compared to VAR, MVAR adjusts model states at remote cells by larger margins and reduces the Mean Squared Error of streamflow analysis by 2–8% at the outlet Tiff City, and by 1–10% at the interior location Lanagan.
This paper presents a comparative geometric analysis of the conditional bias (CB)-informed Kalman filter (KF) with the Kalman filter (KF) in the Euclidean space. The CB-informed KFs considered include the CB-penalized KF (CBPKF) and its ensemble extension, the CB-penalized Ensemble KF (CBEnKF). The geometric illustration for the CBPKF is given for the bi-state model, composed of an observable state and an unobservable state. The CBPKF co-minimizes the error variance and the variance of the Type-II error. As such, CBPKF-updated state error vectors are larger than the KF-updated, the latter of which is based on minimizing the error variance only. Different error vectors in the Euclidean space imply different eigenvectors and covariance ellipses in the state space. To characterize the differences in geometric attributes between the two filters, numerical experiments were carried out using the Lorenz 63 model. The results show that the CBEnKF yields more accurate confidence regions for encompassing the truth, smaller errors in the ensemble mean, and larger norms for Kalman gain and error covariance matrices than the EnKF, particularly when assimilating highly uncertain observations.