Historical reanalyses that span more than a century are needed for a wide range of studies, from understanding large‐scale climate trends to diagnosing the impacts of individual historical extreme weather events. The Twentieth Century Reanalysis (20CR) Project is an effort to fill this need. It is supported by the National Oceanic and Atmospheric Administration (NOAA), the Cooperative Institute for Research in Environmental Sciences (CIRES), and the U.S. Department of Energy (DOE), and is facilitated by collaboration with the international Atmospheric Circulation Reconstructions over the Earth initiative. 20CR is the first ensemble of sub‐daily global atmospheric conditions spanning over 100 years. This provides a best estimate of the weather at any given place and time as well as an estimate of its confidence and uncertainty. While extremely useful, version 2c of this dataset (20CRv2c) has several significant issues, including inaccurate estimates of confidence and a global sea level pressure bias in the mid‐19th century. These and other issues can reduce its effectiveness for studies at many spatial and temporal scales. Therefore, the 20CR system underwent a series of developments to generate a significant new version of the reanalysis. The version 3 system (NOAA‐CIRES‐DOE 20CRv3) uses upgraded data assimilation methods including an adaptive inflation algorithm; has a newer, higher‐resolution forecast model that specifies dry air mass; and assimilates a larger set of pressure observations. These changes have improved the ensemble‐based estimates of confidence, removed spin‐up effects in the precipitation fields, and diminished the sea‐level pressure bias. Other improvements include more accurate representations of storm intensity, smaller errors, and large‐scale reductions in model bias. The 20CRv3 system is comprehensively reviewed, focusing on the aspects that have ameliorated issues in 20CRv2c. Despite the many improvements, some challenges remain, including a systematic bias in tropical precipitation and time‐varying biases in southern high‐latitude pressure fields.

The performance of a new historical reanalysis, the NOAA-CIRES-DOE 20^th Century Reanalysis Version 3 (20CRv3), is evaluated via comparisons with other reanalyses and independent observations. This dataset provides global, 3-hourly estimates of the atmosphere from 1806 to 2015 by assimilating only surface pressure observations and prescribing sea surface temperature, sea ice concentration, and radiative forcings. Comparisons with independent observations, other reanalyses, and satellite products suggest that 20CRv3 can reliably produce atmospheric estimates on scales ranging from weather events to long-term climatic trends. Not only does 20CRv3 recreate a “best estimate” of the weather, including extreme events, it also provides an estimate of its confidence through the use of an ensemble. Surface pressure statistics suggest that these confidence estimates are reliable. Comparisons with independent upper-air observations in the Northern Hemisphere demonstrate that 20CRv3 has skill throughout the 20^th century. Upper-air fields from 20CRv3 in the late 20th century and early 21^st century correlate well with full-input reanalyses, and the correlation is predicted by the confidence fields from 20CRv3. The skill of analyzed 500hPa geopotential heights from 20CRv3 for 1979-2015 is comparable to that of modern operational 3- to 4-day forecasts. Finally, 20CRv3 performs well on climate timescales. Long time series and multidecadal averages of mass, circulation, and precipitation fields agree well with modern reanalyses and station- and satellite-based products. 20CRv3 is also able to capture trends in tropospheric layer temperatures that correlate well with independent products in the 20^th century, placing recent trends in a longer historical context.

Lagrangian measurements from passive ocean instruments provide a useful source of data for estimating and forecasting the ocean’s state (velocity field, salinity field, etc.). However, trajectories from these instruments are often highly nonlinear, leading to difficulties with widely used data assimilation algorithms such as the ensemble Kalman filter (EnKF). Additionally, the velocity field is often modeled as a high-dimensional variable, which precludes the use of more accurate methods such as the particle filter (PF). Here, a hybrid particle–ensemble Kalman filter is developed that applies the EnKF update to the potentially high-dimensional velocity variables, and the PF update to the relatively low-dimensional, highly nonlinear drifter position variable. This algorithm is tested with twin experiments on the linear shallow water equations. In experiments with infrequent observations, the hybrid filter consistently outperformed the EnKF, both by better capturing the Bayesian posterior and by better tracking the truth.

Historical reanalyses combine observations of past weather with simulations from modern numerical weather prediction models to provide a consistent, global history of the weather. Recently, a new reanalysis was released that allows observations of the ocean to impact the atmosphere and vice versa. “CERA‐20C: A Coupled Reanalysis of the Twentieth Century” by P. Laloyaux et al. (2018, https://doi.org/10.1029/2018MS001273) describes the first coupled centennial reanalysis, thereby providing more balanced estimates of the ocean‐atmosphere system and allowing for a broader range of studies of the entire Earth system. Results suggest that similar methods could also be leveraged to improve modern weather forecasts.

There has been a recent surge in development of accurate machine learning (ML) weather prediction models, but evaluation of these models has mainly been focused on medium‐range forecasts, not their performance in cycling data assimilation (DA) systems. Cycling DA provides a statistically optimal estimate of the system state, which can then be used as initial conditions for model prediction, given observations and previous model forecasts. Here, real surface pressure observations are assimilated into several popular ML models using an ensemble Kalman filter, where accurate ensemble covariance estimation is essential to constrain unobserved state variables from sparse observations. In this cycling DA system, deterministic ML models accumulate small‐scale noise until they diverge. Mitigating this noise with a spectral filter can stabilize the system, but with larger errors than traditional models. Perturbation experiments illustrate that these models do not accurately represent short‐term error growth, leading to poor estimation of cross‐variable covariances.

Particle filtering methods for data assimilation may suffer from the “curse of dimensionality,” where the required ensemble size grows rapidly as the dimension increases. It would, therefore, be useful to know a priori whether a particle filter is feasible to implement in a given system. Previous work provides an asymptotic relation between the necessary ensemble size and an exponential function of , a statistic that depends on observation-space quantities and that is related to the system dimension when the number of observations is large; for linear, Gaussian systems, the statistic can be computed from eigenvalues of an appropriately normalized covariance matrix. Tests with a low-dimensional system show that these asymptotic results remain useful when the system is nonlinear, with either the standard or optimal proposal implementation of the particle filter. This study explores approximations to the covariance matrices that facilitate computation in high-dimensional systems, as well as different methods to estimate the accumulated system noise covariance for the optimal proposal. Since may be approximated using an ensemble from a simpler data assimilation scheme, such as the ensemble Kalman filter, the asymptotic relations thus allow an estimate of the ensemble size required for a particle filter before its implementation. Finally, the improved performance of particle filters with the optimal proposal, relative to those using the standard proposal, in the same low-dimensional system is demonstrated.