In public discussions of the quality of forecasts, attention typically focuses on the predictive performance in cases of extreme events. However, the restriction of conventional forecast evaluation methods to subsets of extreme observations has unexpected and undesired effects, and is bound to discredit skillful forecasts when the signal-to-noise ratio in the data generating process is low. Conditioning on outcomes is incompatible with the theoretical assumptions of established forecast evaluation methods, thereby confronting forecasters with what we refer to as the forecaster’s dilemma. For probabilistic forecasts, proper weighted scoring rules have been proposed as decision-theoretically justifiable alternatives for forecast evaluation with an emphasis on extreme events. Using theoretical arguments, simulation experiments and a real data study on probabilistic forecasts of U.S. inflation and gross domestic product (GDP) growth, we illustrate and discuss the forecaster’s dilemma along with potential remedies.

This article revisits the accuracy of inflation forecasting using activity and expectations variables. We apply Bayesian model averaging across different regression specifications selected from a set of potential predictors that includes lagged values of inflation, a host of real activity data, term structure data, (relative) price data, and surveys. In this model average, we can entertain different channels of structural instability, by either incorporating stochastic breaks in the regression parameters of each individual specification within this average, or allowing for breaks in the error variance of the overall model average, or both. Thus, our framework simultaneously addresses structural change and model uncertainty that would unavoidably affect any inflation forecast model. The different versions of our framework are used to model U.S. personal consumption expenditures (PCE) deflator and gross domestic product (GDP) deflator inflation rates for the 1960–2011 period. A real-time inflation forecast evaluation shows that averaging over many predictors in a model that at least allows for structural breaks in the error variance results in very accurate point and density forecasts, especially for the post-1984 period. Our framework is especially useful when forecasting, in real-time, the likelihood of lower-than-usual inflation rates over the medium term. This article has online supplementary materials.