A lot of controversy exists around the choice of the most appropriate error measure for assessing the performance of forecasting methods. While statisticians argue for the use of measures with good statistical properties, practitioners prefer measures that are easy to communicate and understand. Moreover, researchers argue that the loss-function for parameterizing a model should be aligned with how the post-performance measurement is made. In this paper we ask: Does it matter? Will the relative ranking of the forecasting methods change significantly if we choose one measure over another? Will a mismatch of the in-sample loss-function and the out-of-sample performance measure decrease the performance of the forecasting models? Focusing on the average ranked point forecast accuracy, we review the most commonly-used measures in both the academia and practice and perform a large-scale empirical study to understand the importance of the choice between measures. Our results suggest that there are only small discrepancies between the different error measures, especially within each measure category (percentage, relative, or scaled).

In alignment with the European Union’s legislation, Greece submitted its final 10-year National Energy and Climate Plan (NECP) in December 2019, setting more ambitious energy and climate targets than those originally proposed in the draft version of the document. Apart from higher penetration of renewable energy sources (RES), the final NECP projects also zero carbon use in power generation till 2030. Although decarbonization has long been regarded beneficial for economies that base their energy production on coal, as it is the case with Greece, the macroeconomic and societal ramifications of faster transitions to carbon-free economies remain highly unexplored. Under this context, in this paper, we soft-link energy models, namely Times-Greece and Primes, with a macroeconomic model, namely Global Trade Analysis Project (GTAP), to measure the effects of the final and draft NECPs on the Greek economy and evaluate the impact of higher decarbonization speeds. We find that the faster transition scenario displays both economic and societal merits, increasing Gross Domestic Product (GDP) and household income by about 1% and 7%, respectively.