The Fractions Skill Score (FSS) is a spatial verification measure that is used for assessing the performance of precipitation forecasts from numerical weather prediction models. Previous studies have shown that the FSS is able to give a direct measure of the error in the placement of the rain. This article takes the approach further and derives analytical expressions and uses Monte‐Carlo simulations for randomly positioned observed and forecast rainfall to reveal further characteristics of the FSS in both infinite and bounded domains. It reveals that the definition of an FSS value that determines the minimum scale at which a forecast should be deemed ‘useful’ (useful forecast criteria) is a meaningful concept and shows how this value increases with increasing fractional rainfall coverage. A study of real forecast data is also presented using 8 years of European Centre for Medium‐Range Weather Forecasts (ECMWF) model forecasts, out to a lead time of 9 days, over domains of differing sizes covering parts of Europe and North Africa. The FSS is examined using different strategies for dealing with the domain boundary and is compared with the analytical study. The findings give practical guidance on how to use the FSS. For most situations a FSS value of >0.5 serves as a good indicator of a useful forecast. The choice of domain size for rainfall forecast verification should consider the typical spatial errors of the forecast. For a domain that is large compared to the typical spatial error, the boundaries have little adverse affect, but this is not the case if the spatial errors start to become comparable to the size of the domain. The evaluation of ECMWF forecasts reveals the extent of the spatial errors that emerge for medium‐range forecasts and show the value of verifying those forecasts using the FSS over an appropriately sized region.
A recently developed object identification algorithm is applied to multisensor precipitation estimates from the Tropical Rainfall Measuring Mission (TRMM 3B42) to detect and quantify the contribution of tropical cyclone precipitation (TCP) to total precipitation between 1998 and 2008. The study period includes 1144 storms. Estimates of TCP derived here are similar in pattern and seasonal variation to earlier estimates but are somewhat higher in magnitude. Annual-mean TCP fractions of over 20% are diagnosed over large swaths of tropical ocean, with seasonal means in some regions of more than 50%. Interannual variability of TCP is examined, and a small but significant downward trend in global TCP from 1998 to 2008 is found, consistent with results from independent studies examining accumulated cyclone energy (ACE). Relationships between annual-mean ACE and TCP in each major tropical cyclone basin are examined. High correlations are found in almost every basin, although different linear relationships exist in each. The highest ACE/TCP ratios are obtained in the North Atlantic and northeast Pacific basins, with lower ratios present in the northwest Pacific and South Pacific basins.
4 January 2015 in Theoretical and Applied Climatology
The Fractions Skill Score (FSS) is a popular spatial verification metric commonly used for precipitation verification. In this study we focus on analysing the ability of FSS to provide meaningful information about the displacement between precipitation in one field compared to another. A simple overlap‐adjusted use of the FSS is introduced and a number of relevant idealized cases are analysed that show that the FSS can indeed be used to determine displacement in a meaningful way. It was found that the displacement provided by the FSS is directly related to the true displacements of precipitation but with larger contiguous precipitation objects having a much larger influence. Overall, the displacement provided via the FSS compares well with the average distance to the closest neighbouring precipitation object (assuming the objects are of similar size). It is recommended that the user should use a frequency (percentile) threshold when focussing on spatial differences unless biases are known to be small and adopt the overlap‐adjusted variant of the FSS displacement. If the frequency bias is very large the FSS‐derived displacements become less reliable. The same is true of any spatial comparison. A recipe for the use of the FSS for determining displacements is provided.
A compact analytical expression of the Fraction Skill Score (FSS) is derived for a case with a single displaced rainband in a rectangular domain. The rainband is oriented parallel to the border and displaced perpendicularly to its orientation. An analytical solution is used to determine some of the properties of the FSS which might also be applicable in other cases. The solution is independent of the length of the rainband (it is valid also for a displaced rainy grid point). The position of the borders perpendicular to the rainband orientation does not influence the FSS value. The position of the other two borders does however influence the FSS value in a complex way; moving a border closer to the rainbands can either increase or decrease the FSS value depending on the location of the borders. FSS is shown to be a monotonically increasing function of the neighbourhood size (regardless of the position of the borders). If the FSS value for a displacement that is half the neighbourhood size is used to define a ‘useful’ FSS value then the usefulness criterion is somewhat different than presented in the original FSS paper (there is no dependence on the frequency of the observations/forecasts). ‘Useful’ FSS values are always >1/2 but depend on the position of the borders and the size of the displacement.