Feature-based diagnostic evaluation of global NWP forecasts

With the resolution of global numerical weather prediction (NWP) models now typically between 10 and 20 km, forecasts are able to capture the evolution of synoptic features that are important drivers for significant surface weather. The position, timing, and intensity of jet cores, surface highs and lows, and changes in the behavior of these forecast features is explored using the Method for Object-based Diagnostic Evaluation (MODE) at the global scale. Previously this was only possible with a more subjective approach. The spatial aspects of the forecast features (objects) and their intensity can be assessed separately. The evolution of paired forecast analysis object attributes such as location and orientation differences, as well as area ratios, can be considered. The differences in the paired object attribute distributions from various model configurations were evaluated using the k-sample Anderson Darling (AD) test. Increases or decreases in hits, false alarms (forecast-not-observed), and misses (observed-not-forecast) features were also assessed. It was found that when focusing purely on the forecast features of interest, differences in seasonal spatial extent biases emerged, intensity biases varied as a function of analysis time, and changes in the attribute distributions could be detected but were largely insignificant, primarily due to sample size. As has been shown for kilometer-scale NWP, results from spatial verification methods are more in line with subjective assessment. This type of objective assessment provides a new dimension to the traditional assessment of global NWP, and provides output that is closer to the way in which forecasts are used.