Testing competing precipitation forecasts accurately and efficiently: The spatial prediction comparison test

Which model is best? Many challenges exist when testing competing forecast models, especially for those with high spatial resolution. Spatial correlation, double penalties, and small-scale errors are just a few such challenges. Many new methods have been developed in recent decades to tackle these issues. The spatial prediction comparison test (SPCT), which was developed for general spatial fields and applied to wind speed, is applied here to precipitation fields; which pose many unique challenges in that they are not normally distributed, are marked by numerous zero-valued grid points, and verification results are particularly sensitive to small-scale errors and double penalties. The SPCT yields a statistical test that solves one important issue for verifying forecasts spatially by accounting for spatial correlation. Important for precipitation forecasts is that the test requires no distributional assumptions, is easy to perform, and can be applied efficiently to either gridded or nongridded spatial fields. The test compares loss functions between two competing forecasts, where any such function can be used, but most still suffer from the limitations of traditional gridpoint-by-gridpoint assessment techniques. Therefore, two new loss functions to the SPCT are introduced here that address these concerns. The first is based on distance maps and the second on image warping. Results are consistent with other spatial assessment methods, but provide a relatively straightforward mechanism for comparing forecasts with a statistically powerful test. The SPCT combined with these loss functions provides a new mechanism for appropriately testing which of two competing precipitation models is best, and whether the result is statistically significant or not.

To Access Resource:

Questions? Email Resource Support Contact:

  • opensky@ucar.edu
    UCAR/NCAR - Library

Resource Type publication
Temporal Range Begin N/A
Temporal Range End N/A
Temporal Resolution N/A
Bounding Box North Lat N/A
Bounding Box South Lat N/A
Bounding Box West Long N/A
Bounding Box East Long N/A
Spatial Representation N/A
Spatial Resolution N/A
Related Links N/A
Additional Information N/A
Resource Format PDF
Standardized Resource Format PDF
Asset Size N/A
Legal Constraints

Copyright 2013 American Meteorological Society (AMS). Permission to use figures, tables, and brief excerpts from this work in scientific and educational works is hereby granted provided that the source is acknowledged. Any use of material in this work that is determined to be "fair use" under Section 107 or that satisfies the conditions specified in Section 108 of the U.S. Copyright Law (17 USC, as revised by P.L. 94-553) does not require the Society's permission. Republication, systematic reproduction, posting in electronic form on servers, or other uses of this material, except as exempted by the above statements, requires written permission or license from the AMS. Additional details are provided in the AMS Copyright Policies, available from the AMS at 617-227-2425 or amspubs@ametsoc.org. Permission to place a copy of this work on this server has been provided by the AMS. The AMS does not guarantee that the copy provided here is an accurate copy of the published work.


Access Constraints None
Software Implementation Language N/A

Resource Support Name N/A
Resource Support Email opensky@ucar.edu
Resource Support Organization UCAR/NCAR - Library
Distributor N/A
Metadata Contact Name N/A
Metadata Contact Email opensky@ucar.edu
Metadata Contact Organization UCAR/NCAR - Library

Author Gilleland, Eric
Publisher UCAR/NCAR - Library
Publication Date 2013-01-01T00:00:00
Digital Object Identifier (DOI) Not Assigned
Alternate Identifier N/A
Resource Version N/A
Topic Category geoscientificInformation
Progress N/A
Metadata Date 2023-08-18T18:48:51.383340
Metadata Record Identifier edu.ucar.opensky::articles:12409
Metadata Language eng; USA
Suggested Citation Gilleland, Eric. (2013). Testing competing precipitation forecasts accurately and efficiently: The spatial prediction comparison test. UCAR/NCAR - Library. http://n2t.net/ark:/85065/d7w66mj5. Accessed 29 June 2025.

Harvest Source