edu.ucar.opensky::articles:43262 UCAR/NCAR - Library OpenSky Support PO Box 3000 Boulder CO 80307-3000 opensky@ucar.edu http://opensky.ucar.edu/ homepage pointOfContact 2025-12-24T17:54:18.621865 ISO 19115-2 Geographic Information - Metadata - Part 2: Extensions for Imagery and Gridded Data ISO 19115-2:2009(E) https://n2t.net/ark:/85065/d76q22nb An assessment of how domain experts evaluate Machine Learning in operational meteorology 2025-03-01T00:00:00Z publication https://n2t.net/ark:/85065/d76q22nb https://doi.org/10.1175/WAF-D-24-0144.1 Harrison, D. author McGovern, A. author Karstens, C. D. author Bostrom, A. author Demuth, Julie L. author Jirak, I. L. author Marsh, P. T. author UCAR/NCAR - Library publisher As an increasing number of machine learning (ML) products enter the research-to-operations (R2O) pipeline, researchers have anecdotally noted a perceived hesitancy by operational forecasters to adopt this relatively new technology. One explanation often cited in the literature is that this perceived hesitancy derives from the complex and opaque nature of ML methods. Because modern ML models are trained to solve tasks by optimizing a potentially complex combination of mathematical weights, thresholds, and nonlinear cost functions, it can be difficult to determine how these models reach a solution from their given input. However, it remains unclear to what degree a model&#8217;s transparency may influence a forecaster&#8217;s decision to use that model or if that impact differs between ML and more traditional (i.e., non-ML) methods. To address this question, a survey was offered to forecaster and researcher participants attending the 2021 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE) with questions about how participants subjectively perceive and compare machine learning products to more traditionally derived products. Results from this study revealed few differences in how participants evaluated machine learning products compared to other types of guidance. However, comparing the responses between operational forecasters, researchers, and academics exposed notable differences in what factors the three groups considered to be most important for determining the operational success of a new forecast product. These results support the need for increased collaboration between the operational and research communities. UCAR/NCAR - Library OpenSky Support PO Box 3000 Boulder CO 80307-3000 opensky@ucar.edu http://opensky.ucar.edu/ homepage pointOfContact PDF Text theme Resource Type 2016-01-01T00:00:00Z publication <span style="font-family:Arial;font-size:10pt;font-style:normal;font-weight:normal;" data-sheets-root="1">Copyright 2025 American Meteorological Society (AMS).</span> None eng geoscientificInformation