IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v176y2023i8d10.1007_s10584-023-03595-0.html
   My bibliography  Save this article

Assessing climate service products with evaluation metrics: an application to decision support tools for climate change adaptation in the USA

Author

Listed:
  • Momtaz Jahan

    (Virginia Tech)

  • Julia Reis

    (Virginia Tech)

  • Julie Shortridge

    (Virginia Tech)

Abstract

As the field of climate services advances, there is increasing recognition of the need for evaluation of climate services. Existing research largely includes detailed evaluations and discussions of a single or small number of services or broad-scale sampling activities that record basic information on a very large number of services. In this work, we propose an intermediate approach to evaluating existing climate services that includes evaluation metrics with definitions for high, medium, and low performance based on a previously proposed evaluation framework. The evaluation metrics characterize the type of information provided by a climate service, the manner in which it is communicated, and the degree to which the tool’s development background is transparently presented. We use the proposed metrics to evaluate a sample of 20 US-based online decision support tools aimed at supporting climate change adaptation. We find that most tools are quite thorough in their presentation of detailed climate information and inclusion of explanatory material. However, few tools provide much discussion of the specific decisions that the tool aims to support or the role that climate information can play in these decisions. This work contributes an approach for intermediate-scale climate service evaluation that provides a middle ground between detailed case studies and broad-scale sampling. This can complement existing evaluation research and also highlight potential issues and gaps in climate service research and evaluation.

Suggested Citation

  • Momtaz Jahan & Julia Reis & Julie Shortridge, 2023. "Assessing climate service products with evaluation metrics: an application to decision support tools for climate change adaptation in the USA," Climatic Change, Springer, vol. 176(8), pages 1-25, August.
    • Handle: RePEc:spr:climat:v:176:y:2023:i:8:d:10.1007_s10584-023-03595-0
    DOI: 10.1007/s10584-023-03595-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-023-03595-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10584-023-03595-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marta Bruno Soares & Carlo Buontempo, 2019. "Challenges to the sustainability of climate services in Europe," Wiley Interdisciplinary Reviews: Climate Change, John Wiley & Sons, vol. 10(4), July.
    2. Catherine Vaughan & Suraje Dessai, 2014. "Climate services for society: origins, institutional arrangements, and design elements for an evaluation framework," Wiley Interdisciplinary Reviews: Climate Change, John Wiley & Sons, vol. 5(5), pages 587-603, September.
    3. Chris Hewitt & Simon Mason & David Walland, 2012. "The Global Framework for Climate Services," Nature Climate Change, Nature, vol. 2(12), pages 831-832, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kirsten Halsnæs & Lisa Bay & Mads Lykke Dømgaard & Per Skougaard Kaspersen & Morten Andreas Dahl Larsen, 2020. "Accelerating Climate Service Development for Renewable Energy, Finance and Cities," Sustainability, MDPI, vol. 12(18), pages 1-18, September.
    2. Gigi Owen & Daniel B. Ferguson & Ben McMahan, 2019. "Contextualizing climate science: applying social learning systems theory to knowledge production, climate services, and use-inspired research," Climatic Change, Springer, vol. 157(1), pages 151-170, November.
    3. Blane Harvey & Lindsey Jones & Logan Cochrane & Roop Singh, 2019. "The evolving landscape of climate services in sub-Saharan Africa: What roles have NGOs played?," Climatic Change, Springer, vol. 157(1), pages 81-98, November.
    4. Elena Lioubimtseva & Charlotte da Cunha, 2023. "The Role of Non-Climate Data in Equitable Climate Adaptation Planning: Lessons from Small French and American Cities," Sustainability, MDPI, vol. 15(2), pages 1-21, January.
    5. Charles Herrick & Jason Vogel, 2022. "Climate Adaptation at the Local Scale: Using Federal Climate Adaptation Policy Regimes to Enhance Climate Services," Sustainability, MDPI, vol. 14(13), pages 1-20, July.
    6. Luke J. Harrington, 2017. "Investigating differences between event-as-class and probability density-based attribution statements with emerging climate change," Climatic Change, Springer, vol. 141(4), pages 641-654, April.
    7. Michel Jancloes & Madeleine Thomson & María Mánez Costa & Chris Hewitt & Carlos Corvalan & Tufa Dinku & Rachel Lowe & Mary Hayden, 2014. "Climate Services to Improve Public Health," IJERPH, MDPI, vol. 11(5), pages 1-5, April.
    8. Kevin Raaphorst & Gerben Koers & Gerald Jan Ellen & Amy Oen & Bjørn Kalsnes & Lisa van Well & Jana Koerth & Rutger van der Brugge, 2020. "Mind the Gap: Towards a Typology of Climate Service Usability Gaps," Sustainability, MDPI, vol. 12(4), pages 1-21, February.
    9. Dragana Bojovic & Andria Nicodemou & Asun Lera St.Clair & Isadora Christel & Francisco J. Doblas-Reyes, 2022. "Exploring the landscape of seasonal forecast provision by Global Producing Centres," Climatic Change, Springer, vol. 172(1), pages 1-23, May.
    10. Adama Ouedraogo & Irene S. Egyir & Mathieu Ouedraogo & John Baptist D. Jatoe, 2022. "Farmers’ Demand for Climate Information Services: A Systematic Review," Sustainability, MDPI, vol. 14(15), pages 1-18, July.
    11. Simona Meiler & Thomas Vogt & Nadia Bloemendaal & Alessio Ciullo & Chia-Ying Lee & Suzana J. Camargo & Kerry Emanuel & David N. Bresch, 2022. "Intercomparison of regional loss estimates from global synthetic tropical cyclone models," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    12. Maurice Skelton, 2020. "How cognitive links and decision-making capacity shape sectoral experts’ recognition of climate knowledge for adaptation," Climatic Change, Springer, vol. 162(3), pages 1535-1553, October.
    13. Clara Mosso & Diego Pons & Cristian Beza-Beza, 2022. "A Long Way toward Climate Smart Agriculture: The Importance of Addressing Gender Inequity in the Agricultural Sector of Guatemala," Land, MDPI, vol. 11(8), pages 1-16, August.
    14. Conor Murphy & Anthony Kettle & Hadush Meresa & Saeed Golian & Michael Bruen & Fiachra O’Loughlin & Per-Erik Mellander, 2023. "Climate Change Impacts on Irish River Flows: High Resolution Scenarios and Comparison with CORDEX and CMIP6 Ensembles," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 37(5), pages 1841-1858, March.
    15. Lledó, Ll. & Torralba, V. & Soret, A. & Ramon, J. & Doblas-Reyes, F.J., 2019. "Seasonal forecasts of wind power generation," Renewable Energy, Elsevier, vol. 143(C), pages 91-100.
    16. Sophie Haines, 2019. "Managing expectations: articulating expertise in climate services for agriculture in Belize," Climatic Change, Springer, vol. 157(1), pages 43-59, November.
    17. Sophie Webber, 2019. "Putting climate services in contexts: advancing multi-disciplinary understandings: introduction to the special issue," Climatic Change, Springer, vol. 157(1), pages 1-8, November.
    18. Arsum Pathak & Laura E. Hilberg & Lara J. Hansen & Bruce A. Stein, 2022. "Key Considerations for the Use of Nature-Based Solutions in Climate Services and Adaptation," Sustainability, MDPI, vol. 14(24), pages 1-12, December.
    19. Surminski, Swenja & Barnes, Jonathan & Vincent, Katharine, 2022. "Can insurance catalyse government planning on climate? Emergent evidence from Sub-Saharan Africa," World Development, Elsevier, vol. 153(C).
    20. Arjan Wardekker & Susanne Lorenz, 2019. "The visual framing of climate change impacts and adaptation in the IPCC assessment reports," Climatic Change, Springer, vol. 156(1), pages 273-292, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:176:y:2023:i:8:d:10.1007_s10584-023-03595-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.