IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-03216630.html
   My bibliography  Save this paper

Evaluating process-based integrated assessment models of climate change mitigation

Author

Listed:
  • Charlie Wilson
  • Céline Guivarch

    (CIRED - Centre International de Recherche sur l'Environnement et le Développement - Cirad - Centre de Coopération Internationale en Recherche Agronomique pour le Développement - EHESS - École des hautes études en sciences sociales - AgroParisTech - ENPC - École des Ponts ParisTech - Université Paris-Saclay - CNRS - Centre National de la Recherche Scientifique)

  • Elmar Kriegler
  • Bas van Ruijven
  • Detlef van Vuuren
  • Volker Krey
  • Valeria Jana Schwanitz
  • Erica Thompson

Abstract

Process-based integrated assessment models (IAMs) project long-term transformation pathways in energy and land-use systems under what-if assumptions. IAM evaluation is necessary to improve the models' usefulness as scientific tools applicable in the complex and contested domain of climate change mitigation. We contribute the first comprehensive synthesis of process-based IAM evaluation research, drawing on a wide range of examples across six different evaluation methods including historical simulations, stylised facts, and model diagnostics. For each evaluation method, we identify progress and milestones to date, and draw out lessons learnt as well as challenges remaining. We find that each evaluation method has distinctive strengths, as well as constraints on its application. We use these insights to propose a systematic evaluation framework combining multiple methods to establish the appropriateness, interpretability, credibility, and relevance of process-based IAMs as useful scientific tools for informing climate policy. We also set out a programme of evaluation research to be mainstreamed both within and outside the IAM community.

Suggested Citation

  • Charlie Wilson & Céline Guivarch & Elmar Kriegler & Bas van Ruijven & Detlef van Vuuren & Volker Krey & Valeria Jana Schwanitz & Erica Thompson, 2021. "Evaluating process-based integrated assessment models of climate change mitigation," Post-Print hal-03216630, HAL.
  • Handle: RePEc:hal:journl:hal-03216630
    DOI: 10.1007/s10584-021-03099-9
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Mathijs Harmsen & Detlef Vuuren & Maarten Berg & Andries Hof & Chris Hope & Volker Krey & Jean-Francois Lamarque & Adriana Marcucci & Drew Shindell & Michiel Schaeffer, 2015. "How well do integrated assessment models represent non-CO 2 radiative forcing?," Climatic Change, Springer, vol. 133(4), pages 565-582, December.
    2. Rosen, Richard A. & Guenther, Edeltraud, 2015. "The economics of mitigating climate change: What can we know?," Technological Forecasting and Social Change, Elsevier, vol. 91(C), pages 93-106.
    3. Fujimori, Shinichiro & Dai, Hancheng & Masui, Toshihiko & Matsuoka, Yuzuru, 2016. "Global energy model hindcasting," Energy, Elsevier, vol. 114(C), pages 293-301.
    4. L. Drouet & V. Bosetti & M. Tavoni, 2015. "Selection of climate policies under the uncertainties in the Fifth Assessment Report of the IPCC," Nature Climate Change, Nature, vol. 5(10), pages 937-940, October.
    5. Iyer, Gokul & Hultman, Nathan & Eom, Jiyong & McJeon, Haewon & Patel, Pralit & Clarke, Leon, 2015. "Diffusion of low-carbon technologies and the feasibility of long-term climate targets," Technological Forecasting and Social Change, Elsevier, vol. 90(PA), pages 103-118.
    6. Chaturvedi, Vaibhav & Kim, Sonny & Smith, Steven J. & Clarke, Leon & Yuyu, Zhou & Kyle, Page & Patel, Pralit, 2013. "Model evaluation and hindcasting: An experiment with an integrated assessment model," Energy, Elsevier, vol. 61(C), pages 479-490.
    7. Detlef Vuuren & Jason Lowe & Elke Stehfest & Laila Gohar & Andries Hof & Chris Hope & Rachel Warren & Malte Meinshausen & Gian-Kasper Plattner, 2011. "How well do integrated assessment models simulate climate change?," Climatic Change, Springer, vol. 104(2), pages 255-285, January.
    8. Jonathan Koomey & Paul Craig & Ashok Gadgil & David Lorenzetti, 2003. "Improving Long-Range Energy Modeling: A Plea for Historical Retrospectives," The Energy Journal, International Association for Energy Economics, vol. 0(Number 4), pages 75-92.
    9. Wilkerson, Jordan T. & Leibowicz, Benjamin D. & Turner, Delavane D. & Weyant, John P., 2015. "Comparison of integrated assessment models: Carbon price impacts on U.S. energy," Energy Policy, Elsevier, vol. 76(C), pages 18-31.
    10. Schwanitz, Valeria Jana & Wierling, August, 2016. "Offshore wind investments – Realism about cost developments is necessary," Energy, Elsevier, vol. 106(C), pages 170-181.
    11. Trutnevyte, Evelina, 2016. "Does cost optimization approximate the real-world energy transition?," Energy, Elsevier, vol. 106(C), pages 182-193.
    12. Kriegler, Elmar & Petermann, Nils & Krey, Volker & Schwanitz, Valeria Jana & Luderer, Gunnar & Ashina, Shuichi & Bosetti, Valentina & Eom, Jiyong & Kitous, Alban & Méjean, Aurélie & Paroussos, Leonida, 2015. "Diagnostic indicators for integrated assessment models of climate policy," Technological Forecasting and Social Change, Elsevier, vol. 90(PA), pages 45-61.
    13. Edmonds, Jae & Reilly, John, 1983. "A long-term global energy- economic model of carbon dioxide release from fossil fuel use," Energy Economics, Elsevier, vol. 5(2), pages 74-88, April.
    14. McJeon, Haewon C. & Clarke, Leon & Kyle, Page & Wise, Marshall & Hackbarth, Andrew & Bryant, Benjamin P. & Lempert, Robert J., 2011. "Technology interactions among low-carbon energy technologies: What can we learn from a large number of scenarios?," Energy Economics, Elsevier, vol. 33(4), pages 619-631, July.
    15. David Anthoff & Richard Tol, 2013. "Erratum to: The uncertainty about the social cost of carbon: A decomposition analysis using fund," Climatic Change, Springer, vol. 121(2), pages 413-413, November.
    16. Chris Hope & Mat Hope, 2013. "The social cost of CO2 in a low-growth world," Nature Climate Change, Nature, vol. 3(8), pages 722-724, August.
    17. DeCarolis, Joseph & Daly, Hannah & Dodds, Paul & Keppo, Ilkka & Li, Francis & McDowall, Will & Pye, Steve & Strachan, Neil & Trutnevyte, Evelina & Usher, Will & Winning, Matthew & Yeh, Sonia & Zeyring, 2017. "Formalizing best practice for energy system optimization modelling," Applied Energy, Elsevier, vol. 194(C), pages 184-198.
    18. Zoi Vrontisi & Gunnar Luderer & Bert Saveyn & Kimon Keramidas & Lara Aleluia Reis & Lavinia Baumstark & Christoph Bertram & Harmen Sytze de Boer & Laurent Drouet & Kostas Fragkiadakis & Oliver Fricko , 2018. "Enhancing global climate policy ambition towards a 1.5 °C stabilization: a short-term multi-model assessment," Post-Print halshs-01782274, HAL.
    19. Guivarch, Céline & Hallegatte, Stéphane & Crassous, Renaud, 2009. "The resilience of the Indian economy to rising oil prices as a validation test for a global energy-environment-economy CGE model," Energy Policy, Elsevier, vol. 37(11), pages 4259-4266, November.
    20. Gaskins, Darius W, Jr & Weyant, John P, 1993. "Model Comparisons of the Costs of Reducing CO2 Emissions," American Economic Review, American Economic Association, vol. 83(2), pages 318-323, May.
    21. Sibel Eker & Elena Rovenskaya & Michael Obersteiner & Simon Langan, 2018. "Practice and perspectives in the validation of resource management models," Nature Communications, Nature, vol. 9(1), pages 1-10, December.
    22. Henri Waisman & Céline Guivarch & Fabio Grazi & Jean Hourcade, 2012. "The I maclim-R model: infrastructures, technical inertia and the costs of low carbon futures under imperfect foresight," Climatic Change, Springer, vol. 114(1), pages 101-120, September.
    23. DeCarolis, Joseph F. & Hunter, Kevin & Sreepathi, Sarat, 2012. "The case for repeatable analysis with energy economy optimization models," Energy Economics, Elsevier, vol. 34(6), pages 1845-1853.
    24. Richard Rosen & Edeltraud Guenther, 2014. "The Economics of Mitigating Climate Change?," Challenge, Taylor & Francis Journals, vol. 57(4), pages 57-81.
    25. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 13, pages 1-17.
    26. Pye, Steve & Sabio, Nagore & Strachan, Neil, 2015. "An integrated systematic analysis of uncertainties in UK energy transition pathways," Energy Policy, Elsevier, vol. 87(C), pages 673-684.
    27. Michael Greenstone & Elizabeth Kopits & Ann Wolverton, 2013. "Developing a Social Cost of Carbon for US Regulatory Analysis: A Methodology and Interpretation," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 7(1), pages 23-46, January.
    28. Huntington, Hillard G & Weyant, John P & Sweeney, James L, 1982. "Modeling for insights, not numbers: the experiences of the energy modeling forum," Omega, Elsevier, vol. 10(5), pages 449-462.
    29. Nicholas Stern, 2016. "Economics: Current climate models are grossly misleading," Nature, Nature, vol. 530(7591), pages 407-409, February.
    30. Maurizio Gargiulo & Brian Ó Gallachóir, 2013. "Long-term energy models: Principles, characteristics, focus, and limitations," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 2(2), pages 158-177, March.
    31. David Anthoff & Richard Tol, 2013. "The uncertainty about the social cost of carbon: A decomposition analysis using fund," Climatic Change, Springer, vol. 117(3), pages 515-530, April.
    32. van Ruijven, Bas & Urban, Frauke & Benders, René M.J. & Moll, Henri C. & van der Sluijs, Jeroen P. & de Vries, Bert & van Vuuren, Detlef P., 2008. "Modeling Energy and Development: An Evaluation of Models and Concepts," World Development, Elsevier, vol. 36(12), pages 2801-2821, December.
    33. Frances C. Moore & Delavane B. Diaz, 2015. "Temperature impacts on economic growth warrant stringent mitigation policy," Nature Climate Change, Nature, vol. 5(2), pages 127-131, February.
    34. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," LSE Research Online Documents on Economics 103310, London School of Economics and Political Science, LSE Library.
    35. Nebojsa Nakicenovic & Robert Lempert & Anthony Janetos, 2014. "A Framework for the Development of New Socio-economic Scenarios for Climate Change Research: Introductory Essay," Climatic Change, Springer, vol. 122(3), pages 351-361, February.
    36. Pilavachi, P.A. & Dalamaga, Th. & Rossetti di Valdalbero, D. & Guilmot, J.-F., 2008. "Ex-post evaluation of European energy models," Energy Policy, Elsevier, vol. 36(5), pages 1726-1735, May.
    37. Rosen, Richard A. & Guenther, Edeltraud, 2016. "The energy policy relevance of the 2014 IPCC Working Group III report on the macro-economics of mitigating climate change," Energy Policy, Elsevier, vol. 93(C), pages 330-334.
    38. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," Economics Discussion Papers 2019-23, Kiel Institute for the World Economy (IfW Kiel).
    39. Frances C. Moore & Delavane B. Diaz, 2015. "Erratum: Temperature impacts on economic growth warrant stringent mitigation policy," Nature Climate Change, Nature, vol. 5(3), pages 280-280, March.
    40. Alan Manne & Richard Richels, 1992. "Buying Greenhouse Insurance: The Economic Costs of CO2 Emission Limits," MIT Press Books, The MIT Press, edition 1, volume 1, number 026213280x, April.
    41. Felix Creutzig & Peter Agoston & Jan Christoph Goldschmidt & Gunnar Luderer & Gregory Nemet & Robert C. Pietzcker, 2017. "The underestimated potential of solar energy to mitigate climate change," Nature Energy, Nature, vol. 2(9), pages 1-9, September.
    42. van Ruijven, Bas J. & van Vuuren, Detlef P. & de Vries, Bert J.M. & Isaac, Morna & van der Sluijs, Jeroen P. & Lucas, Paul L. & Balachandra, P., 2011. "Model projections for household energy use in India," Energy Policy, Elsevier, vol. 39(12), pages 7747-7761.
    43. Borgonovo, E., 2010. "Sensitivity analysis with finite changes: An application to modified EOQ models," European Journal of Operational Research, Elsevier, vol. 200(1), pages 127-138, January.
    44. Michael Oppenheimer & Christopher M. Little & Roger M. Cooke, 2016. "Expert judgement and uncertainty quantification for climate change," Nature Climate Change, Nature, vol. 6(5), pages 445-451, May.
    45. Roger M. Cooke, 2015. "Messaging climate change uncertainty," Nature Climate Change, Nature, vol. 5(1), pages 8-10, January.
    46. Marisa Beck & Tobias Krueger, 2016. "The epistemic, ethical, and political dimensions of uncertainty in integrated assessment modeling," Wiley Interdisciplinary Reviews: Climate Change, John Wiley & Sons, vol. 7(5), pages 627-645, September.
    47. John Weyant, 2017. "Some Contributions of Integrated Assessment Models of Global Climate Change," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 11(1), pages 115-137.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ken Oshiro & Shinichiro Fujimori, 2024. "Limited impact of hydrogen co-firing on prolonging fossil-based power generation under low emissions scenarios," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    2. Chang, Miguel & Lund, Henrik & Thellufsen, Jakob Zinck & Østergaard, Poul Alberg, 2023. "Perspectives on purpose-driven coupling of energy system models," Energy, Elsevier, vol. 265(C).
    3. da Silva Neves, Marcus Vinicius & Szklo, Alexandre & Schaeffer, Roberto, 2023. "Fossil fuel facilities exergy return for a frontier of analysis incorporating CO2 capture: The case of a coal power plant," Energy, Elsevier, vol. 284(C).
    4. Rising, James A. & Taylor, Charlotte & Ives, Matthew C. & Ward, Robert E.T., 2022. "Challenges and innovations in the economic evaluation of the risks of climate change," Ecological Economics, Elsevier, vol. 197(C).
    5. Clare Hanmer & Charlie Wilson & Oreane Y. Edelenbosch & Detlef P. van Vuuren, 2022. "Translating Global Integrated Assessment Model Output into Lifestyle Change Pathways at the Country and Household Level," Energies, MDPI, vol. 15(5), pages 1-31, February.
    6. Heinisch, Verena & Dujardin, Jérôme & Gabrielli, Paolo & Jain, Pranjal & Lehning, Michael & Sansavini, Giovanni & Sasse, Jan-Philipp & Schaffner, Christian & Schwarz, Marius & Trutnevyte, Evelina, 2023. "Inter-comparison of spatial models for high shares of renewable electricity in Switzerland," Applied Energy, Elsevier, vol. 350(C).
    7. Rising, James A. & Taylor, Charlotte & Ives, Matthew C. & Ward, Robert E.t., 2022. "Challenges and innovations in the economic evaluation of the risks of climate change," LSE Research Online Documents on Economics 114941, London School of Economics and Political Science, LSE Library.
    8. Mohamed Elsayed Gabr & Amira Mahmoud El Shorbagy & Hamdy Badee Faheem, 2022. "Utilizing the Harvesting of Rainwater to Provide Safe Road Transportation Efficiency and Increase Water Resources in the Context of Climatic Change," Sustainability, MDPI, vol. 14(15), pages 1-20, August.
    9. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2023. "Hindcasting to inform the development of bottom-up electricity system models: The cases of endogenous demand and technology learning," Applied Energy, Elsevier, vol. 340(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ajay Gambhir & Isabela Butnar & Pei-Hao Li & Pete Smith & Neil Strachan, 2019. "A Review of Criticisms of Integrated Assessment Models and Proposed Approaches to Address These, through the Lens of BECCS," Energies, MDPI, vol. 12(9), pages 1-21, May.
    2. Huntington, Hillard G., 2021. "Model evaluation for policy insights: Reflections on the forum process," Energy Policy, Elsevier, vol. 156(C).
    3. DeCarolis, Joseph & Daly, Hannah & Dodds, Paul & Keppo, Ilkka & Li, Francis & McDowall, Will & Pye, Steve & Strachan, Neil & Trutnevyte, Evelina & Usher, Will & Winning, Matthew & Yeh, Sonia & Zeyring, 2017. "Formalizing best practice for energy system optimization modelling," Applied Energy, Elsevier, vol. 194(C), pages 184-198.
    4. Richard S.J. Tol, 2021. "Estimates of the social cost of carbon have not changed over time," Working Paper Series 0821, Department of Economics, University of Sussex Business School.
    5. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2022. "Accuracy indicators for evaluating retrospective performance of energy system models," Applied Energy, Elsevier, vol. 325(C).
    6. J. Farmer & Cameron Hepburn & Penny Mealy & Alexander Teytelboym, 2015. "A Third Wave in the Economics of Climate Change," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 62(2), pages 329-357, October.
    7. Richard S. J. Tol, 2021. "Estimates of the social cost of carbon have increased over time," Papers 2105.03656, arXiv.org, revised Aug 2022.
    8. Trutnevyte, Evelina, 2016. "Does cost optimization approximate the real-world energy transition?," Energy, Elsevier, vol. 106(C), pages 182-193.
    9. Richard S J Tol, 2018. "The Economic Impacts of Climate Change," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 4-25.
    10. Tiruwork B. Tibebu & Eric Hittinger & Qing Miao & Eric Williams, 2024. "Adoption Model Choice Affects the Optimal Subsidy for Residential Solar," Energies, MDPI, vol. 17(3), pages 1-19, February.
    11. Kenneth Gillingham & William D. Nordhaus & David Anthoff & Geoffrey Blanford & Valentina Bosetti & Peter Christensen & Haewon McJeon & John Reilly & Paul Sztorc, 2015. "Modeling Uncertainty in Climate Change: A Multi-Model Comparison," NBER Working Papers 21637, National Bureau of Economic Research, Inc.
    12. Nikas, A. & Gambhir, A. & Trutnevyte, E. & Koasidis, K. & Lund, H. & Thellufsen, J.Z. & Mayer, D. & Zachmann, G. & Miguel, L.J. & Ferreras-Alonso, N. & Sognnaes, I. & Peters, G.P. & Colombo, E. & Howe, 2021. "Perspective of comprehensive and comprehensible multi-model energy and climate science in Europe," Energy, Elsevier, vol. 215(PA).
    13. Alexander R. Barron & Allen A. Fawcett & Marc A. C. Hafstead & James R. Mcfarland & Adele C. Morris, 2018. "Policy Insights From The Emf 32 Study On U.S. Carbon Tax Scenarios," Climate Change Economics (CCE), World Scientific Publishing Co. Pte. Ltd., vol. 9(01), pages 1-47, February.
    14. Rising, James A. & Taylor, Charlotte & Ives, Matthew C. & Ward, Robert E.T., 2022. "Challenges and innovations in the economic evaluation of the risks of climate change," Ecological Economics, Elsevier, vol. 197(C).
    15. Schultes, Anselm & Piontek, Franziska & Soergel, Bjoern & Rogelj, Joeri & Baumstark, Lavinia & Kriegler, Elmar & Edenhofer, Ottmar & Luderer, Gunnar, 2020. "Economic damages from on-going climate change imply deeper near-term emission cuts," MPRA Paper 103655, University Library of Munich, Germany.
    16. Shumilov, Andrei, 2021. "Анализ Неопределенности В Интегрированных Моделях Климата И Экономики: Обзор Литературы [Uncertainty analysis in integrated assessment models of the economics of climate change: a literature survey," MPRA Paper 110171, University Library of Munich, Germany.
    17. Rong Wang & Juan Moreno-Cruz & Ken Caldeira, 2017. "Will the use of a carbon tax for revenue generation produce an incentive to continue carbon emissions?," Post-Print hal-03226925, HAL.
    18. Price, James & Keppo, Ilkka, 2017. "Modelling to generate alternatives: A technique to explore uncertainty in energy-environment-economy models," Applied Energy, Elsevier, vol. 195(C), pages 356-369.
    19. Yongyang Cai, 2020. "The Role of Uncertainty in Controlling Climate Change," Papers 2003.01615, arXiv.org, revised Oct 2020.
    20. Milad Eghtedari Naeini & Benjamin D. Leibowicz & J. Eric Bickel, 2020. "Can you trust a model whose output keeps changing? Interpreting changes in the social cost of carbon produced by the DICE model," Environment Systems and Decisions, Springer, vol. 40(3), pages 301-320, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-03216630. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.