IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v91y2015icp64-77.html
   My bibliography  Save this article

Evaluating simulation-derived scenarios for effective decision support

Author

Listed:
  • Parker, Andrew M.
  • Srinivasan, Sinduja V.
  • Lempert, Robert J.
  • Berry, Sandra H.

Abstract

Scenario planning traditionally relies on qualitative methods to choose its scenarios. Recently, quantitative decision support tools have also begun to facilitate such choices. This study uses behavioral experiments and structured decision-maker interviews to evaluate the results of “scenario discovery,” a quantitative method that defines scenarios as sets of future states of the world in which proposed policies fail to meet their goals. Statistical cluster-finding and principal component algorithms applied to large databases of computer simulation model results then help users to identify such scenarios. The two experiments examine the results of this process and demonstrate a user preference for increased accuracy and simplicity achieved through rotating the space of uncertain model input parameters, but primarily when the rotated parameters are conceptually similar. Interviews with experts suggest utility for both qualitatively- and quantitatively-derived scenarios. The former were easier to understand and had the most utility for scoping. The latter were perceived as containing more relevant information and having more utility for understanding tradeoffs and making choices among them. Overall, this study suggests the value of quantitative tools for facilitating scenario choice, while also highlighting the importance of formal evaluation in judging the utility of new methods for decision support.

Suggested Citation

  • Parker, Andrew M. & Srinivasan, Sinduja V. & Lempert, Robert J. & Berry, Sandra H., 2015. "Evaluating simulation-derived scenarios for effective decision support," Technological Forecasting and Social Change, Elsevier, vol. 91(C), pages 64-77.
    • Handle: RePEc:eee:tefoso:v:91:y:2015:i:c:p:64-77
    DOI: 10.1016/j.techfore.2014.01.010
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162514000419
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2014.01.010?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Julie Rozenberg & Céline Guivarch & Robert Lempert & Stéphane Hallegatte, 2014. "Building SSPs for climate policy analysis: a scenario elicitation methodology to map the space of possible future challenges to mitigation and adaptation," Climatic Change, Springer, vol. 122(3), pages 509-522, February.
    2. Robert J. Lempert & David G. Groves & Steven W. Popper & Steve C. Bankes, 2006. "A General, Analytic Method for Generating Robust Strategies and Narrative Scenarios," Management Science, INFORMS, vol. 52(4), pages 514-528, April.
    3. Kwakkel, Jan H. & Auping, Willem L. & Pruyt, Erik, 2013. "Dynamic scenario discovery under deep uncertainty: The future of copper," Technological Forecasting and Social Change, Elsevier, vol. 80(4), pages 789-800.
    4. McJeon, Haewon C. & Clarke, Leon & Kyle, Page & Wise, Marshall & Hackbarth, Andrew & Bryant, Benjamin P. & Lempert, Robert J., 2011. "Technology interactions among low-carbon energy technologies: What can we learn from a large number of scenarios?," Energy Economics, Elsevier, vol. 33(4), pages 619-631, July.
    5. Trutnevyte, Evelina & Stauffacher, Michael & Scholz, Roland W., 2012. "Linking stakeholder visions with resource allocation scenarios and multi-criteria assessment," European Journal of Operational Research, Elsevier, vol. 219(3), pages 762-772.
    6. Cira,Dean A. & Kalra,Nidhi Rajiv & Lempert,Robert J. & Lotsch,Alexander & Mao, Zhimin & Peyraud, Suzanne & Bach,Sinh Tan, 2013. "Ensuring robust flood risk management in Ho Chi Minh city," Policy Research Working Paper Series 6465, The World Bank.
    7. Tietje, Olaf, 2005. "Identification of a small reliable and efficient set of consistent scenarios," European Journal of Operational Research, Elsevier, vol. 162(2), pages 418-432, April.
    8. Wright, George & Bradfield, Ron & Cairns, George, 2013. "Does the intuitive logics method – and its recent enhancements – produce “effective” scenarios?," Technological Forecasting and Social Change, Elsevier, vol. 80(4), pages 631-642.
    9. Robert Lempert, 2013. "Scenarios that illuminate vulnerabilities and robust responses," Climatic Change, Springer, vol. 117(4), pages 627-646, April.
    10. Hamarat, Caner & Kwakkel, Jan H. & Pruyt, Erik, 2013. "Adaptive Robust Design under deep uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 408-418.
    11. Yousra Tourki & Jeffrey Keisler & Igor Linkov, 2013. "Scenario analysis: a review of methods and applications for engineering and environmental systems," Environment Systems and Decisions, Springer, vol. 33(1), pages 3-20, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cheng, M.N. & Wong, Jane W.K. & Cheung, C.F. & Leung, K.H., 2016. "A scenario-based roadmapping method for strategic planning and forecasting: A case study in a testing, inspection and certification company," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 44-62.
    2. Steininger, Dennis M. & Gatzemeier, Simon, 2019. "Digitally forecasting new music product success via active crowdsourcing," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 167-180.
    3. Steinmann, Patrick & Auping, Willem L. & Kwakkel, Jan H., 2020. "Behavior-based scenario discovery using time series clustering," Technological Forecasting and Social Change, Elsevier, vol. 156(C).
    4. Marzena Kramarz & Katarzyna Dohn & Edyta Przybylska & Lilla Knop, 2020. "Scenarios for the Development of Multimodal Transport in the TRITIA Cross-Border Area," Sustainability, MDPI, vol. 12(17), pages 1-41, August.
    5. Densing, M. & Panos, E. & Hirschberg, S., 2016. "Meta-analysis of energy scenario studies: Example of electricity scenarios for Switzerland," Energy, Elsevier, vol. 109(C), pages 998-1015.
    6. Trutnevyte, Evelina & McDowall, Will & Tomei, Julia & Keppo, Ilkka, 2016. "Energy scenario choices: Insights from a retrospective review of UK energy futures," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 326-337.
    7. Adagha, Ovo & Levy, Richard M. & Carpendale, Sheelagh & Gates, Cormack & Lindquist, Mark, 2017. "Evaluation of a visual analytics decision support tool for wind farm placement planning in Alberta: Findings from a focus group study," Technological Forecasting and Social Change, Elsevier, vol. 117(C), pages 70-83.
    8. Sahar Shahpari & Janelle Allison & Matthew Tom Harrison & Roger Stanley, 2021. "An Integrated Economic, Environmental and Social Approach to Agricultural Land-Use Planning," Land, MDPI, vol. 10(4), pages 1-18, April.
    9. Jahel, Camille & Bourgeois, Robin & Bourgoin, Jérémy & Daré, William's & De Lattre-Gasquet, Marie & Delay, Etienne & Dumas, Patrice & Le Page, Christophe & Piraux, Marc & Prudhomme, Rémi, 2023. "The future of social-ecological systems at the crossroads of quantitative and qualitative methods," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    10. Kwakkel, J.H. & Cunningham, S.C., 2016. "Improving scenario discovery by bagging random boxes," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 124-134.
    11. Goerigk, Marc & Hartisch, Michael, 2023. "A framework for inherently interpretable optimization models," European Journal of Operational Research, Elsevier, vol. 310(3), pages 1312-1324.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steinmann, Patrick & Auping, Willem L. & Kwakkel, Jan H., 2020. "Behavior-based scenario discovery using time series clustering," Technological Forecasting and Social Change, Elsevier, vol. 156(C).
    2. Densing, M. & Panos, E. & Hirschberg, S., 2016. "Meta-analysis of energy scenario studies: Example of electricity scenarios for Switzerland," Energy, Elsevier, vol. 109(C), pages 998-1015.
    3. Evelina Trutnevyte & Céline Guivarch & Robert Lempert & Neil Strachan, 2016. "Reinvigorating the scenario technique to expand uncertainty consideration," Climatic Change, Springer, vol. 135(3), pages 373-379, April.
    4. Jan H. Kwakkel, 2019. "A generalized many‐objective optimization approach for scenario discovery," Futures & Foresight Science, John Wiley & Sons, vol. 1(2), June.
    5. Céline Guivarch & Julie Rozenberg & Vanessa Schweizer, 2016. "The diversity of socio-economic pathways and CO2 emissions scenarios: Insights from the investigation of a scenarios database," Post-Print halshs-01292901, HAL.
    6. Tiberius, Victor & Siglow, Caroline & Sendra-García, Javier, 2020. "Scenarios in business and management: The current stock and research opportunities," Journal of Business Research, Elsevier, vol. 121(C), pages 235-242.
    7. Kwakkel, J.H. & Cunningham, S.C., 2016. "Improving scenario discovery by bagging random boxes," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 124-134.
    8. Moallemi, Enayat A. & Elsawah, Sondoss & Ryan, Michael J., 2020. "Strengthening ‘good’ modelling practices in robust decision support: A reporting guideline for combining multiple model-based methods," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 175(C), pages 3-24.
    9. Erik Pruyt & Jan H. Kwakkel, 2014. "Radicalization under deep uncertainty: a multi-model exploration of activism, extremism, and terrorism," System Dynamics Review, System Dynamics Society, vol. 30(1-2), pages 1-28, January.
    10. Julie Rozenberg & Céline Guivarch & Robert Lempert & Stéphane Hallegatte, 2014. "Building SSPs for climate policy analysis: a scenario elicitation methodology to map the space of possible future challenges to mitigation and adaptation," Climatic Change, Springer, vol. 122(3), pages 509-522, February.
    11. Julie E. Shortridge & Seth D. Guikema, 2016. "Scenario Discovery with Multiple Criteria: An Evaluation of the Robust Decision‐Making Framework for Climate Change Adaptation," Risk Analysis, John Wiley & Sons, vol. 36(12), pages 2298-2312, December.
    12. Eker, Sibel & van Daalen, Els, 2015. "A model-based analysis of biomethane production in the Netherlands and the effectiveness of the subsidization policy under uncertainty," Energy Policy, Elsevier, vol. 82(C), pages 178-196.
    13. Trutnevyte, Evelina, 2016. "Does cost optimization approximate the real-world energy transition?," Energy, Elsevier, vol. 106(C), pages 182-193.
    14. Moallemi, Enayat A. & Elsawah, Sondoss & Ryan, Michael J., 2020. "Robust decision making and Epoch–Era analysis: A comparison of two robustness frameworks for decision-making under uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    15. Dittrich, Ruth & Wreford, Anita & Moran, Dominic, 2016. "A survey of decision-making approaches for climate change adaptation: Are robust methods the way forward?," Ecological Economics, Elsevier, vol. 122(C), pages 79-89.
    16. Bhave, Ajay Gajanan & Conway, Declan & Dessai, Suraje & Stainforth, David A., 2017. "Barriers and opportunities for robust decision making approaches to support climate change adaptation in the developing world," LSE Research Online Documents on Economics 68318, London School of Economics and Political Science, LSE Library.
    17. Robert Lempert, 2013. "Scenarios that illuminate vulnerabilities and robust responses," Climatic Change, Springer, vol. 117(4), pages 627-646, April.
    18. Li, Francis G.N. & Trutnevyte, Evelina, 2017. "Investment appraisal of cost-optimal and near-optimal pathways for the UK electricity sector transition to 2050," Applied Energy, Elsevier, vol. 189(C), pages 89-109.
    19. Trutnevyte, Evelina & McDowall, Will & Tomei, Julia & Keppo, Ilkka, 2016. "Energy scenario choices: Insights from a retrospective review of UK energy futures," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 326-337.
    20. Lempert Robert J., 2014. "Embedding (some) benefit-cost concepts into decision support processes with deep uncertainty," Journal of Benefit-Cost Analysis, De Gruyter, vol. 5(3), pages 487-514, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:91:y:2015:i:c:p:64-77. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.