IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v125y2014icp67-81.html
   My bibliography  Save this article

Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment

Author

Listed:
  • Rae, Andrew
  • Alexander, Rob
  • McDermid, John

Abstract

Quantitative risk assessment (QRA) is widely practiced in system safety, but there is insufficient evidence that QRA in general is fit for purpose. Defenders of QRA draw a distinction between poor or misused QRA and correct, appropriately used QRA, but this distinction is only useful if we have robust ways to identify the flaws in an individual QRA. In this paper we present a comprehensive maturity model for QRA which covers all the potential flaws discussed in the risk assessment literature and in a collection of risk assessment peer reviews. We provide initial validation of the completeness and realism of the model.

Suggested Citation

  • Rae, Andrew & Alexander, Rob & McDermid, John, 2014. "Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment," Reliability Engineering and System Safety, Elsevier, vol. 125(C), pages 67-81.
    • Handle: RePEc:eee:reensy:v:125:y:2014:i:c:p:67-81
    DOI: 10.1016/j.ress.2013.09.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832013002652
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2013.09.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stroeve, Sybert H. & Blom, Henk A.P. & Bakker, G.J. (Bert), 2013. "Contrasting safety assessments of a runway incursion scenario: Event sequence analysis versus multi-agent dynamic risk modelling," Reliability Engineering and System Safety, Elsevier, vol. 109(C), pages 133-149.
    2. Craig R. Fox & Robert T. Clemen, 2005. "Subjective Probability Assessment in Decision Analysis: Partition Dependence and Bias Toward the Ignorance Prior," Management Science, INFORMS, vol. 51(9), pages 1417-1432, September.
    3. Aven, Terje & Heide, Bjørnar, 2009. "Reliability and validity of risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1862-1868.
    4. Daníelsson, Jón, 2008. "Blame the models," Journal of Financial Stability, Elsevier, vol. 4(4), pages 321-328, December.
    5. John F. Ahearne, 1999. "The responsibilities of a probabilistic safety analyst," Journal of Risk Research, Taylor & Francis Journals, vol. 2(4), pages 295-306, October.
    6. John A. List, 2001. "Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards," American Economic Review, American Economic Association, vol. 91(5), pages 1498-1507, December.
    7. James T. Hamilton & W. Kip Viscusi, 1999. "Are Risk Regulators Rational? Evidence from Hazardous Waste Cleanup Decisions," American Economic Review, American Economic Association, vol. 89(4), pages 1010-1027, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nguyen, Son & Chen, Peggy Shu-Ling & Du, Yuquan & Thai, Vinh V., 2021. "An Operational Risk Analysis Model for Container Shipping Systems considering Uncertainty Quantification," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    2. Shenae Lee & Gabriele Landucci & Genserik Reniers & Nicola Paltrinieri, 2019. "Validation of Dynamic Risk Analysis Supporting Integrated Operations Across Systems," Sustainability, MDPI, vol. 11(23), pages 1-25, November.
    3. Sujan, Mark A. & Habli, Ibrahim & Kelly, Tim P. & Gühnemann, Astrid & Pozzi, Simone & Johnson, Christopher W., 2017. "How can health care organisations make and justify decisions about risk reduction? Lessons from a cross-industry review and a health care stakeholder consensus development process," Reliability Engineering and System Safety, Elsevier, vol. 161(C), pages 1-11.
    4. Amundrud, Øystein & Aven, Terje, 2015. "On how to understand and acknowledge risk," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 42-47.
    5. Stephen Thomas & Katrina M Groth, 2023. "Toward a hybrid causal framework for autonomous vehicle safety analysis," Journal of Risk and Reliability, , vol. 237(2), pages 367-388, April.
    6. Alceu Souza & Ariane Maria Machado de Oliveira & Dayla Karolina Fossile & Emmanuel Óguchi Ogu & Luciano Luiz Dalazen & Claudimar Pereira da Veiga, 2020. "Business Plan Analysis Using Multi-Index Methodology: Expectations of Return and Perceived Risks," SAGE Open, , vol. 10(1), pages 21582440199, January.
    7. Aven, Terje, 2018. "How the integration of System 1-System 2 thinking and recent risk perspectives can improve risk assessment and management," Reliability Engineering and System Safety, Elsevier, vol. 180(C), pages 237-244.
    8. Christian C. Blanco & Felipe Caro & Charles J. Corbett, 2019. "Managing Safety‐Related Disruptions: Evidence from the U.S. Nuclear Power Industry," Risk Analysis, John Wiley & Sons, vol. 39(10), pages 2197-2213, October.
    9. Nguyen, Son & Chen, Peggy Shu-Ling & Du, Yuquan & Shi, Wenming, 2019. "A quantitative risk analysis model with integrated deliberative Delphi platform for container shipping operational risks," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 129(C), pages 203-227.
    10. Denney, Ewen & Pai, Ganesh & Whiteside, Iain, 2019. "The role of safety architectures in aviation safety cases," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    11. Marie Røyksund & Roger Flage, 2019. "When Is a Risk Assessment Deficient According to an Uncertainty‐Based Risk Perspective?," Risk Analysis, John Wiley & Sons, vol. 39(4), pages 761-776, April.
    12. Lewis, Austin D. & Groth, Katrina M., 2022. "Metrics for evaluating the performance of complex engineering system health monitoring models," Reliability Engineering and System Safety, Elsevier, vol. 223(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ginger Zhe Jin & Andrew Kato & John A. List, 2010. "That’S News To Me! Information Revelation In Professional Certification Markets," Economic Inquiry, Western Economic Association International, vol. 48(1), pages 104-122, January.
    2. Richard A. Hofler & John A. List, 2004. "Valuation on the Frontier: Calibrating Actual and Hypothetical Statements of Value," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 86(1), pages 213-221.
    3. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    4. Pasquale Lucio Scandizzo & Maria Rita Pierleoni, 2018. "Assessing The Olympic Games: The Economic Impact And Beyond," Journal of Economic Surveys, Wiley Blackwell, vol. 32(3), pages 649-682, July.
    5. Giles Atkinson & Sian Morse-Jones & Susana Mourato & Allan Provins, 2012. "‘When to Take “No” for an Answer’? Using Entreaties to Reduce Protests in Contingent Valuation Studies," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(4), pages 497-523, April.
    6. Marieka M. Klawitter & C. Leigh Anderson & Mary Kay Gugerty, 2013. "Savings And Personal Discount Rates In A Matched Savings Program For Low-Income Families," Contemporary Economic Policy, Western Economic Association International, vol. 31(3), pages 468-485, July.
    7. Rosqvist, Tony, 2010. "On the validation of risk analysis—A commentary," Reliability Engineering and System Safety, Elsevier, vol. 95(11), pages 1261-1265.
    8. Magali Delmas & Ivan Montiel, 2009. "Greening the Supply Chain: When Is Customer Pressure Effective?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 18(1), pages 171-201, March.
    9. Frode Alfnes & Chengyan Yue & Helen H. Jensen, 2010. "Cognitive dissonance as a means of reducing hypothetical bias," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 37(2), pages 147-163, June.
    10. Aven, Terje & Renn, Ortwin, 2018. "Improving government policy on risk: Eight key principles," Reliability Engineering and System Safety, Elsevier, vol. 176(C), pages 230-241.
    11. John A. List & Robert P. Berrens & Alok K. Bohara & Joe Kerkvliet, 2004. "Examining the Role of Social Isolation on Stated Preferences," American Economic Review, American Economic Association, vol. 94(3), pages 741-752, June.
    12. Craig E. Landry & John A. List, 2007. "Using Ex Ante Approaches to Obtain Credible Signals for Value in Contingent Markets: Evidence from the Field," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 89(2), pages 420-429.
    13. Els Hannes & Diana Kusumastuti & Maikel Espinosa & Davy Janssens & Koen Vanhoof & Geert Wets, 2012. "Mental maps and travel behaviour: meanings and models," Journal of Geographical Systems, Springer, vol. 14(2), pages 143-165, April.
    14. Claudio Borio & Mathias Drehmann, 2011. "Toward an Operational Framework for Financial Stability: “Fuzzy” Measurement and Its Consequences," Central Banking, Analysis, and Economic Policies Book Series, in: Rodrigo Alfaro (ed.),Financial Stability, Monetary Policy, and Central Banking, edition 1, volume 15, chapter 4, pages 063-123, Central Bank of Chile.
    15. Dmitriy Volinskiy & Wiktor Adamowicz & Michele Veeman, 2011. "Predicting versus testing: a conditional cross‐forecasting accuracy measure for hypothetical bias," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 55(3), pages 429-450, July.
    16. List John A. & Sinha Paramita & Taylor Michael H., 2006. "Using Choice Experiments to Value Non-Market Goods and Services: Evidence from Field Experiments," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 6(2), pages 1-39, January.
    17. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2013. "Experimental methods: Extra-laboratory experiments-extending the reach of experimental economics," Journal of Economic Behavior & Organization, Elsevier, vol. 91(C), pages 93-100.
    18. Liangyan Wang & Xun Deng & Haipeng (Allan) Chen, 2024. "A rose by any other name would smell as sweet? The impact of hierarchical labeling on consumers’ choices in tiered pricing plans," Marketing Letters, Springer, vol. 35(2), pages 259-273, June.
    19. Carlsson, Fredrik & Kataria, Mitesh, 2016. "How are you? How's it going? What's up? What's happening? Nudging people to tell us how they really are," Working Papers in Economics 649, University of Gothenburg, Department of Economics.
    20. Antão, P. & Sun, S. & Teixeira, A.P. & Guedes Soares, C., 2023. "Quantitative assessment of ship collision risk influencing factors from worldwide accident and fleet data," Reliability Engineering and System Safety, Elsevier, vol. 234(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:125:y:2014:i:c:p:67-81. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.