IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v93y2008i5p657-674.html
   My bibliography  Save this article

TU Delft expert judgment data base

Author

Listed:
  • Cooke, Roger M.
  • Goossens, Louis L.H.J.

Abstract

We review the applications of structured expert judgment uncertainty quantification using the “classical model†developed at the Delft University of Technology over the last 17 years [Cooke RM. Experts in uncertainty. Oxford: Oxford University Press; 1991; Expert judgment study on atmospheric dispersion and deposition. Report Faculty of Technical Mathematics and Informatics No.01-81, Delft University of Technology; 1991]. These involve 45 expert panels, performed under contract with problem owners who reviewed and approved the results. With a few exceptions, all these applications involved the use of seed variables; that is, variables from the experts’ area of expertise for which the true values are available post hoc. Seed variables are used to (1) measure expert performance, (2) enable performance-based weighted combination of experts’ distributions, and (3) evaluate and hopefully validate the resulting combination or “decision maker†. This article reviews the classical model for structured expert judgment and the performance measures, reviews applications, comparing performance-based decision makers with “equal weight†decision makers, and collects some lessons learned.

Suggested Citation

  • Cooke, Roger M. & Goossens, Louis L.H.J., 2008. "TU Delft expert judgment data base," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 657-674.
    • Handle: RePEc:eee:reensy:v:93:y:2008:i:5:p:657-674
    DOI: 10.1016/j.ress.2007.03.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832007000944
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2007.03.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert L. Winkler & Thomas S. Wallsten & Ronald G. Whitfield & Harvey M. Richmond & Stanley R. Hayes & Arlene S. Rosenbaum, 1995. "An Assessment of the Risk of Chronic Lung Injury Attributable to Long-Term Ozone Exposure," Operations Research, INFORMS, vol. 43(1), pages 19-28, February.
    2. L.H.J. Goossens & R.M. Cooke & F Woudenberg & P Van Der Torn, 1998. "Expert judgement and lethal toxicity of inhaled chemicals," Journal of Risk Research, Taylor & Francis Journals, vol. 1(2), pages 117-133, April.
    3. Du, C. & Kurowicka, D. & Cooke, R.M., 2006. "Techniques for generic probabilistic inversion," Computational Statistics & Data Analysis, Elsevier, vol. 50(5), pages 1164-1187, March.
    4. Arno Willems & Mart Janssen ** & Chris Verstegen ** & Tim Bedford, 2005. "Expert quantification of uncertainties in a risk analysis for an infrastructure project," Journal of Risk Research, Taylor & Francis Journals, vol. 8(1), pages 3-17, January.
    5. Bernd Kraan & Tim Bedford, 2005. "Probabilistic Inversion of Expert Judgments in the Quantification of Model Uncertainty," Management Science, INFORMS, vol. 51(6), pages 995-1006, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. R. E. J. Neslo & W. Oei & M. P. Janssen, 2017. "Insight into “Calculated Risk”: An Application to the Prioritization of Emerging Infectious Diseases for Blood Transfusion Safety," Risk Analysis, John Wiley & Sons, vol. 37(9), pages 1783-1795, September.
    2. Roger M. Cooke & Margaret MacDonell, 2008. "Regulating Under Uncertainty: Newsboy for Exposure Limits," Risk Analysis, John Wiley & Sons, vol. 28(3), pages 577-587, June.
    3. Bier, Vicki M. & Kosanoglu, Fuat, 2015. "Target-oriented utility theory for modeling the deterrent effects of counterterrorism," Reliability Engineering and System Safety, Elsevier, vol. 136(C), pages 35-46.
    4. Chen Wang & Vicki M. Bier, 2013. "Expert Elicitation of Adversary Preferences Using Ordinal Judgments," Operations Research, INFORMS, vol. 61(2), pages 372-385, April.
    5. Kosanoglu, Fuat & Bier, Vicki M., 2020. "Target-oriented utility for interdiction of transportation networks," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    6. Roland W. Scholz & Ralf Hansmann, 2007. "Combining Experts' Risk Judgments on Technology Performance of Phytoremediation: Self‐Confidence Ratings, Averaging Procedures, and Formative Consensus Building," Risk Analysis, John Wiley & Sons, vol. 27(1), pages 225-240, February.
    7. Donald L. Keefer & Craig W. Kirkwood & James L. Corner, 2004. "Perspective on Decision Analysis Applications, 1990–2001," Decision Analysis, INFORMS, vol. 1(1), pages 4-22, March.
    8. Ine H. J. Van Der Fels‐Klerx & Louis H. J. Goossens & Helmut W. Saatkamp & Suzan H. S. Horst, 2002. "Elicitation of Quantitative Data from a Heterogeneous Expert Panel: Formal Process and Application in Animal Health," Risk Analysis, John Wiley & Sons, vol. 22(1), pages 67-81, February.
    9. Bedford, Tim & Wilson, Kevin J. & Daneshkhah, Alireza, 2014. "Assessing parameter uncertainty on coupled models using minimum information methods," Reliability Engineering and System Safety, Elsevier, vol. 125(C), pages 3-12.
    10. Lin, Shi-Woei & Bier, Vicki M., 2008. "A study of expert overconfidence," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 711-721.
    11. Ioanna Ioannou & Jaime E. Cadena & Willy Aspinall & David Lange & Daniel Honfi & Tiziana Rossetto, 2022. "Prioritization of hazards for risk and resilience management through elicitation of expert judgement," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 112(3), pages 2773-2795, July.
    12. Christoph Werner & Tim Bedford & John Quigley, 2018. "Sequential Refined Partitioning for Probabilistic Dependence Assessment," Risk Analysis, John Wiley & Sons, vol. 38(12), pages 2683-2702, December.
    13. Robert T. Clemen & Gregory W. Fischer & Robert L. Winkler, 2000. "Assessing Dependence: Some Experimental Results," Management Science, INFORMS, vol. 46(8), pages 1100-1115, August.
    14. H. J. Van der Fels‐Klerx & Roger M. Cooke & Maarten N. Nauta & Louis H. Goossens & Arie H. Havelaar, 2005. "A Structured Expert Judgment Study for a Model of Campylobacter Transmission During Broiler‐Chicken Processing," Risk Analysis, John Wiley & Sons, vol. 25(1), pages 109-124, February.
    15. Ríos Insua, David & Cano, Javier & Pellot, Michael & Ortega, Ricardo, 2016. "Multithreat multisite protection: A security case study," European Journal of Operational Research, Elsevier, vol. 252(3), pages 888-899.
    16. Werner, Christoph & Bedford, Tim & Cooke, Roger M. & Hanea, Anca M. & Morales-Nápoles, Oswaldo, 2017. "Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions," European Journal of Operational Research, Elsevier, vol. 258(3), pages 801-819.
    17. Dorota Kurowicka & Catalin Bucura & Roger Cooke & Arie Havelaar, 2010. "Probabilistic Inversion in Priority Setting of Emerging Zoonoses," Risk Analysis, John Wiley & Sons, vol. 30(5), pages 715-723, May.
    18. Robert L. Winkler, 2015. "The Importance of Communicating Uncertainties in Forecasts: Overestimating the Risks from Winter Storm Juno," Risk Analysis, John Wiley & Sons, vol. 35(3), pages 349-353, March.
    19. Dorota Kurowicka & Maarten Nauta & Katarzyna Jozwiak & Roger Cooke, 2010. "Updating Parameters of the Chicken Processing Line Model," Risk Analysis, John Wiley & Sons, vol. 30(6), pages 934-944, June.
    20. Wilson, Alyson G. & Anderson-Cook, Christine M. & Huzurbazar, Aparna V., 2011. "A case study for quantifying system reliability and uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1076-1084.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:93:y:2008:i:5:p:657-674. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.