IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v51y2005i6p995-1006.html
   My bibliography  Save this article

Probabilistic Inversion of Expert Judgments in the Quantification of Model Uncertainty

Author

Listed:
  • Bernd Kraan

    (Department of Information, Technology and Systems, Delft University of Technology, CROSS, Mekelweg 4, 2628 CD Delft, The Netherlands, and Risk and Uncertainty Management (RandUM), Plein Delftzicht 51, 2627 CA Delft, The Netherlands)

  • Tim Bedford

    (Department of Management Science, Strathclyde Business School, University of Strathclyde, 40 George Street, Glasgow G1 1QE, Scotland)

Abstract

Expert judgment is frequently used to assess parameter values of quantitative management science models, particularly in decision-making contexts. Experts can, however, only be expected to assess observable quantities, not abstract model parameters. This means that we need a method for translating expert assessed uncertainties on model outputs into uncertainties on model parameter values. This process is called probabilistic inversion. The probability distribution on model parameters obtained in this way can be used in a variety of ways, but in particular in an uncertainty analysis or as a Bayes prior. This paper discusses computational algorithms that have proven successful in various projects and gives examples from environmental modelling and banking. Those algorithms are given a theoretical basis by adopting a minimum information approach to modelling partial information. The role of minimum information is two-fold: It enables us to resolve the problem of nonuniqueness of distributions given the information we have, and it provides numerical stability to the algorithm by guaranteeing convergence properties.

Suggested Citation

  • Bernd Kraan & Tim Bedford, 2005. "Probabilistic Inversion of Expert Judgments in the Quantification of Model Uncertainty," Management Science, INFORMS, vol. 51(6), pages 995-1006, June.
    • Handle: RePEc:inm:ormnsc:v:51:y:2005:i:6:p:995-1006
    DOI: 10.1287/mnsc.1050.0370
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.1050.0370
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.1050.0370?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zellner, Arnold, 2002. "Information processing and Bayesian analysis," Journal of Econometrics, Elsevier, vol. 107(1-2), pages 41-50, March.
    2. Miller, Douglas J. & Liu, Wei-han, 2002. "On the recovery of joint distributions from limited information," Journal of Econometrics, Elsevier, vol. 107(1-2), pages 259-274, March.
    3. Craig P. S & Goldstein M. & Rougier J. C & Seheult A. H, 2001. "Bayesian Forecasting for Complex Systems Using Computer Simulators," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 717-729, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bier, Vicki M. & Kosanoglu, Fuat, 2015. "Target-oriented utility theory for modeling the deterrent effects of counterterrorism," Reliability Engineering and System Safety, Elsevier, vol. 136(C), pages 35-46.
    2. Aleksandrina Goeva & Henry Lam & Huajie Qian & Bo Zhang, 2019. "Optimization-Based Calibration of Simulation Input Models," Operations Research, INFORMS, vol. 67(5), pages 1362-1382, September.
    3. Bedford, Tim & Wilson, Kevin J. & Daneshkhah, Alireza, 2014. "Assessing parameter uncertainty on coupled models using minimum information methods," Reliability Engineering and System Safety, Elsevier, vol. 125(C), pages 3-12.
    4. Ioanna Ioannou & Jaime E. Cadena & Willy Aspinall & David Lange & Daniel Honfi & Tiziana Rossetto, 2022. "Prioritization of hazards for risk and resilience management through elicitation of expert judgement," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 112(3), pages 2773-2795, July.
    5. Cooke, Roger M. & Goossens, Louis L.H.J., 2008. "TU Delft expert judgment data base," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 657-674.
    6. Ríos Insua, David & Cano, Javier & Pellot, Michael & Ortega, Ricardo, 2016. "Multithreat multisite protection: A security case study," European Journal of Operational Research, Elsevier, vol. 252(3), pages 888-899.
    7. de Jonge, Bram & Klingenberg, Warse & Teunter, Ruud & Tinga, Tiedo, 2015. "Optimum maintenance strategy under uncertainty in the lifetime distribution," Reliability Engineering and System Safety, Elsevier, vol. 133(C), pages 59-67.
    8. Werner, Christoph & Bedford, Tim & Cooke, Roger M. & Hanea, Anca M. & Morales-Nápoles, Oswaldo, 2017. "Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions," European Journal of Operational Research, Elsevier, vol. 258(3), pages 801-819.
    9. R. E. J. Neslo & W. Oei & M. P. Janssen, 2017. "Insight into “Calculated Risk”: An Application to the Prioritization of Emerging Infectious Diseases for Blood Transfusion Safety," Risk Analysis, John Wiley & Sons, vol. 37(9), pages 1783-1795, September.
    10. Chen Wang & Vicki M. Bier, 2013. "Expert Elicitation of Adversary Preferences Using Ordinal Judgments," Operations Research, INFORMS, vol. 61(2), pages 372-385, April.
    11. Wilson, Alyson G. & Anderson-Cook, Christine M. & Huzurbazar, Aparna V., 2011. "A case study for quantifying system reliability and uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1076-1084.
    12. Kosanoglu, Fuat & Bier, Vicki M., 2020. "Target-oriented utility for interdiction of transportation networks," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    13. Christoph Werner & Tim Bedford & John Quigley, 2018. "Sequential Refined Partitioning for Probabilistic Dependence Assessment," Risk Analysis, John Wiley & Sons, vol. 38(12), pages 2683-2702, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrew Patton, 2002. "(IAM Series No 001) On the Out-Of-Sample Importance of Skewness and Asymetric Dependence for Asset Allocation," FMG Discussion Papers dp431, Financial Markets Group.
    2. Antonio Ciccone & Marek Jarociński, 2010. "Determinants of Economic Growth: Will Data Tell?," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(4), pages 222-246, October.
    3. Ley, Eduardo, 2006. "Statistical inference as a bargaining game," Economics Letters, Elsevier, vol. 93(1), pages 142-149, October.
    4. Diana, Tony, 2011. "Improving schedule reliability based on copulas: An application to five of the most congested US airports," Journal of Air Transport Management, Elsevier, vol. 17(5), pages 284-287.
    5. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    6. Arnold Zellner, 2003. "Some Recent Developments in Econometric Inference," Econometric Reviews, Taylor & Francis Journals, vol. 22(2), pages 203-215.
    7. Murray D. Smith, 2005. "Using Copulas to Model Switching Regimes with an Application to Child Labour," The Economic Record, The Economic Society of Australia, vol. 81(s1), pages 47-57, August.
    8. Campbell, Katherine, 2006. "Statistical calibration of computer simulations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1358-1363.
    9. Komunjer, Ivana & Ragusa, Giuseppe, 2016. "Existence And Characterization Of Conditional Density Projections," Econometric Theory, Cambridge University Press, vol. 32(4), pages 947-987, August.
    10. Wang, Zitian & Wang, Lili & Tan, Shaohua, 2008. "Emergent and spontaneous computation of factor relationships from a large factor set," Journal of Economic Dynamics and Control, Elsevier, vol. 32(12), pages 3939-3959, December.
    11. Lahiri, Kajal & Sheng, Xuguang, 2010. "Learning and heterogeneity in GDP and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 26(2), pages 265-292, April.
    12. Zellner, Arnold, 2004. "To test or not to test and if so, how?: Comments on "size matters"," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 581-586, November.
    13. Goldstein, Michael & Bedford, Tim, 2007. "The Bayes linear approach to inference and decision-making for a reliability programme," Reliability Engineering and System Safety, Elsevier, vol. 92(10), pages 1344-1352.
    14. Andrew J. Patton, 2004. "On the Out-of-Sample Importance of Skewness and Asymmetric Dependence for Asset Allocation," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 130-168.
    15. Alexander Veremyev & Peter Tsyurmasto & Stan Uryasev & R. Rockafellar, 2014. "Calibrating probability distributions with convex-concave-convex functions: application to CDO pricing," Computational Management Science, Springer, vol. 11(4), pages 341-364, October.
    16. Shahsavani, D. & Grimvall, A., 2009. "An adaptive design and interpolation technique for extracting highly nonlinear response surfaces from deterministic models," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1173-1182.
    17. Zellner, Arnold, 2007. "Some aspects of the history of Bayesian information processing," Journal of Econometrics, Elsevier, vol. 138(2), pages 388-404, June.
    18. Nott, David J. & Marshall, Lucy & Fielding, Mark & Liong, Shie-Yui, 2014. "Mixtures of experts for understanding model discrepancy in dynamic computer models," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 491-505.
    19. Wu, Ximing, 2010. "Exponential Series Estimator of multivariate densities," Journal of Econometrics, Elsevier, vol. 156(2), pages 354-366, June.
    20. Mira, José & Sánchez, María Jesús, 2004. "Prediction of deterministic functions: an application of a Gaussian kriging model to a time series outlier problem," Computational Statistics & Data Analysis, Elsevier, vol. 44(3), pages 477-491, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:51:y:2005:i:6:p:995-1006. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.