IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v19y2022i3p189-204.html
   My bibliography  Save this article

Methods for Eliciting Informative Prior Distributions: A Critical Review

Author

Listed:
  • Julia R. Falconer

    (Department of Mathematics, University of Waikato, Hamilton 3216, New Zealand; New Zealand Institute of Security and Crime Science, University of Waikato, Hamilton 3216, New Zealand)

  • Eibe Frank

    (Department of Computer Science, University of Waikato, Hamilton 3216, New Zealand)

  • Devon L. L. Polaschek

    (New Zealand Institute of Security and Crime Science, University of Waikato, Hamilton 3216, New Zealand; School of Psychology, University of Waikato, Hamilton 3216, New Zealand)

  • Chaitanya Joshi

    (Department of Mathematics, University of Waikato, Hamilton 3216, New Zealand; New Zealand Institute of Security and Crime Science, University of Waikato, Hamilton 3216, New Zealand)

Abstract

Eliciting informative prior distributions for Bayesian inference can often be complex and challenging. Although popular methods rely on asking experts probability-based questions to quantify uncertainty, these methods are not without their drawbacks, and many alternative elicitation methods exist. This paper explores methods for eliciting informative priors categorized by type and briefly discusses their strengths and limitations. Most of the review literature in this field focuses on a particular type of elicitation approach. The primary aim of this work, however, is to provide a more complete yet macro view of the state of the art by highlighting new (and old) approaches in one clear easy-to-read article. Two representative applications are used throughout to explore the suitability, or lack thereof, of the existing methods, one of which highlights a challenge that has not been addressed in the literature yet. We identify some of the gaps in the present work and discuss directions for future research.

Suggested Citation

  • Julia R. Falconer & Eibe Frank & Devon L. L. Polaschek & Chaitanya Joshi, 2022. "Methods for Eliciting Informative Prior Distributions: A Critical Review," Decision Analysis, INFORMS, vol. 19(3), pages 189-204, September.
  • Handle: RePEc:inm:ordeca:v:19:y:2022:i:3:p:189-204
    DOI: 10.1287/deca.2022.0451
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.2022.0451
    Download Restriction: no

    File URL: https://libkey.io/10.1287/deca.2022.0451?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Klayman, Joshua & Soll, Jack B. & Gonzalez-Vallejo, Claudia & Barlas, Sema, 1999. "Overconfidence: It Depends on How, What, and Whom You Ask, , , , , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 79(3), pages 216-247, September.
    2. A.M. Hanea & M.F. McBride & M.A. Burgman & B.C. Wintle, 2018. "Classical meets modern in the IDEA protocol for structured expert judgement," Journal of Risk Research, Taylor & Francis Journals, vol. 21(4), pages 417-433, April.
    3. Cameron J. Williams & Kevin J. Wilson & Nina Wilson, 2021. "A comparison of prior elicitation aggregation using the classical method and SHELF," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 920-940, July.
    4. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    5. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    6. Christine L. Exley & Judd B. Kessler, 2019. "The Gender Gap in Self-Promotion," NBER Working Papers 26345, National Bureau of Economic Research, Inc.
    7. Dennis Lindley, 1983. "Reconciliation of Probability Distributions," Operations Research, INFORMS, vol. 31(5), pages 866-880, October.
    8. Edwards, Ward & Barron, F. Hutton, 1994. "SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement," Organizational Behavior and Human Decision Processes, Elsevier, vol. 60(3), pages 306-325, December.
    9. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    10. Christine L. Exley & Judd B. Kessler, 2019. "The Gender Gap in Self-Promotion," Working Papers 2019-058, Human Capital and Economic Opportunity Working Group.
    11. Geoffrey Jones & Wesley O. Johnson, 2014. "Prior Elicitation: Interactive Spreadsheet Graphics With Sliders Can Be Fun, and Informative," The American Statistician, Taylor & Francis Journals, vol. 68(1), pages 42-51, February.
    12. Robert T. Eckenrode, 1965. "Weighting Multiple Criteria," Management Science, INFORMS, vol. 12(3), pages 180-192, November.
    13. Chen Wang & Vicki M. Bier, 2013. "Expert Elicitation of Adversary Preferences Using Ordinal Judgments," Operations Research, INFORMS, vol. 61(2), pages 372-385, April.
    14. Ali E. Abbas & David V. Budescu & Hsiu-Ting Yu & Ryan Haggerty, 2008. "A Comparison of Two Probability Encoding Methods: Fixed Probability vs. Fixed Variable Values," Decision Analysis, INFORMS, vol. 5(4), pages 190-202, December.
    15. Robert L. Winkler, 1968. "The Consensus of Subjective Probability Distributions," Management Science, INFORMS, vol. 15(2), pages 61-75, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peng Yang & Yuansong Zhao & Lei Nie & Jonathon Vallejo & Ying Yuan, 2023. "SAM: Self‐adapting mixture prior to dynamically borrow information from historical data in clinical trials," Biometrics, The International Biometric Society, vol. 79(4), pages 2857-2868, December.
    2. Julia R. Falconer & Eibe Frank & Devon L. L. Polaschek & Chaitanya Joshi, 2024. "Eliciting Informative Priors by Modeling Expert Decision Making," Decision Analysis, INFORMS, vol. 21(2), pages 77-90, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jason R. W. Merrick, 2008. "Getting the Right Mix of Experts," Decision Analysis, INFORMS, vol. 5(1), pages 43-52, March.
    2. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    3. Anca M. Hanea & Marissa F. McBride & Mark A. Burgman & Bonnie C. Wintle, 2018. "The Value of Performance Weights and Discussion in Aggregated Expert Judgments," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1781-1794, September.
    4. Ali E. Abbas, 2009. "A Kullback-Leibler View of Linear and Log-Linear Pools," Decision Analysis, INFORMS, vol. 6(1), pages 25-37, March.
    5. Gilberto Montibeller & Detlof von Winterfeldt, 2015. "Cognitive and Motivational Biases in Decision and Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1230-1251, July.
    6. Neil A. Stiber & Mitchell J. Small & Marina Pantazidou, 2004. "Site‐Specific Updating and Aggregation of Bayesian Belief Network Models for Multiple Experts," Risk Analysis, John Wiley & Sons, vol. 24(6), pages 1529-1538, December.
    7. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    8. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    9. Patrizio Frederic & Mario Di Bacco & Frank Lad, 2012. "Combining expert probabilities using the product of odds," Theory and Decision, Springer, vol. 73(4), pages 605-619, October.
    10. Jason R. W. Merrick & J. Rene van Dorp & Amita Singh, 2005. "Analysis of Correlated Expert Judgments from Extended Pairwise Comparisons," Decision Analysis, INFORMS, vol. 2(1), pages 17-29, March.
    11. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    12. Ina Ganguli & Marieke Huysentruyt & Chloé Le Coq, 2021. "How Do Nascent Social Entrepreneurs Respond to Rewards? A Field Experiment on Motivations in a Grant Competition," Management Science, INFORMS, vol. 67(10), pages 6294-6316, October.
    13. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    14. Oriana Bandiera & Nidhi Parekh & Barbara Petrongolo & Michelle Rao, 2022. "Men are from Mars, and Women Too: A Bayesian Meta‐analysis of Overconfidence Experiments," Economica, London School of Economics and Political Science, vol. 89(S1), pages 38-70, June.
    15. Benjamin Hansen & Drew McNichols, 2020. "Information and the Persistence of the Gender Wage Gap: Early Evidence from California's Salary History Ban," NBER Working Papers 27054, National Bureau of Economic Research, Inc.
    16. Christian Kascha & Francesco Ravazzolo, 2010. "Combining inflation density forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 231-250.
    17. Dr. James Mitchell, 2009. "Measuring Output Gap Uncertainty," National Institute of Economic and Social Research (NIESR) Discussion Papers 342, National Institute of Economic and Social Research.
    18. Ritwik Banerjee & Priyoma Mustafi, 2020. "Using social recognition to address the gender difference in volunteering for low-promotability tasks," Papers 2012.13514, arXiv.org.
    19. Franz Dietrich & Christian List, 2017. "Probabilistic opinion pooling generalized. Part two: the premise-based approach," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 48(4), pages 787-814, April.
    20. Martine J. Barons & Lael E. Walsh & Edward E. Salakpi & Linda Nichols, 2024. "A Decision Support System for Sustainable Agriculture and Food Loss Reduction under Uncertain Agricultural Policy Frameworks," Agriculture, MDPI, vol. 14(3), pages 1-21, March.
    21. Haeckl, Simone, 2022. "Image concerns in ex-ante self-assessments–Gender differences and behavioral consequences," Labour Economics, Elsevier, vol. 76(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:19:y:2022:i:3:p:189-204. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.