IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.07277.html
   My bibliography  Save this paper

Mechanisms for belief elicitation without ground truth

Author

Listed:
  • Niklas Valentin Lehmann

Abstract

This review article examines the challenge of eliciting truthful information from multiple individuals when such information cannot be verified against an objective truth, a problem known as information elicitation without verification (IEWV). This article reviews over 25 mechanisms designed to incentivize truth-telling in such scenarios, and their effectiveness in empirical studies. The analysis finds that although many mechanisms theoretically ensure truthfulness as a Bayesian Nash Equilibrium, empirical evidence of such mechanisms working in practice is very limited and generally weak. Consequently, more empirical research is needed to validate mechanisms. Given that many mechanisms are very complex and cannot be easily conveyed to research subjects, this review suggests that simpler, more intuitive mechanisms may be easier to test and apply.

Suggested Citation

  • Niklas Valentin Lehmann, 2024. "Mechanisms for belief elicitation without ground truth," Papers 2409.07277, arXiv.org.
    • Handle: RePEc:arx:papers:2409.07277
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2409.07277
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Keren, Gideon, 1987. "Facing uncertainty in the game of bridge: A calibration study," Organizational Behavior and Human Decision Processes, Elsevier, vol. 39(1), pages 98-114, February.
    2. Koessler, Frédéric & Noussair, Charles & Ziegelmeyer, Anthony, 2012. "Information aggregation and belief elicitation in experimental parimutuel betting markets," Journal of Economic Behavior & Organization, Elsevier, vol. 83(2), pages 195-208.
    3. Krüger, Fabian & Pavlova, Lora, 2024. "Quantifying subjective uncertainty in survey expectations," International Journal of Forecasting, Elsevier, vol. 40(2), pages 796-810.
    4. repec:cup:judgdm:v:13:y:2018:i:4:p:322-333 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schilirò, Daniele & Graziano, Mario, 2011. "Scelte e razionalità nei modelli economici: un'analisi multidisciplinare [Choices and rationality in economic models: a multidisciplinary analysis]," MPRA Paper 31910, University Library of Munich, Germany.
    2. Brenner, Lyle & Griffin, Dale & Koehler, Derek J., 2005. "Modeling patterns of probability calibration with random support theory: Diagnosing case-based judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 97(1), pages 64-81, May.
    3. McKenzie, Craig R.M. & Liersch, Michael J. & Yaniv, Ilan, 2008. "Overconfidence in interval estimates: What does expertise buy you?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(2), pages 179-191, November.
    4. Angelini, Giovanni & De Angelis, Luca & Singleton, Carl, 2022. "Informational efficiency and behaviour within in-play prediction markets," International Journal of Forecasting, Elsevier, vol. 38(1), pages 282-299.
    5. Terrance Odean., 1996. "Volume, Volatility, Price and Profit When All Trader Are Above Average," Research Program in Finance Working Papers RPF-266, University of California at Berkeley.
    6. Oberlechner, Thomas & Osler, Carol, 2012. "Survival of Overconfidence in Currency Markets," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 47(1), pages 91-113, February.
    7. Bolger, Fergus & Onkal-Atay, Dilek, 2004. "The effects of feedback on judgmental interval predictions," International Journal of Forecasting, Elsevier, vol. 20(1), pages 29-39.
    8. Mukund Chari & H. Kevin Steensma & Charles Connaughton, 2020. "Previous and Prospective Career Mobility, Client Capture, and Compromised Professional Judgment: The Withholding of Known Relevant Prior Art by Patent Lawyers on Behalf of Their Clients," Organization Science, INFORMS, vol. 31(2), pages 489-507, March.
    9. Lybbert, Travis J. & Barrett, Christopher B. & McPeak, John G. & Luseno, Winnie K., 2007. "Bayesian Herders: Updating of Rainfall Beliefs in Response to External Forecasts," World Development, Elsevier, vol. 35(3), pages 480-497, March.
    10. Joris Wauters & Zivile Zekaite & Garo Garabedian, 2024. "Owner-occupied housing costs, policy communication, and inflation expectations," Working Paper Research 449, National Bank of Belgium.
    11. Dennis Dittrich & Werner Guth & Boris Maciejovsky, 2005. "Overconfidence in investment decisions: An experimental approach," The European Journal of Finance, Taylor & Francis Journals, vol. 11(6), pages 471-491.
    12. George Wright & Fergus Bolger & Gene Rowe, 2002. "An Empirical Test of the Relative Validity of Expert and Lay Judgments of Risk," Risk Analysis, John Wiley & Sons, vol. 22(6), pages 1107-1122, December.
    13. Onkal, Dilek & Muradoglu, Gulnur, 1996. "Effects of task format on probabilistic forecasting of stock prices," International Journal of Forecasting, Elsevier, vol. 12(1), pages 9-24, March.
    14. Zaleskiewicz, Tomasz, 2011. "Financial forecasts during the crisis: Were experts more accurate than laypeople?," Journal of Economic Psychology, Elsevier, vol. 32(3), pages 384-390, June.
    15. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    16. Haikady N Nagaraja & Shane Sanders, 2020. "The aggregation paradox for statistical rankings and nonparametric tests," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-21, March.
    17. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    18. Michał Krawczyk & Maciej Wilamowski, 2022. "Calibration and incentives: evidence from contract bridge," Working Papers 2022-06, Faculty of Economic Sciences, University of Warsaw.
    19. Mario GRAZIANO & Daniele SCHILIRÒ, 2011. "Rationality And Choices In Economics: Behavioral And Evolutionary Approaches," Theoretical and Practical Research in the Economic Fields, ASERS Publishing, vol. 2(2), pages 182-195.
    20. Helen Fischer & Stefanie Schütte & Anneliese Depoux & Dorothee Amelung & Rainer Sauerborn, 2018. "How Well Do COP22 Attendees Understand Graphs on Climate Change Health Impacts from the Fifth IPCC Assessment Report?," IJERPH, MDPI, vol. 15(5), pages 1-11, April.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.07277. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.