IDEAS home Printed from https://ideas.repec.org/p/tiu/tiucen/c62319d0-674f-4def-8e9f-d19faff3ca6f.html
   My bibliography  Save this paper

Belief Elicitation : A Horse Race among Truth Serums

Author

Listed:
  • Trautmann, S.T.

    (Tilburg University, Center For Economic Research)

  • van de Kuilen, G.

    (Tilburg University, Center For Economic Research)

Abstract

In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Trautmann, S.T. & van de Kuilen, G., 2011. "Belief Elicitation : A Horse Race among Truth Serums," Discussion Paper 2011-117, Tilburg University, Center for Economic Research.
    • Handle: RePEc:tiu:tiucen:c62319d0-674f-4def-8e9f-d19faff3ca6f
    as

    Download full text from publisher

    File URL: https://pure.uvt.nl/ws/portalfiles/portal/1362519/2011-117.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    2. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2010. "Subjective beliefs formation and elicitation rules: experimental evidence," Post-Print halshs-00543828, HAL.
    3. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    4. Frank Heinemann & Rosemarie Nagel & Peter Ockenfels, 2009. "Measuring Strategic Uncertainty in Coordination Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(1), pages 181-221.
    5. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    6. Ernst Fehr & Klaus M. Schmidt, "undated". "Theories of Fairness and Reciprocity - Evidence and Economic Applications," IEW - Working Papers 075, Institute for Empirical Research in Economics - University of Zurich.
    7. Shiller, Robert J & Kon-Ya, Fumiko & Tsutsui, Yoshiro, 1996. "Why Did the Nikkei Crash? Expanding the Scope of Expectations Data Collection," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 156-164, February.
    8. Orazio P. Attanasio, 2009. "Expectations and Perceptions in Developing Countries: Their Measurement and Their Use," American Economic Review, American Economic Association, vol. 99(2), pages 87-92, May.
    9. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    10. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    11. William A. Branch, 2004. "The Theory of Rationally Heterogeneous Expectations: Evidence from Survey Data on Inflation Expectations," Economic Journal, Royal Economic Society, vol. 114(497), pages 592-621, July.
    12. Ernst Fehr & Klaus M. Schmidt, 1999. "A Theory of Fairness, Competition, and Cooperation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(3), pages 817-868.
    13. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818742, September.
    14. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524117, September.
    15. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    16. Antonio Cabrales & Raffaele Miniaci & Marco Piovesan & Giovanni Ponti, 2010. "Social Preferences and Strategic Uncertainty: An Experiment on Markets and Contracts," American Economic Review, American Economic Association, vol. 100(5), pages 2261-2278, December.
    17. Luigi Guiso & Giuseppe Parigi, 1999. "Investment and Demand Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 185-227.
    18. Huck, Steffen & Weizsacker, Georg, 2002. "Do players correctly estimate what others do? : Evidence of conservatism in beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 47(1), pages 71-85, January.
    19. Ferdinand Vieider, 2011. "Separating real incentives and accountability," Experimental Economics, Springer;Economic Science Association, vol. 14(4), pages 507-518, November.
    20. Mohammed Abdellaoui & Frank Vossmann & Martin Weber, 2005. "Choice-Based Elicitation and Decomposition of Decision Weights for Gains and Losses Under Uncertainty," Management Science, INFORMS, vol. 51(9), pages 1384-1399, September.
    21. Rey-Biel, Pedro, 2009. "Equilibrium play and best response to (stated) beliefs in normal form games," Games and Economic Behavior, Elsevier, vol. 65(2), pages 572-585, March.
    22. Delavande, Adeline & Giné, Xavier & McKenzie, David, 2011. "Measuring subjective expectations in developing countries: A critical review and new evidence," Journal of Development Economics, Elsevier, vol. 94(2), pages 151-163, March.
    23. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818735, September.
    24. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524124, September.
    25. Michael D. Hurd, 2009. "Subjective Probabilities in Household Surveys," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 543-564, May.
    26. Machina, Mark J & Schmeidler, David, 1992. "A More Robust Definition of Subjective Probability," Econometrica, Econometric Society, vol. 60(4), pages 745-780, July.
    27. Wei Li, 2007. "Changing One's Mind when the Facts Change: Incentives of Experts and the Design of Reporting Protocols," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1175-1194.
    28. Olivier Armantier & Nicolas Treich, 2009. "Subjective Probabilities In Games: An Application To The Overbidding Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1079-1102, November.
    29. Hessel Oosterbeek & Randolph Sloof & Gijs van de Kuilen, 2004. "Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis," Experimental Economics, Springer;Economic Science Association, vol. 7(2), pages 171-188, June.
    30. Lisa Cameron & Manisha Shah, 2015. "Risk-Taking Behavior in the Wake of Natural Disasters," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 484-515.
    31. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    32. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    33. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818728, September.
    34. Charles Bellemare & Sabine Kröger & Arthur van Soest, 2008. "Measuring Inequity Aversion in a Heterogeneous Population Using Experimental Decisions and Subjective Probabilities," Econometrica, Econometric Society, vol. 76(4), pages 815-839, July.
    35. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    36. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    37. Khwaja, Ahmed & Sloan, Frank & Salm, Martin, 2006. "Evidence on preferences and subjective beliefs of risk takers: The case of smokers," International Journal of Industrial Organization, Elsevier, vol. 24(4), pages 667-682, July.
    38. Guiso, Luigi & Jappelli, Tullio & Terlizzese, Daniele, 1992. "Earnings uncertainty and precautionary saving," Journal of Monetary Economics, Elsevier, vol. 30(2), pages 307-337, November.
    39. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524131, September.
    40. Carman, K.G. & Kooreman, P., 2010. "Flu Shots, Mammogram, and the Perception of Probabilities," Other publications TiSEM fba970b8-6fc7-449b-acf9-9, Tilburg University, School of Economics and Management.
    41. Richard H. Thaler & Eric J. Johnson, 1990. "Gambling with the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice," Management Science, INFORMS, vol. 36(6), pages 643-660, June.
    42. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    43. Chris Starmer, 2000. "Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 332-382, June.
    44. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    45. Li Hao & Daniel Houser, 2010. "Getting It Right the First Time: Belief Elicitation with Novice Participants," Working Papers 1015, George Mason University, Interdisciplinary Center for Economic Science.
    46. Ronald Bosman & Frans van Winden, 2002. "Emotional Hazard in a Power-to-take Experiment," Economic Journal, Royal Economic Society, vol. 112(476), pages 147-169, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    2. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    3. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    4. Casal, Sandro & Fallucchi, Francesco & Quercia, Simone, 2019. "The role of morals in three-player ultimatum games," Journal of Economic Psychology, Elsevier, vol. 70(C), pages 67-79.
    5. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    6. Chen Li & Uyanga Turmunkh & Peter P. Wakker, 2019. "Trust as a decision under ambiguity," Experimental Economics, Springer;Economic Science Association, vol. 22(1), pages 51-75, March.
    7. Theo Offerman & Asa B. Palley, 2016. "Lossed in translation: an off-the-shelf method to recover probabilistic beliefs from loss-averse agents," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 1-30, March.
    8. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    9. Dominik Bauer & Irenaeus Wolff, 2018. "Biases in Beliefs: Experimental Evidence," TWI Research Paper Series 109, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    10. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    11. Bauer, Dominik & Wolff, Irenaeus, 2019. "Biases in Beliefs," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203601, Verein für Socialpolitik / German Economic Association.
    12. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    13. Eva I. Hoppe & Patrick W. Schmitz, 2013. "Contracting under Incomplete Information and Social Preferences: An Experimental Study," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(4), pages 1516-1544.
    14. Martin Beckenkamp & Heike Hennig-Schmidt & Frank P. Maier-Rigaud, 2007. "Cooperation in Symmetric and Asymmetric Prisoner's Dilemma Games," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2006_25, Max Planck Institute for Research on Collective Goods.
    15. Fischbacher, Urs & Fong, Christina M. & Fehr, Ernst, 2009. "Fairness, errors and the power of competition," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 527-545, October.
    16. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    17. Ponti, Giovanni & Carbone, Enrica, 2009. "Positional learning with noise," Research in Economics, Elsevier, vol. 63(4), pages 225-241, December.
    18. Popescu, Andreea Victoria, 2020. "Essays in asset pricing and auctions," Other publications TiSEM 879f7643-7123-4bc8-a5e7-6, Tilburg University, School of Economics and Management.
    19. Ederer, Florian & Patacconi, Andrea, 2010. "Interpersonal comparison, status and ambition in organizations," Journal of Economic Behavior & Organization, Elsevier, vol. 75(2), pages 348-363, August.
    20. Adriana Breaban & Charles N. Noussair & Andreea Victoria Popescu, 2018. "Your money or your time? Experimental evidence on overbidding in all-pay auctions," Working Papers 18-20, Chapman University, Economic Science Institute.

    More about this item

    Keywords

    belief measurement; subjective probability; scoring rules; outcome matching; probability matching; internal validity; external validity;
    All these keywords.

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tiu:tiucen:c62319d0-674f-4def-8e9f-d19faff3ca6f. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Richard Broekman (email available below). General contact details of provider: http://center.uvt.nl .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.