IDEAS home Printed from https://ideas.repec.org/p/hrv/faseco/5343169.html
   My bibliography  Save this paper

A Choice Prediction Competition: Choices From Experience and From Description

Author

Listed:
  • Roth, Alvin E.
  • Herzog, Stefan
  • Hau, Robin
  • Hertwig, Ralph
  • Erev, Ido
  • Ert, Eyal
  • Haruvy, Ernan
  • Stewart, Terrence
  • West, Robert
  • Lebiere, Christian

Abstract

Erev, Ert, and Roth organized three choice prediction competitions focused on three related choice tasks: one shot decisions from description (decisions under risk), one shot decisions from experience, and repeated decisions from experience. Each competition was based on two experimental datasets: An estimation dataset, and a competition dataset. The studies that generated the two datasets used the same methods and subject pool, and examined decision problems randomly selected from the same distribution. After collecting the experimental data to be used for estimation, the organizers posted them on the Web, together with their fit with several baseline models, and challenged other researchers to compete to predict the results of the second (competition) set of experimental sessions. Fourteen teams responded to the challenge: the last seven authors of this paper are members of the winning teams. The results highlight the robustness of the difference between decisions from description and decisions from experience. The best predictions of decisions from descriptions were obtained with a stochastic variant of prospect theory assuming that the sensitivity to the weighted values decreases with the distance between the cumulative payoff functions. The best predictions of decisions from experience were obtained with models that assume reliance on small samples. Merits and limitations of the competition method are discussed.

Suggested Citation

  • Roth, Alvin E. & Herzog, Stefan & Hau, Robin & Hertwig, Ralph & Erev, Ido & Ert, Eyal & Haruvy, Ernan & Stewart, Terrence & West, Robert & Lebiere, Christian, 2009. "A Choice Prediction Competition: Choices From Experience and From Description," Scholarly Articles 5343169, Harvard University Department of Economics.
    • Handle: RePEc:hrv:faseco:5343169
    as

    Download full text from publisher

    File URL: http://dash.harvard.edu/bitstream/handle/1/5343169/Roth_ChoicePrediction.doc
    Download Restriction: no
    File URL: http://dash.harvard.edu/bitstream/handle/1/5343169/Roth_ChoicePrediction.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alvin E. Roth, 2009. "What Have We Learned from Market Design?," Innovation Policy and the Economy, University of Chicago Press, vol. 9(1), pages 79-112.
    2. Richard P. Larrick & Jack B. Soll, 2006. "Erratum--Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(2), pages 309-310, February.
    3. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    4. Haruvy, Ernan & Erev, Ido & Sonsino, Doron, 2001. "The Medium Prizes Paradox: Evidence from a Simulated Casino," Journal of Risk and Uncertainty, Springer, vol. 22(3), pages 251-261, May.
    5. Jerker Denrell & James G. March, 2001. "Adaptation as Information Restriction: The Hot Stove Effect," Organization Science, INFORMS, vol. 12(5), pages 523-538, October.
    6. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    7. Rakow, Tim & Demes, Kali A. & Newell, Ben R., 2008. "Biased samples not mode of presentation: Re-examining the apparent underweighting of rare events in experience-based choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 106(2), pages 168-179, July.
    8. Ido Erev & Alvin Roth & Robert Slonim & Greg Barron, 2007. "Learning and equilibrium as useful approximations: Accuracy of prediction on randomly selected constant sum games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 33(1), pages 29-51, October.
    9. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    10. Hibon, Michele & Evgeniou, Theodoros, 2005. "To combine or not to combine: selecting among forecasts and their combinations," International Journal of Forecasting, Elsevier, vol. 21(1), pages 15-24.
    11. Milton Friedman & L. J. Savage, 1948. "The Utility Analysis of Choices Involving Risk," Journal of Political Economy, University of Chicago Press, vol. 56(4), pages 279-279.
    12. Erev, Ido & Bereby-Meyer, Yoella & Roth, Alvin E., 1999. "The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models," Journal of Economic Behavior & Organization, Elsevier, vol. 39(1), pages 111-128, May.
    13. Reinhard Selten, 1998. "Axiomatic Characterization of the Quadratic Scoring Rule," Experimental Economics, Springer;Economic Science Association, vol. 1(1), pages 43-61, June.
    14. Arifovic, Jasmina & McKelvey, Richard D. & Pevnitskaya, Svetlana, 2006. "An initial implementation of the Turing tournament to learning in repeated two-person games," Games and Economic Behavior, Elsevier, vol. 57(1), pages 93-122, October.
    15. repec:bla:econom:v:58:y:1991:i:229:p:1-16 is not listed on IDEAS
    16. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Stefano DellaVigna & Devin Pope, 2018. "Predicting Experimental Results: Who Knows What?," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2410-2456.
    2. Takemi Fujikawa, 2009. "On the relative importance of the hot stove effect and the tendency to rely on small samples," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 4(5), pages 429-435, August.
    3. Zhihua Li & Julia Müller & Peter P. Wakker & Tong V. Wang, 2018. "The Rich Domain of Ambiguity Explored," Management Science, INFORMS, vol. 64(7), pages 3227-3240, July.
    4. Stefano DellaVigna & Devin Pope, 2018. "What Motivates Effort? Evidence and Expert Forecasts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(2), pages 1029-1069.
    5. Kim, Younjun, 2015. "Essays on firm location decisions, regional development and choices under risk," ISU General Staff Papers 201501010800005579, Iowa State University, Department of Economics.
    6. Laudenbach, Christine & Loos, Benjamin & Pirschel, Jenny & Wohlfart, Johannes, 2020. "The trading response of individual investors to local bankruptcies," SAFE Working Paper Series 272, Leibniz Institute for Financial Research SAFE.
    7. repec:cup:judgdm:v:4:y:2009:i:5:p:429-435 is not listed on IDEAS
    8. Christine Laudenbach & Benjamin Loos & Jenny Pirschel & Johannes Wohlfart, 2020. "The Trading Response of Individual Investors to Local Bankruptcies," CESifo Working Paper Series 8191, CESifo.
    9. Ido Erev & Alvin E. Roth & Robert Slonim, 2016. "Minimax across a population of games," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 2(2), pages 144-156, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. von der Gracht, Heiko A. & Hommel, Ulrich & Prokesch, Tobias & Wohlenberg, Holger, 2016. "Testing weighting approaches for forecasting in a Group Wisdom Support System environment," Journal of Business Research, Elsevier, vol. 69(10), pages 4081-4094.
    2. Tomás Lejarraga & Johannes Müller-Trede, 2017. "When Experience Meets Description: How Dyads Integrate Experiential and Descriptive Information in Risky Decisions," Management Science, INFORMS, vol. 63(6), pages 1953-1971, June.
    3. repec:cup:judgdm:v:8:y:2013:i:2:p:91-105 is not listed on IDEAS
    4. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    5. Graefe, Andreas & Armstrong, J. Scott & Jones, Randall J. & Cuzán, Alfred G., 2014. "Combining forecasts: An application to elections," International Journal of Forecasting, Elsevier, vol. 30(1), pages 43-54.
    6. Karsten Hueffer & Miguel A. Fonseca & Anthony Leiserowitz & Karen M. Taylor, 2013. "The wisdom of crowds: Predicting a weather and climate-related event," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 8(2), pages 91-105, March.
    7. Jonathan Parke & Adrian Parke, 2013. "Does Size Really Matter? A Review Of The Role Of Stake And Prize Levels In Relation To Gambling-Related Harm," Journal of Gambling Business and Economics, University of Buckingham Press, vol. 7(3), pages 77-110.
    8. Aurélien Baillon & Laure Cabantous & Peter Wakker, 2012. "Aggregating imprecise or conflicting beliefs: An experimental investigation using modern ambiguity theories," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 115-147, April.
    9. Jordan Tong & Daniel Feiler, 2017. "A Behavioral Model of Forecasting: Naive Statistics on Mental Samples," Management Science, INFORMS, vol. 63(11), pages 3609-3627, November.
    10. Nicholas Barberis, 2012. "A Model of Casino Gambling," Management Science, INFORMS, vol. 58(1), pages 35-51, January.
    11. Hooi Hooi Lean & Michael McAleer & Wing-Keung Wong, 2013. "Risk-averse and Risk-seeking Investor Preferences for Oil Spot and Futures," Documentos de Trabajo del ICAE 2013-31, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico, revised Aug 2013.
    12. Christian Gollier & James Hammitt & Nicolas Treich, 2013. "Risk and choice: A research saga," Journal of Risk and Uncertainty, Springer, vol. 47(2), pages 129-145, October.
    13. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    14. Martin Kukuk & Stefan Winter, 2008. "An Alternative Explanation of the Favorite-Longshot Bias," Journal of Gambling Business and Economics, University of Buckingham Press, vol. 2(2), pages 79-96, September.
    15. Moshe Levy & Haim Levy, 2013. "Prospect Theory: Much Ado About Nothing?," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 7, pages 129-144, World Scientific Publishing Co. Pte. Ltd..
    16. Lisa L. Posey & Vickie Bajtelsmit, 2017. "Insurance and Endogenous Bankruptcy Risk: When is it Rational to Choose Gambling, Insurance, and Potential Bankruptcy?," The Geneva Risk and Insurance Review, Palgrave Macmillan;International Association for the Study of Insurance Economics (The Geneva Association), vol. 42(1), pages 15-40, March.
    17. Evgeny Kagan & Alexander Rybalov, 2022. "Subjective Trusts and Prospects: Some Practical Remarks on Decision Making with Imperfect Information," SN Operations Research Forum, Springer, vol. 3(1), pages 1-24, March.
    18. Jakusch, Sven Thorsten, 2017. "On the applicability of maximum likelihood methods: From experimental to financial data," SAFE Working Paper Series 148, Leibniz Institute for Financial Research SAFE, revised 2017.
    19. Lean, Hooi Hooi & McAleer, Michael & Wong, Wing-Keung, 2015. "Preferences of risk-averse and risk-seeking investors for oil spot and futures before, during and after the Global Financial Crisis," International Review of Economics & Finance, Elsevier, vol. 40(C), pages 204-216.
    20. Ofir Yakobi & Doron Cohen & Eitan Naveh & Ido Erev, 2020. "Reliance on small samples and the value of taxing reckless behaviors," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(2), pages 266-281, March.
    21. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2013. "Salience and Consumer Choice," Journal of Political Economy, University of Chicago Press, vol. 121(5), pages 803-843.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hrv:faseco:5343169. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Office for Scholarly Communication (email available below). General contact details of provider: https://edirc.repec.org/data/deharus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.