IDEAS home Printed from https://ideas.repec.org/p/pen/papers/18-010.html
   My bibliography  Save this paper

The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness

Author

Listed:
  • Jon Kleinberg

    (Department of Computer Science, Cornell University)

  • Annie Liang

    (Department of Economics, University of Pennsylvania)

  • Sendhil Mullainathan

    (Department of Economics, Harvard University)

Abstract

When testing a theory, we should ask not just whether its predictions match what we see in the data, but also about its “completeness†: how much of the predictable variation in the data does the theory capture? Deï¬ ning completeness is conceptually challenging, but we show how methods based on machine learning can provide tractable measures of completeness. We also identify a model domain—the human perception and generation of randomness—where measures of completeness can be feasibly analyzed; from these measures we discover there is signiï¬ cant structure in the problem that existing theories have yet to capture.

Suggested Citation

  • Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2017. "The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness," PIER Working Paper Archive 18-010, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 09 Aug 2017.
    • Handle: RePEc:pen:papers:18-010
    as

    Download full text from publisher

    File URL: https://economics.sas.upenn.edu/system/files/working-papers/18-010%20PIER%20Paper%20Submission.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Matthew Rabin, 2002. "Inference by Believers in the Law of Small Numbers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 775-816.
    2. Chen, Daniel L. & Moskowitz, Tobias J. & Shue, Kelly, 2016. "Decision-Making Under the Gambler’s Fallacy: Evidence From Asylum Courts, Loan Officers, and Baseball Umpires," IAST Working Papers 16-43, Institute for Advanced Study in Toulouse (IAST).
    3. Barberis, Nicholas & Shleifer, Andrei & Vishny, Robert, 1998. "A model of investor sentiment," Journal of Financial Economics, Elsevier, vol. 49(3), pages 307-343, September.
    4. Daniel L. Chen & Tobias J. Moskowitz & Kelly Shue, 2016. "Decision Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1181-1242.
    5. Peysakhovich, Alexander & Naecker, Jeffrey, 2017. "Using methods from machine learning to evaluate behavioral models of choice under risk and ambiguity," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 373-384.
    6. Dimitris Batzilis & Sonia Jaffe & Steven Levitt & John A. List & Jeffrey Picel, 2019. "Behavior in Strategic Settings: Evidence from a Million Rock-Paper-Scissors Games," Games, MDPI, vol. 10(2), pages 1-34, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shoshan, Vered & Hazan, Tamir & Plonsky, Ori, 2023. "BEAST-Net: Learning novel behavioral insights using a neural network adaptation of a behavioral model," OSF Preprints kaeny, Center for Open Science.
    2. Gisches, Eyran J. & Qi, Hang & Becker, William J. & Rapoport, Amnon, 2021. "Strategic retailers and myopic consumers: Competitive pricing of perishable goods," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 92(C).
    3. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," NBER Working Papers 25200, National Bureau of Economic Research, Inc.
    4. Isil Erel & Léa H Stern & Chenhao Tan & Michael S Weisbach, 2021. "Selecting Directors Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3226-3264, National Bureau of Economic Research, Inc.
    5. Pedro Bordalo & John Conlon & Nicola Gennaioli & Spencer Kwon & Andrei Shleifer, 2023. "How People Use Statistics," Working Papers 699, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    6. Daoud, Adel & Kim, Rockli & Subramanian, S.V., 2019. "Predicting women's height from their socioeconomic status: A machine learning approach," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," NBER Working Papers 25200, National Bureau of Economic Research, Inc.
    2. Drew Fudenberg & Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2019. "Measuring the Completeness of Theories," Papers 1910.07022, arXiv.org.
    3. Daniel J. Benjamin & Don A. Moore & Matthew Rabin, 2017. "Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments," NBER Working Papers 23927, National Bureau of Economic Research, Inc.
    4. Tobias J. Moskowitz, 2021. "Asset Pricing and Sports Betting," Journal of Finance, American Finance Association, vol. 76(6), pages 3153-3209, December.
    5. Jonas Radbruch & Amelie Schiprowski, 2020. "Interview Sequences and the Formation of Subjective Assessments," ECONtribute Discussion Papers Series 045, University of Bonn and University of Cologne, Germany.
    6. Maximilian Späth & Daniel Goller, 2023. "Gender differences in investment reactions to irrelevant information," CEPA Discussion Papers 67, Center for Economic Policy Analysis.
    7. Shrestha, Maheshwor, 2019. "Death scares: How potential work-migrants infer mortality rates from migrant deaths," Journal of Development Economics, Elsevier, vol. 141(C).
    8. He, Kevin, 2022. "Mislearning from censored data: The gambler's fallacy and other correlational mistakes in optimal-stopping problems," Theoretical Economics, Econometric Society, vol. 17(3), July.
    9. Doidge, Mary & Feng, Hongli & Hennessy, David A., 2017. "A test of the gambler’s and hot hand fallacies in farmers’ weather and market predictions," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258457, Agricultural and Applied Economics Association.
    10. Qingxia Kong & Georg D. Granic & Nicolas S. Lambert & Chung Piaw Teo, 2020. "Judgment Error in Lottery Play: When the Hot Hand Meets the Gambler’s Fallacy," Management Science, INFORMS, vol. 66(2), pages 844-862, February.
    11. Brian Dillon & Travis J. Lybbert, 2024. "The gambler’s fallacy prevails in lottery play," Journal of Risk and Uncertainty, Springer, vol. 69(1), pages 33-56, August.
    12. Neszveda, G., 2019. "Essays on behavioral finance," Other publications TiSEM 05059039-5236-42a3-be1b-3, Tilburg University, School of Economics and Management.
    13. Brett Green & Jeffrey Zwiebel, 2018. "The Hot-Hand Fallacy: Cognitive Mistakes or Equilibrium Adjustments? Evidence from Major League Baseball," Management Science, INFORMS, vol. 64(11), pages 5315-5348, November.
    14. Chen, Daniel L. & Philippe, Arnaud, 2023. "Clash of norms judicial leniency on defendant birthdays," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 324-344.
    15. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    16. Alejandro Núnez Arroyo, 2018. "Information seeking with selective memory," Documentos CEDE 17131, Universidad de los Andes, Facultad de Economía, CEDE.
    17. Wu, Chen-Hui & Wu, Chin-Shun & Liu, Victor W., 2009. "The conservatism bias in an emerging stock market: Evidence from Taiwan," Pacific-Basin Finance Journal, Elsevier, vol. 17(4), pages 494-505, September.
    18. Payzan-LeNestour, Elise & Pradier, Lionnel & Putniņš, Tālis J., 2023. "Biased risk perceptions: Evidence from the laboratory and financial markets," Journal of Banking & Finance, Elsevier, vol. 154(C).
    19. Tiziana Assenza & Te Bao & Cars Hommes & Domenico Massaro, 2014. "Experiments on Expectations in Macroeconomics and Finance," Research in Experimental Economics, in: Experiments in Macroeconomics, volume 17, pages 11-70, Emerald Group Publishing Limited.
    20. Emara, Noha & Owens, David & Smith, John & Wilmer, Lisa, 2017. "Serial correlation in National Football League play calling and its effects on outcomes," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 69(C), pages 125-132.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pen:papers:18-010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Administrator (email available below). General contact details of provider: https://edirc.repec.org/data/deupaus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.