IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2401.16754.html
   My bibliography  Save this paper

AI Oversight and Human Mistakes: Evidence from Centre Court

Author

Listed:
  • David Almog
  • Romain Gauriot
  • Lionel Page
  • Daniel Martin

Abstract

Powered by the increasing predictive capabilities of machine learning algorithms, artificial intelligence (AI) systems have begun to be used to overrule human mistakes in many settings. We provide the first field evidence this AI oversight carries psychological costs that can impact human decision-making. We investigate one of the highest visibility settings in which AI oversight has occurred: the Hawk-Eye review of umpires in top tennis tournaments. We find that umpires lowered their overall mistake rate after the introduction of Hawk-Eye review, in line with rational inattention given psychological costs of being overruled by AI. We also find that umpires increased the rate at which they called balls in, which produced a shift from making Type II errors (calling a ball out when in) to Type I errors (calling a ball in when out). We structurally estimate the psychological costs of being overruled by AI using a model of rational inattentive umpires, and our results suggest that because of these costs, umpires cared twice as much about Type II errors under AI oversight.

Suggested Citation

  • David Almog & Romain Gauriot & Lionel Page & Daniel Martin, 2024. "AI Oversight and Human Mistakes: Evidence from Centre Court," Papers 2401.16754, arXiv.org, revised Feb 2024.
    • Handle: RePEc:arx:papers:2401.16754
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2401.16754
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mitchell Hoffman & Lisa B Kahn & Danielle Li, 2018. "Discretion in Hiring," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 765-800.
    2. Andrew Caplin & Daniel Martin, 2015. "A Testable Theory of Imperfect Perception," Economic Journal, Royal Economic Society, vol. 125(582), pages 184-202, February.
    3. Leonardo Bursztyn & Robert Jensen, 2017. "Social Image and Economic Behavior in the Field: Identifying, Understanding, and Shaping Social Pressure," Annual Review of Economics, Annual Reviews, vol. 9(1), pages 131-153, September.
    4. Aaron Chalfin & Oren Danieli & Andrew Hillis & Zubin Jelveh & Michael Luca & Jens Ludwig & Sendhil Mullainathan, 2016. "Productivity and Selection of Human Capital with Machine Learning," American Economic Review, American Economic Association, vol. 106(5), pages 124-127, May.
    5. Luigi Butera & Robert Metcalfe & William Morrison & Dmitry Taubinsky, 2022. "Measuring the Welfare Effects of Shame and Pride," American Economic Review, American Economic Association, vol. 112(1), pages 122-168, January.
    6. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    7. Wayne B. Gray & Jay P. Shimshack, 2011. "The Effectiveness of Environmental Monitoring and Enforcement: A Review of the Empirical Evidence," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 5(1), pages 3-24, Winter.
    8. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    9. Daniel S. Nagin & James B. Rebitzer & Seth Sanders & Lowell J. Taylor, 2002. "Monitoring, Motivation, and Management: The Determinants of Opportunistic Behavior in a Field Experiment," American Economic Review, American Economic Association, vol. 92(4), pages 850-873, September.
    10. Benjamin A. Olken, 2007. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 115(2), pages 200-249.
    11. Kirby Nielsen & John Rehbeck, 2022. "When Choices Are Mistakes," American Economic Review, American Economic Association, vol. 112(7), pages 2237-2268, July.
    12. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    13. Andrew Caplin & Mark Dean, 2013. "Behavioral Implications of Rational Inattention with Shannon Entropy," NBER Working Papers 19318, National Bureau of Economic Research, Inc.
    14. Greer K. Gosnell & John A. List & Robert D. Metcalfe, 2020. "The Impact of Management Practices on Employee Productivity: A Field Experiment with Airline Captains," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1195-1233.
    15. Vivek Bhattacharya & Greg Howard, 2022. "Rational Inattention in the Infield," American Economic Journal: Microeconomics, American Economic Association, vol. 14(4), pages 348-393, November.
    16. Gustavo J. Bobonis & Luis R. Cámara Fuertes & Rainer Schwabe, 2016. "Monitoring Corruptible Politicians," American Economic Review, American Economic Association, vol. 106(8), pages 2371-2405, August.
    17. Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    18. Ely, Jeffrey & Gauriot, Romain & Page, Lionel, 2017. "Do agents maximise? Risk taking on first and second serves in tennis," Journal of Economic Psychology, Elsevier, vol. 63(C), pages 135-142.
    19. David C Chan & Matthew Gentzkow & Chuan Yu, 2022. "Selection with Variation in Diagnostic Skill: Evidence from Radiologists [The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 729-783.
    20. Ran Abramitzky & Liran Einav & Shimon Kolkowitz & Roy Mill, 2012. "On The Optimality Of Line Call Challenges In Professional Tennis," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(3), pages 939-964, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hannes Ullrich & Michael Allan Ribers, 2023. "Machine predictions and human decisions with variation in payoffs and skill: the case of antibiotic prescribing," Berlin School of Economics Discussion Papers 0027, Berlin School of Economics.
    2. Matveenko, Andrei & Starkov, Egor, 2023. "Sparking curiosity or tipping the scales? Targeted advertising with consumer learning," Journal of Economic Behavior & Organization, Elsevier, vol. 213(C), pages 172-192.
    3. Figueroa, Nicolás & Guadalupi, Carla, 2021. "Testing the sender: When signaling is not enough," Journal of Economic Theory, Elsevier, vol. 197(C).
    4. Andrew Caplin & Mark Dean & John Leahy, 2022. "Rationally Inattentive Behavior: Characterizing and Generalizing Shannon Entropy," Journal of Political Economy, University of Chicago Press, vol. 130(6), pages 1676-1715.
    5. Lunn, Pete & Somerville, Jason J., 2015. "Surplus Identification with Non-Linear Returns," Papers WP522, Economic and Social Research Institute (ESRI).
    6. Spyros Galanis & Sergei Mikhalishchev, 2024. "Information Aggregation with Costly Information Acquisition," Papers 2406.07186, arXiv.org, revised Nov 2024.
    7. Li, Anqi & Hu, Lin, 2023. "Electoral accountability and selection with personalized information aggregation," Games and Economic Behavior, Elsevier, vol. 140(C), pages 296-315.
    8. Bauer, Kevin & Gill, Andrej, 2021. "Mirror, mirror on the wall: Machine predictions and self-fulfilling prophecies," SAFE Working Paper Series 313, Leibniz Institute for Financial Research SAFE.
    9. Bauer, Kevin & Pfeuffer, Nicolas & Abdel-Karim, Benjamin M. & Hinz, Oliver & Kosfeld, Michael, 2020. "The terminator of social welfare? The economic consequences of algorithmic discrimination," SAFE Working Paper Series 287, Leibniz Institute for Financial Research SAFE.
    10. Andrew Caplin & Mark Dean, 2015. "Revealed Preference, Rational Inattention, and Costly Information Acquisition," American Economic Review, American Economic Association, vol. 105(7), pages 2183-2203, July.
    11. Elliott Ash & Sergio Galletta & Tommaso Giommoni, 2021. "A Machine Learning Approach to Analyze and Support Anti-Corruption Policy," CESifo Working Paper Series 9015, CESifo.
    12. Matveenko, Andrei & Mikhalishchev, Sergei, 2021. "Attentional role of quota implementation," Journal of Economic Theory, Elsevier, vol. 198(C).
    13. Martin, Daniel & Muñoz-Rodriguez, Edwin, 2022. "Cognitive costs and misperceived incentives: Evidence from the BDM mechanism," European Economic Review, Elsevier, vol. 148(C).
    14. Brocas, Isabelle & Carrillo, Juan D., 2021. "Value computation and modulation: A neuroeconomic theory of self-control as constrained optimization," Journal of Economic Theory, Elsevier, vol. 198(C).
    15. Fuhai Hong & Dong Zhang, 2023. "Bureaucratic beliefs and law enforcement," Public Choice, Springer, vol. 196(3), pages 357-379, September.
    16. Harris, Donna & Borcan , Oana & Serra, Danila & Telli, Henry & Schettini, Bruno & Dercon, Stefan, 2024. "Proud to Belong: The Impact of Ethics Training on Police Officers in Ghana," CEPR Discussion Papers 19141, C.E.P.R. Discussion Papers.
    17. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    18. Jiang, Shuguang & Wei, Qian & Zhao, Lei, 2024. "Synergizing anti-corruption strategies: Group monitoring and endogenous crackdown – An experimental investigation," China Economic Review, Elsevier, vol. 85(C).
    19. repec:ags:aaea22:335618 is not listed on IDEAS
    20. Tommaso Denti & Doron Ravid, 2023. "Robust Predictions in Games with Rational Inattention," Papers 2306.09964, arXiv.org.
    21. Derksen, Laura & Kerwin, Jason Theodore & Reynoso, Natalia Ordaz & Sterck, Olivier, 2021. "Appointments: A More Effective Commitment Device for Health Behaviors," SocArXiv y8gh7, Center for Open Science.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2401.16754. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.