IDEAS home Printed from https://ideas.repec.org/p/wiw/wus055/60237853.html
   My bibliography  Save this paper

Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study

Author

Listed:
  • Greiner, Ben
  • Grünwald, Philipp
  • Lindner, Thomas
  • Lintner, Georg
  • Wiernsperger, Martin

Abstract

Managerial decision-makers are increasingly supported by advanced data analytics and other AI-based technologies, but are often found to be hesitant to follow the algorithmic advice. We examine how compensation contract design and framing of an AI algorithm influence decision-makers’ reliance on algorithmic advice and performance in a price estimation task. Based on a large sample of almost 1,500 participants, we find that compared to a fixed compensation, both compensation contracts based on individual performance and tournament contracts lead to an increase in effort duration and to more reliance on algorithmic advice. We further find that using an AI algorithm that is framed as incorporating also human expertise has positive effects on advice utilization, especially for decision-makers with fixed pay contracts. By showing how widely used control practices such as incentives and task framing influence the interaction of human decision-makers with AI algorithms, our findings have direct implications for managerial practice.

Suggested Citation

  • Greiner, Ben & Grünwald, Philipp & Lindner, Thomas & Lintner, Georg & Wiernsperger, Martin, 2024. "Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study," Department for Strategy and Innovation Working Paper Series 01/2024, WU Vienna University of Economics and Business.
    • Handle: RePEc:wiw:wus055:60237853
    as

    Download full text from publisher

    File URL: https://research.wu.ac.at/en/publications/137467e7-9d47-4a6c-87dc-673ff4f58009
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Sebastian Krakowski & Johannes Luger & Sebastian Raisch, 2023. "Artificial intelligence and the changing sources of competitive advantage," Strategic Management Journal, Wiley Blackwell, vol. 44(6), pages 1425-1452, June.
    3. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    4. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    5. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    6. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    7. Kevin Bauer & Moritz von Zahn & Oliver Hinz, 2023. "Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing," Information Systems Research, INFORMS, vol. 34(4), pages 1582-1602, December.
    8. Charness, Gary & Gneezy, Uri & Halladay, Brianna, 2016. "Experimental methods: Pay one or pay all," Journal of Economic Behavior & Organization, Elsevier, vol. 131(PA), pages 141-150.
    9. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    10. Bonner, Sarah E. & Sprinkle, Geoffrey B., 2002. "The effects of monetary incentives on effort and task performance: theories, evidence, and a framework for research," Accounting, Organizations and Society, Elsevier, vol. 27(4-5), pages 303-345.
    11. Libby, Robert & Luft, Joan, 1993. "Determinants of judgment performance in accounting settings: Ability, knowledge, motivation, and environment," Accounting, Organizations and Society, Elsevier, vol. 18(5), pages 425-450, July.
    12. Libby, R & Lipe, Mg, 1992. "Incentives, Effort, And The Cognitive-Processes Involved In Accounting-Related Judgments," Journal of Accounting Research, Wiley Blackwell, vol. 30(2), pages 249-273.
    13. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    14. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    15. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    16. Kevin Bauer & Oliver Hinz & Wil Aalst & Christof Weinhardt, 2021. "Expl(AI)n It to Me – Explainable AI and Information Systems Research," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(2), pages 79-82, April.
    17. Benjamin P. Commerford & Sean A. Dennis & Jennifer R. Joe & Jenny W. Ulla, 2022. "Man Versus Machine: Complex Estimates and Auditor Reliance on Artificial Intelligence," Journal of Accounting Research, Wiley Blackwell, vol. 60(1), pages 171-201, March.
    18. Schotter,Andrew, 2023. "Advice, Social Learning and the Evolution of Conventions," Cambridge Books, Cambridge University Press, number 9781316518076, January.
    19. Labro, Eva & Lang, Mark & Omartian, James D., 2023. "Predictive analytics and centralization of authority," Journal of Accounting and Economics, Elsevier, vol. 75(1).
    20. Joshua Tasoff & Wenjie Zhang, 2022. "The Performance of Time-Preference and Risk-Preference Measures in Surveys," Management Science, INFORMS, vol. 68(2), pages 1149-1173, February.
    21. Costello, Anna M. & Down, Andrea K. & Mehta, Mihir N., 2020. "Machine + man: A field experiment on the role of discretion in augmenting AI-based lending models," Journal of Accounting and Economics, Elsevier, vol. 70(2).
    22. Frechette, Guillaume R. & Schotter, Andrew (ed.), 2015. "Handbook of Experimental Economic Methodology," OUP Catalogue, Oxford University Press, number 9780195328325.
    23. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    24. Shujun Ding & Philip Beaulieu, 2011. "The Role of Financial Incentives in Balanced Scorecard‐Based Performance Evaluations: Correcting Mood Congruency Biases," Journal of Accounting Research, Wiley Blackwell, vol. 49(5), pages 1223-1247, December.
    25. Prithwiraj Choudhury & Evan Starr & Rajshree Agarwal, 2020. "Machine learning and human capital complementarities: Experimental evidence on bias mitigation," Strategic Management Journal, Wiley Blackwell, vol. 41(8), pages 1381-1411, August.
    26. Andrew Prahl & Lyn Van Swol, 2017. "Understanding algorithm aversion: When is advice from automation discounted?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(6), pages 691-702, September.
    27. Ashton, Rh, 1990. "Pressure And Performance In Accounting Decision Settings - Paradoxical Effects Of Incentives, Feedback, And Justification," Journal of Accounting Research, Wiley Blackwell, vol. 28, pages 148-180.
    28. Miao Liu, 2022. "Assessing Human Information Processing in Lending Decisions: A Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 607-651, May.
    29. Christoph March & Anthony Ziegelmeyer & Ben Greiner & René Cyranek, 2016. "Pay Few Subjects but Pay Them Well: Cost-Effectiveness of Random Incentive Systems," CESifo Working Paper Series 5988, CESifo.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bohren, Noah & Hakimov, Rustamdjan & Lalive, Rafael, 2024. "Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments," IZA Discussion Papers 17302, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aycinena, Diego & Bogliacino, Francesco & Kimbrough, Erik O., 2024. "Measuring norms: Assessing the threat of social desirability bias to the Bicchieri and Xiao elicitation method," Journal of Economic Behavior & Organization, Elsevier, vol. 222(C), pages 225-239.
    2. Burdea, Valeria & Woon, Jonathan, 2022. "Online belief elicitation methods," Journal of Economic Psychology, Elsevier, vol. 90(C).
    3. Markus Jung & Mischa Seiter, 2021. "Towards a better understanding on mitigating algorithm aversion in forecasting: an experimental study," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 32(4), pages 495-516, December.
    4. Rafkin, Charlie & Shreekumar, Advik & Vautrey, Pierre-Luc, 2021. "When guidance changes: Government stances and public beliefs," Journal of Public Economics, Elsevier, vol. 196(C).
    5. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    6. Barrafrem, Kinga & Hausfeld, Jan, 2020. "Tracing risky decisions for oneself and others: The role of intuition and deliberation," Journal of Economic Psychology, Elsevier, vol. 77(C).
    7. Gangadharan, Lata & Grossman, Philip J. & Xue, Nina, 2024. "Belief elicitation under competing motivations: Does it matter how you ask?," European Economic Review, Elsevier, vol. 169(C).
    8. Chen, Yan & He, YingHua, 2021. "Information acquisition and provision in school choice: An experimental study," Journal of Economic Theory, Elsevier, vol. 197(C).
    9. Dominik Bauer & Irenaeus Wolff, 2018. "Biases in Beliefs: Experimental Evidence," TWI Research Paper Series 109, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    10. Ondrej Rydval, 2012. "The Causal Effect of Cognitive Abilities on Economic Behavior: Evidence from a Forecasting Task with Varying Cognitive Load," Jena Economics Research Papers 2011-064, Friedrich-Schiller-University Jena.
    11. Bauer, Dominik & Wolff, Irenaeus, 2019. "Biases in Beliefs," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203601, Verein für Socialpolitik / German Economic Association.
    12. Ondrej Rydval, 2007. "The Interaction between Financial Incentives and Task-specific Cognitive Capital: More Evidence in Support of Camerer and Hogarth (1999)," Jena Economics Research Papers 2007-039, Friedrich-Schiller-University Jena.
    13. Ondrej Rydval, 2011. "The Effect of Financial Incentives and Task-specific Cognitive Abilities on Task Performance," Jena Economics Research Papers 2011-050, Friedrich-Schiller-University Jena.
    14. Tse, Tiffany Tsz Kwan & Hanaki, Nobuyuki & Mao, Bolin, 2024. "Beware the performance of an algorithm before relying on it: Evidence from a stock price forecasting experiment," Journal of Economic Psychology, Elsevier, vol. 102(C).
    15. Burro, Giovanni & Castagnetti, Alessandro, 2024. "The ego is no fool: Absence of motivated belief formation in strategic interactions," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 109(C).
    16. Gamba, Astrid & Regner, Tobias, 2019. "Preferences-dependent learning in the centipede game: The persistence of mistrust," European Economic Review, Elsevier, vol. 120(C).
    17. Lee, Natalie, 2023. "Feigning ignorance for long-term gains," Games and Economic Behavior, Elsevier, vol. 138(C), pages 42-71.
    18. Crosetto, Paolo & Filippin, Antonio & Katuščák, Peter & Smith, John, 2020. "Central tendency bias in belief elicitation," Journal of Economic Psychology, Elsevier, vol. 78(C).
    19. Bicchieri, Cristina & Dimant, Eugen & Gächter, Simon & Nosenzo, Daniele, 2022. "Social proximity and the erosion of norm compliance," Games and Economic Behavior, Elsevier, vol. 132(C), pages 59-72.
    20. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.

    More about this item

    Keywords

    artificial intelligence; algorithmic advice; human-augmented algorithmic advice; trust; financial incentives; decision-making;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wiw:wus055:60237853. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: WU Library (email available below). General contact details of provider: https://research.wu.ac.at/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.