IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_11721.html
   My bibliography  Save this paper

The Perks and Perils of Machine Learning in Business and Economic Research

Author

Listed:
  • Tom L. Dudda
  • Lars Hornuf

Abstract

We examine predictive machine learning studies from 50 top business and economic journals published between 2010 and 2023. We investigate their transparency regarding the predictive performance of machine learning models compared to less complex traditional statistical models that require fewer resources in terms of time and energy. We find that the adoption of machine learning varies by discipline, and is most frequently used in information systems, marketing, and operations research journals. Our analysis also reveals that 28% of studies do not benchmark the predictive performance of machine learning models against traditional statistical models. These studies receive fewer citations, arguably due to a less rigorous analysis. Studies including traditional statistical models as benchmarks typically report high outperformance for the best machine learning model. However, the performance improvement is substantially lower for the average reported machine learning model. We contend that, due to opaque reporting practices, it often remains unclear whether the predictive gains justify the increased costs of more complex models. We advocate for standardized, transparent model reporting that relates predictive gains to the efficiency of machine learning models compared to less-costly traditional statistical models.

Suggested Citation

  • Tom L. Dudda & Lars Hornuf, 2025. "The Perks and Perils of Machine Learning in Business and Economic Research," CESifo Working Paper Series 11721, CESifo.
  • Handle: RePEc:ces:ceswps:_11721
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp11721.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sean Cao Robert H. Smith & Wei Jiang & Baozhong Yang J. Mack Robinson & Alan L Zhang & Tarun Ramadorai, 2023. "How to Talk When a Machine Is Listening: Corporate Disclosure in the Age of AI," The Review of Financial Studies, Society for Financial Studies, vol. 36(9), pages 3603-3642.
    2. Lingfei Wu & Dashun Wang & James A. Evans, 2019. "Large teams develop and small teams disrupt science and technology," Nature, Nature, vol. 566(7744), pages 378-382, February.
    3. Edward Miguel, 2021. "Evidence on Research Transparency in Economics," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 193-214, Summer.
    4. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    5. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    6. Dominik Papies & Peter Ebbes & Elea McDonnell Feit, 2023. "Endogeneity and Causal Inference in Marketing," World Scientific Book Chapters, in: Russell S Winer & Scott A Neslin (ed.), THE HISTORY OF MARKETING SCIENCE, chapter 8, pages 253-300, World Scientific Publishing Co. Pte. Ltd..
    7. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    8. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    9. Matthew Rosenblatt & Link Tejavibulya & Rongtao Jiang & Stephanie Noble & Dustin Scheinost, 2024. "Data leakage inflates prediction performance in connectome-based machine learning models," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    10. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    11. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    12. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    13. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments," American Economic Review, American Economic Association, vol. 105(5), pages 486-490, May.
    14. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    15. Ruomeng Cui & Santiago Gallino & Antonio Moreno & Dennis J. Zhang, 2018. "The Operational Value of Social Media Information," Production and Operations Management, Production and Operations Management Society, vol. 27(10), pages 1749-1769, October.
    16. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Albert J Menkveld & Michael Razen & Utz Weitzel, 2024. "Computational Reproducibility in Finance: Evidence from 1,000 Tests," The Review of Financial Studies, Society for Financial Studies, vol. 37(11), pages 3558-3593.
    17. Lisa Messeri & M. J. Crockett, 2024. "Artificial intelligence and illusions of understanding in scientific research," Nature, Nature, vol. 627(8002), pages 49-58, March.
    18. Xi Chen & Yang Ha (Tony) Cho & Yiwei Dou & Baruch Lev, 2022. "Predicting Future Earnings Changes Using Machine Learning and Detailed Financial Data," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 467-515, May.
    19. Kraus, Mathias & Feuerriegel, Stefan & Oztekin, Asil, 2020. "Deep learning in business analytics and operations research: Models, applications and managerial implications," European Journal of Operational Research, Elsevier, vol. 281(3), pages 628-641.
    20. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    21. Huang, Dashan & Li, Jiangyuan & Wang, Liyao, 2021. "Are disagreements agreeable? Evidence from information aggregation," Journal of Financial Economics, Elsevier, vol. 141(1), pages 83-101.
    22. Franceschet, Massimo & Costantini, Antonio, 2010. "The effect of scholar collaboration on impact and quality of academic papers," Journal of Informetrics, Elsevier, vol. 4(4), pages 540-553.
    23. Susan Athey & Guido W. Imbens, 2019. "Machine Learning Methods That Economists Should Know About," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 685-725, August.
    24. Xin Xu & Feng Xiong & Zhe An, 2023. "Using Machine Learning to Predict Corporate Fraud: Evidence Based on the GONE Framework," Journal of Business Ethics, Springer, vol. 186(1), pages 137-158, August.
    25. Ransome Epie Bawack & Samuel Fosso Wamba & Kevin Daniel André Carillo & Shahriar Akter, 2022. "Artificial intelligence in E-Commerce: a bibliometric study and literature review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(1), pages 297-338, March.
    26. Clément Bosquet & Pierre-Philippe Combes, 2013. "Are academics who publish more also more cited? Individual determinants of publication and citation records," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 831-857, December.
    27. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    28. Nosek, BA & Alter, G & Banks, GC & Borsboom, D & Bowman, SD & Breckler, SJ & Buck, S & Chambers, CD & Chin, G & Christensen, G & Contestabile, M & Dafoe, A & Eich, E & Freese, J & Glennerster, R & Gor, 2015. "Promoting an open research culture," Department of Economics, Working Paper Series qt7wh1000s, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    29. Mark Cecchini & Haldun Aytug & Gary J. Koehler & Praveen Pathak, 2010. "Detecting Management Fraud in Public Companies," Management Science, INFORMS, vol. 56(7), pages 1146-1160, July.
    30. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    31. Glenn Hoetker, 2007. "The use of logit and probit models in strategic management research: Critical issues," Strategic Management Journal, Wiley Blackwell, vol. 28(4), pages 331-343, April.
    32. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    33. Miguel, E & Camerer, C & Casey, K & Cohen, J & Esterling, KM & Gerber, A & Glennerster, R & Green, DP & Humphreys, M & Imbens, G & Laitin, D & Madon, T & Nelson, L & Nosek, BA & Petersen, M & Sedlmayr, 2014. "Promoting Transparency in Social Science Research," Department of Economics, Working Paper Series qt0wt4q2q8, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    34. Turan G Bali & Heiner Beckmeyer & Mathis Mörke & Florian Weigert & Stefano Giglio, 2023. "Option Return Predictability with Machine Learning and Big Data," The Review of Financial Studies, Society for Financial Studies, vol. 36(9), pages 3548-3602.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yulin Liu & Luyao Zhang, 2022. "Cryptocurrency Valuation: An Explainable AI Approach," Papers 2201.12893, arXiv.org, revised Jul 2023.
    2. Kristof Lommers & Ouns El Harzli & Jack Kim, 2021. "Confronting Machine Learning With Financial Research," Papers 2103.00366, arXiv.org, revised Mar 2021.
    3. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    4. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    5. Byun, Suk-Joon & Cho, Sangheum & Kim, Da-Hea, 2024. "Can a machine learn from behavioral biases? Evidence from stock return predictability of deep learning models," Journal of Behavioral and Experimental Finance, Elsevier, vol. 41(C).
    6. Rubesam, Alexandre, 2022. "Machine learning portfolios with equal risk contributions: Evidence from the Brazilian market," Emerging Markets Review, Elsevier, vol. 51(PB).
    7. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
    8. Colak, Gonul & Fu, Mengchuan & Hasan, Iftekhar, 2022. "On modeling IPO failure risk," Economic Modelling, Elsevier, vol. 109(C).
    9. Vasilios Plakandaras & Ioannis Pragidis & Paris Karypidis, 2024. "Deciphering the U.S. metropolitan house price dynamics," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 52(2), pages 434-485, March.
    10. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    11. Lin William Cong & Guanhao Feng & Jingyu He & Xin He, 2022. "Growing the Efficient Frontier on Panel Trees," NBER Working Papers 30805, National Bureau of Economic Research, Inc.
    12. Pan, Shuiyang & Long, Suwan(Cheng) & Wang, Yiming & Xie, Ying, 2023. "Nonlinear asset pricing in Chinese stock market: A deep learning approach," International Review of Financial Analysis, Elsevier, vol. 87(C).
    13. Emanuel Kohlscheen, 2022. "Quantifying the Role of Interest Rates, the Dollar and Covid in Oil Prices," Papers 2208.14254, arXiv.org, revised Oct 2022.
    14. Branco, Rafael R. & Rubesam, Alexandre & Zevallos, Mauricio, 2024. "Forecasting realized volatility: Does anything beat linear models?," Journal of Empirical Finance, Elsevier, vol. 78(C).
    15. Nicolas Gavoille & Anna Zasova, 2021. "What we pay in the shadows: Labor tax evasion, minimum wage hike and employment," SSE Riga/BICEPS Research Papers 6, Baltic International Centre for Economic Policy Studies (BICEPS);Stockholm School of Economics in Riga (SSE Riga).
    16. Hanauer, Matthias X. & Kononova, Marina & Rapp, Marc Steffen, 2022. "Boosting agnostic fundamental analysis: Using machine learning to identify mispricing in European stock markets," Finance Research Letters, Elsevier, vol. 48(C).
    17. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    18. Andrew J. Patton & Yasin Simsek, 2023. "Generalized Autoregressive Score Trees and Forests," Papers 2305.18991, arXiv.org.
    19. Gavoille, Nicolas & Zasova, Anna, 2023. "What we pay in the shadows: Labor tax evasion, minimum wage hike and employment," Journal of Public Economics, Elsevier, vol. 228(C).
    20. Georges, Christophre & Pereira, Javier, 2021. "Market stability with machine learning agents," Journal of Economic Dynamics and Control, Elsevier, vol. 122(C).

    More about this item

    Keywords

    machine learning; predictive modelling; transparent model reporting;
    All these keywords.

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_11721. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.