IDEAS home Printed from https://ideas.repec.org/a/spr/reaccs/v29y2024i4d10.1007_s11142-023-09781-w.html
   My bibliography  Save this article

The explanatory power of explanatory variables

Author

Listed:
  • Erik Johannesson

    (Independent researcher)

  • James A. Ohlson

    (City University of Hong Kong)

  • Sophia Weihuan Zhai

    (Xiamen University)

Abstract

This paper examines the current empirical accounting research paradigm. We ask: In general, do the estimated regressions support the promoted narratives? We focus on a regression model’s main variable of interest and consider the extent to which it contributes to the explanation of the dependent variable. We replicate 10 recently published accounting studies, all of which rely on significant t-statistics, per conventional levels, to claim rejection of the null hypothesis. Our examination shows that in eight studies, the incremental explanatory power contributed by the main variable of interest is effectively zero. For the remaining two, the incremental contribution is at best marginal. These findings highlight the apparent overreliance on t-statistics as the primary evaluation metric. A closer examination of the data shows that the t-statistics produced reject the null hypothesis primarily due to a large number of observations (N). Empirical accounting studies often require N > 10,000 to reject the null hypothesis. To avoid the drawback of t-statistics’ connection with N, we consider the implications of using Standardized Regressions (SR). The magnitude of SR coefficients indicates variables’ relevance directly. Empirical analyses establish a strong correlation between a variable’s estimated SR coefficient magnitude and its incremental explanatory power, without reference to N or t-statistics.

Suggested Citation

  • Erik Johannesson & James A. Ohlson & Sophia Weihuan Zhai, 2024. "The explanatory power of explanatory variables," Review of Accounting Studies, Springer, vol. 29(4), pages 3053-3083, December.
  • Handle: RePEc:spr:reaccs:v:29:y:2024:i:4:d:10.1007_s11142-023-09781-w
    DOI: 10.1007/s11142-023-09781-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11142-023-09781-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11142-023-09781-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alberto Abadie, 2020. "Statistical Nonsignificance in Empirical Economics," American Economic Review: Insights, American Economic Association, vol. 2(2), pages 193-208, June.
    2. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    3. Campbell R. Harvey, 2017. "Presidential Address: The Scientific Outlook in Financial Economics," Journal of Finance, American Finance Association, vol. 72(4), pages 1399-1440, August.
    4. Hirst, Mk, 1983. "Reliance On Accounting Performance-Measures, Task Uncertainty, And Dysfunctional Behavior - Some Extensions," Journal of Accounting Research, Wiley Blackwell, vol. 21(2), pages 596-605.
    5. Mark Andrew Gannon & Carlos Alberto de Bragança Pereira & Adriano Polpo, 2019. "Blending Bayesian and Classical Tools to Define Optimal Sample-Size-Dependent Significance Levels," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 213-222, March.
    6. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    7. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    8. Fama, Eugene F & MacBeth, James D, 1973. "Risk, Return, and Equilibrium: Empirical Tests," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 607-636, May-June.
    9. Tarun Chordia & Amit Goyal & Alessio Saretto & Andrew KarolyiEditor, 2020. "Anomalies and False Rejections," Review of Finance, European Finance Association, vol. 33(5), pages 2134-2179.
    10. Gary C. Biddle & Gim S. Seow & Andrew F. Siegel, 1995. "Relative versus Incremental Information Content," Contemporary Accounting Research, John Wiley & Sons, vol. 12(1), pages 1-23, September.
    11. Thomas R. Dyckman & Stephen A. Zeff, 2015. "Accounting Research: Past, Present, and Future," Abacus, Accounting Foundation, University of Sydney, vol. 51(4), pages 511-524, December.
    12. Mitchell A. Petersen, 2009. "Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches," The Review of Financial Studies, Society for Financial Studies, vol. 22(1), pages 435-480, January.
    13. Mulherin, J. Harold & Netter, Jeffry M. & Poulsen, Annette B., 2018. "Observations on research and publishing from nineteen years as editors of the Journal of Corporate Finance," Journal of Corporate Finance, Elsevier, vol. 49(C), pages 120-124.
    14. James A. Ohlson, 2015. "Accounting Research and Common Sense," Abacus, Accounting Foundation, University of Sydney, vol. 51(4), pages 525-535, December.
    15. Todd Mitton, 2022. "Methodological Variation in Empirical Corporate Finance," The Review of Financial Studies, Society for Financial Studies, vol. 35(2), pages 527-575.
    16. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    17. James A. Ohlson, 2022. "Researchers’ data analysis choices: an excess of false positives?," Review of Accounting Studies, Springer, vol. 27(2), pages 649-667, June.
    18. Thomas R. Dyckman, 2016. "Significance Testing: We Can Do Better," Abacus, Accounting Foundation, University of Sydney, vol. 52(2), pages 319-342, June.
    19. Ferreira, Miguel A. & Matos, Pedro, 2008. "The colors of investors' money: The role of institutional investors around the world," Journal of Financial Economics, Elsevier, vol. 88(3), pages 499-533, June.
    20. Tarun Chordia & Amit Goyal & Alessio Saretto, 2020. "Anomalies and False Rejections," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2134-2179.
    21. Luzi Hail & Mark Lang & Christian Leuz, 2020. "Reproducibility in Accounting Research: Views of the Research Community," Journal of Accounting Research, Wiley Blackwell, vol. 58(2), pages 519-543, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    2. Heath, Davidson & Ringgenberg, Matthew C. & Samadi, Mehrdad & Werner, Ingrid M., 2019. "Reusing Natural Experiments," Working Paper Series 2019-21, Ohio State University, Charles A. Dice Center for Research in Financial Economics.
    3. James A. Ohlson, 2022. "Researchers’ data analysis choices: an excess of false positives?," Review of Accounting Studies, Springer, vol. 27(2), pages 649-667, June.
    4. Todd Mitton, 2022. "Methodological Variation in Empirical Corporate Finance," The Review of Financial Studies, Society for Financial Studies, vol. 35(2), pages 527-575.
    5. Chen, Xiaomeng Charlene & Jones, Stewart & Hasan, Mostafa Monzur & Zhao, Ruoyun & Alam, Nurul, 2023. "Does strategic deviation influence firms’ use of supplier finance?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 85(C).
    6. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).
    7. Stewart Jones & Nurul Alam, 2019. "A machine learning analysis of citation impact among selected Pacific Basin journals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 59(4), pages 2509-2552, December.
    8. Jones, Stewart & Wang, Tim, 2019. "Predicting private company failure: A multi-class analysis," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 61(C), pages 161-188.
    9. Cheng, Tingting & Yan, Cheng & Yan, Yayi, 2021. "Improved inference for fund alphas using high-dimensional cross-sectional tests," Journal of Empirical Finance, Elsevier, vol. 61(C), pages 57-81.
    10. Thomas R. Dyckman & Stephen A. Zeff, 2019. "Important Issues in Statistical Testing and Recommended Improvements in Accounting Research," Econometrics, MDPI, vol. 7(2), pages 1-11, May.
    11. Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024. "Nonstandard Errors," Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
      • Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Utz Weitzel & Michael Razen & Sebastian Neussüs & Michael Kirchler & Magnus Johannesson & Juergen Huber & Felix Holzmeister & Anna Dreber & Albert J. Menkveld & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
      • Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
      • Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-04676112, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
      • Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
      • Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
    12. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    13. Andrew Y. Chen & Tom Zimmermann, 2022. "Publication Bias in Asset Pricing Research," Papers 2209.13623, arXiv.org, revised Sep 2023.
    14. Baltussen, Guido & Swinkels, Laurens & Van Vliet, Pim, 2021. "Global factor premiums," Journal of Financial Economics, Elsevier, vol. 142(3), pages 1128-1154.
    15. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    16. Cakici, Nusret & Zaremba, Adam & Bianchi, Robert J. & Pham, Nga, 2021. "False discoveries in the anomaly research: New insights from the Stock Exchange of Melbourne (1927–1987)," Pacific-Basin Finance Journal, Elsevier, vol. 70(C).
    17. Kim, Jae H. & Shamsuddin, Abul, 2023. "Stock market anomalies: An extreme bounds analysis," International Review of Financial Analysis, Elsevier, vol. 90(C).
    18. Laure de Batz & Evžen Kočenda, 2024. "Financial crime and punishment: A meta‐analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 38(4), pages 1338-1398, September.
    19. Fan Yang & Tomas Havranek & Zuzana Irsova & Jiri Novak, 2024. "Is research on hedge fund performance published selectively? A quantitative survey," Journal of Economic Surveys, Wiley Blackwell, vol. 38(4), pages 1085-1131, September.
    20. Andrew Y. Chen, 2022. "Most claimed statistical findings in cross-sectional return predictability are likely true," Papers 2206.15365, arXiv.org, revised Jan 2025.

    More about this item

    Keywords

    Incremental explanatory power; Large N; Standardized regressions; SR;
    All these keywords.

    JEL classification:

    • M0 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - General
    • M41 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - Accounting

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reaccs:v:29:y:2024:i:4:d:10.1007_s11142-023-09781-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.