IDEAS home Printed from https://ideas.repec.org/a/bla/brjirl/v62y2024i1p3-27.html
   My bibliography  Save this article

Power and bias in industrial relations research

Author

Listed:
  • Anthony Doucouliagos
  • Hristos Doucouliagos
  • T. D. Stanley

Abstract

We survey 20,439 estimates from 64 distinct research areas to assess power, bias and statistical significance in industrial relations research. The average estimate published in industrial relations research lacks adequate power; average power is 33 per cent, and median power is only 14 per cent, much lower than the conventional 80 per cent standard. Low power means that industrial relations researchers will find it more difficult to detect important associations pertaining to workplace relations. Low power also leads to exaggerated research findings. We find substantial publication bias in industrial relations research, though nearly half of the research areas have little or no bias.

Suggested Citation

  • Anthony Doucouliagos & Hristos Doucouliagos & T. D. Stanley, 2024. "Power and bias in industrial relations research," British Journal of Industrial Relations, London School of Economics, vol. 62(1), pages 3-27, March.
  • Handle: RePEc:bla:brjirl:v:62:y:2024:i:1:p:3-27
    DOI: 10.1111/bjir.12746
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/bjir.12746
    Download Restriction: no

    File URL: https://libkey.io/10.1111/bjir.12746?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Carla Haelermans & Lex Borghans, 2012. "Wage Effects of On-the-Job Training: A Meta-Analysis," British Journal of Industrial Relations, London School of Economics, vol. 50(3), pages 502-528, September.
    2. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    3. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    4. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    5. Alexandre Flage, 2019. "Discrimination against gays and lesbians in hiring decisions: a meta-analysis," International Journal of Manpower, Emerald Group Publishing Limited, vol. 41(6), pages 671-691, September.
    6. Henrik Kleven & Camille Landais & Johanna Posch & Andreas Steinhauer & Josef Zweimüller, 2019. "Child Penalties across Countries: Evidence and Explanations," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 122-126, May.
    7. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," The Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    8. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    9. Lichter, Andreas & Peichl, Andreas & Siegloch, Sebastian, 2015. "The own-wage elasticity of labor demand: A meta-regression analysis," European Economic Review, Elsevier, vol. 80(C), pages 94-119.
    10. Hristos Doucouliagos & Janto Haman & T.D. Stanley, 2012. "Pay for Performance and Corporate Governance Reform," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 51(3), pages 670-703, July.
    11. Patrice Laroche, 2016. "A Meta-Analysis of the Union–Job Satisfaction Relationship," British Journal of Industrial Relations, London School of Economics, vol. 54(4), pages 709-741, December.
    12. Anna Sokolova & Todd Sorensen, 2021. "Monopsony in Labor Markets: A Meta-Analysis," ILR Review, Cornell University, ILR School, vol. 74(1), pages 27-55, January.
    13. Megan Linde Leonard & T. D. Stanley & Hristos Doucouliagos, 2014. "Does the UK Minimum Wage Reduce Employment? A Meta-Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 52(3), pages 499-520, September.
    14. de Linde Leonard, Megan & Stanley, T.D., 2015. "Married with children: What remains when observable biases are removed from the reported male marriage wage premium," Labour Economics, Elsevier, vol. 33(C), pages 72-80.
    15. Hristos Doucouliagos & Richard B. Freeman & Patrice Laroche & T. D. Stanley, 2018. "How Credible Is Trade Union Research? Forty Years of Evidence on the Monopoly–Voice Trade-Off," ILR Review, Cornell University, ILR School, vol. 71(2), pages 287-305, March.
    16. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    17. Philipp Heimberger, 2021. "Does employment protection affect unemployment? A meta-analysis," Oxford Economic Papers, Oxford University Press, vol. 73(3), pages 982-1007.
    18. Zohid Askarov & Anthony Doucouli & Hristos Doucouli & T D Stanley, 2023. "The Significance of Data-Sharing Policy," Journal of the European Economic Association, European Economic Association, vol. 21(3), pages 1191-1226.
    19. Chletsos, Michael & Giotis, Georgios P., 2015. "The employment effect of minimum wage using 77 international studies since 1992: A meta-analysis," MPRA Paper 61321, University Library of Munich, Germany.
    20. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    21. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    22. Hristos Doucouliagos & Richard Freeman & Patrice Laroche & T.D. Stanley, 2018. "How Credible Is Trade Union Research? Forty Years of Evidence on the Monopoly–Voice Trade-Off," Post-Print hal-02137429, HAL.
    23. Keith Whitfield & George Strauss, 2000. "Methods Matter: Changes in Industrial Relations Research and their Implications," British Journal of Industrial Relations, London School of Economics, vol. 38(1), pages 141-151, March.
    24. Dale Belman & Paul J. Wolfson, 2014. "What Does the Minimum Wage Do?," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number wdmwd.
    25. Neal S Young & John P A Ioannidis & Omar Al-Ubaydli, 2008. "Why Current Publication Practices May Distort Science," PLOS Medicine, Public Library of Science, vol. 5(10), pages 1-5, October.
    26. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    27. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    28. Jessica S. Merkle & Michelle Andrea Phillips, 2018. "The Wage Impact Of Teachers Unions: A Meta‐Analysis," Contemporary Economic Policy, Western Economic Association International, vol. 36(1), pages 93-115, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zohid Askarov & Anthony Doucouli & Hristos Doucouli & T D Stanley, 2023. "The Significance of Data-Sharing Policy," Journal of the European Economic Association, European Economic Association, vol. 21(3), pages 1191-1226.
    2. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
    3. repec:wly:econjl:v::y:2017:i:605:p:f236-f265 is not listed on IDEAS
    4. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    5. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    6. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    8. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    9. Doucouliagos, Hristos & Hinz, Thomas & Zigova, Katarina, 2022. "Bias and careers: Evidence from the aid effectiveness literature," European Journal of Political Economy, Elsevier, vol. 71(C).
    10. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    11. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    12. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    13. Tomas Havranek & Zuzana Irsova & Lubica Laslopova & Olesia Zeynalova, 2020. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," Working Papers IES 2020/29, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
    14. Abel Brodeur & Scott Carrell & David Figlio & Lester Lusher, 2023. "Unpacking P-hacking and Publication Bias," American Economic Review, American Economic Association, vol. 113(11), pages 2974-3002, November.
    15. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
    16. Nazila Alinaghi & W. Robert Reed, 2016. "Meta-Analysis and Publication Bias: How Well Does the FAT-PET-PEESE Procedure Work?," Working Papers in Economics 16/26, University of Canterbury, Department of Economics and Finance.
    17. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2020. "Remittances and economic growth: A meta-analysis," World Development, Elsevier, vol. 134(C).
    18. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    19. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    20. Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan & Heyes, Anthony, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," MetaArXiv uxf39, Center for Open Science.
    21. Campbell, Douglas & Brodeur, Abel & Dreber, Anna & Johannesson, Magnus & Kopecky, Joseph & Lusher, Lester & Tsoy, Nikita, 2024. "The Robustness Reproducibility of the American Economic Review," I4R Discussion Paper Series 124, The Institute for Replication (I4R).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:brjirl:v:62:y:2024:i:1:p:3-27. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.