IDEAS home Printed from https://ideas.repec.org/a/sae/socres/v21y2016i1p102-115.html
   My bibliography  Save this article

Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong

Author

Listed:
  • Stephen Gorard

Abstract

This paper reminds readers of the absurdity of statistical significance testing, despite its continued widespread use as a supposed method for analysing numeric data. There have been complaints about the poor quality of research employing significance tests for a hundred years, and repeated calls for researchers to stop using and reporting them. There have even been attempted bans. Many thousands of papers have now been written, in all areas of research, explaining why significance tests do not work. There are too many for all to be cited here. This paper summarises the logical problems as described in over 100 of these prior pieces. It then presents a series of demonstrations showing that significance tests do not work in practice. In fact, they are more likely to produce the wrong answer than a right one. The confused use of significance testing has practical and damaging consequences for people's lives. Ending the use of significance tests is a pressing ethical issue for research. Anyone knowing the problems, as described over one hundred years, who continues to teach, use or publish significance tests is acting unethically, and knowingly risking the damage that ensues.

Suggested Citation

  • Stephen Gorard, 2016. "Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong," Sociological Research Online, , vol. 21(1), pages 102-115, February.
    • Handle: RePEc:sae:socres:v:21:y:2016:i:1:p:102-115
    DOI: 10.5153/sro.3857
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.5153/sro.3857
    Download Restriction: no
    File URL: https://libkey.io/10.5153/sro.3857?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Athena Engman, 2013. "Is there life after P>0.05? Statistical significance and quantitative sociology," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(1), pages 257-270, January.
    2. Andreas Schwab & Eric Abrahamson & William H. Starbuck & Fiona Fidler, 2011. "PERSPECTIVE---Researchers Should Make Thoughtful Assessments Instead of Null-Hypothesis Significance Tests," Organization Science, INFORMS, vol. 22(4), pages 1105-1120, August.
    3. Louis Guttman, 1985. "The illogic of statistical inference for cumulative science," Applied Stochastic Models and Data Analysis, John Wiley & Sons, vol. 1(1), pages 3-9.
    4. Marks R. Nester, 1996. "An Applied Statistician's Creed," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 45(4), pages 401-410, December.
    5. Stephen Gorard, 2003. "Understanding Probabilities and Re-Considering Traditional Research Training," Sociological Research Online, , vol. 8(1), pages 104-112, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Stephen Gorard, 2017. "Significance Testing is Still Wrong, and Damages Real Lives: A Brief Reply to Spreckelsen and Van Der Horst, and Nicholson and McCusker," Sociological Research Online, , vol. 22(2), pages 204-210, May.
    2. James Nicholson & Sean Mccusker, 2016. "Damaging the Case for Improving Social Science Methodology through Misrepresentation: Re-Asserting Confidence in Hypothesis Testing as a Valid Scientific Process," Sociological Research Online, , vol. 21(2), pages 136-147, May.
    3. Thees F Spreckelsen & Mariska Van Der Horst, 2016. "Is Banning Significance Testing the Best Way to Improve Applied Social Science Research? – Questions on Gorard (2016)," Sociological Research Online, , vol. 21(3), pages 95-105, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tomas Macak, 2021. "Stability of Dependencies of Contingent Subgroups with Merged Groups: Vaccination Case Study," Mathematics, MDPI, vol. 9(22), pages 1-12, November.
    2. Peter Jaskiewicz & Joern H. Block & James G. Combs & Danny Miller, 2017. "The Effects of Founder and Family Ownership on Hired CEOs’ Incentives and Firm Performance," Entrepreneurship Theory and Practice, , vol. 41(1), pages 73-103, January.
    3. Dybro Liengaard, Benjamin, 2024. "Measurement invariance testing in partial least squares structural equation modeling," Journal of Business Research, Elsevier, vol. 177(C).
    4. Katrin Zulauf & Ralf Wagner, 2023. "Countering Negotiation Power Asymmetries by Using the Adjusted Winner Algorithm," SN Operations Research Forum, Springer, vol. 4(1), pages 1-20, March.
    5. Christopher Hansen & Joern Block & Matthias Neuenkirch, 2020. "Family Firm Performance Over The Business Cycle: A Meta‐Analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 476-511, July.
    6. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    7. Marc Orlitzky, 2011. "Institutionalized dualism: statistical significance testing as myth and ceremony," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 22(1), pages 47-77, September.
    8. Lindsay, R. Murray, 1995. "Reconsidering the status of tests of significance: An alternative criterion of adequacy," Accounting, Organizations and Society, Elsevier, vol. 20(1), pages 35-53, January.
    9. Block, Joern H. & Miller, Danny & Wagner, Dominik, 2014. "Bayesian methods in family business research," Journal of Family Business Strategy, Elsevier, vol. 5(1), pages 97-104.
    10. R. Murray Lindsay, 1994. "Publication System Biases Associated with the Statistical Testing Paradigm," Contemporary Accounting Research, John Wiley & Sons, vol. 11(1), pages 33-57, June.
    11. van den Oever, Koen, 2017. "Uncharted waters : A behavioral approach to when, why and which organizational changes are adopted," Other publications TiSEM 0136c8c2-ecdd-4f82-8ca7-d, Tilburg University, School of Economics and Management.
    12. Paul C. Nystrom & Ehsan S. Soofi, 2012. "Rare, outlier and extreme: beyond the Gaussian model and measures," International Journal of Complexity in Leadership and Management, Inderscience Enterprises Ltd, vol. 2(1/2), pages 6-38.
    13. Andrew Briggs & Richard Nixon & Simon Dixon & Simon Thompson, 2005. "Parametric modelling of cost data: some simulation evidence," Health Economics, John Wiley & Sons, Ltd., vol. 14(4), pages 421-428, April.
    14. Norman Fickel, 2001. "Sequential Regression: A Neodescriptive Approach to Multicollinearity," EERI Research Paper Series EERI_RP_2001_09, Economics and Econometrics Research Institute (EERI), Brussels.
    15. Thomas C. Powell, 2020. "Can Quantitative Research Solve Social Problems? Pragmatism and the Ethics of Social Research," Journal of Business Ethics, Springer, vol. 167(1), pages 41-48, November.
    16. Jerker Denrell & Christina Fang & Chengwei Liu, 2015. "Perspective—Chance Explanations in the Management Sciences," Organization Science, INFORMS, vol. 26(3), pages 923-940, June.
    17. Blakeley B. McShane & David Gal, 2016. "Blinding Us to the Obvious? The Effect of Statistical Training on the Evaluation of Evidence," Management Science, INFORMS, vol. 62(6), pages 1707-1718, June.
    18. Martin Bichler & Ulrich Frank & David Avison & Julien Malaurent & Peter Fettke & Dirk Hovorka & Jan Krämer & Daniel Schnurr & Benjamin Müller & Leena Suhl & Bernhard Thalheim, 2016. "Theories in Business and Information Systems Engineering," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 58(4), pages 291-319, August.
    19. Arjen Witteloostuijn, 2020. "New-day statistical thinking: A bold proposal for a radical change in practices," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 51(2), pages 274-278, March.
    20. Gerrit Anton De Waal & Paul Knott, 2019. "Npd Tools, Thoroughness And Performance In Small Firms," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 23(06), pages 1-26, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:socres:v:21:y:2016:i:1:p:102-115. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.