IDEAS home Printed from https://ideas.repec.org/a/sae/socres/v21y2016i1p102-115.html
   My bibliography  Save this article

Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong

Author

Listed:
  • Stephen Gorard

Abstract

This paper reminds readers of the absurdity of statistical significance testing, despite its continued widespread use as a supposed method for analysing numeric data. There have been complaints about the poor quality of research employing significance tests for a hundred years, and repeated calls for researchers to stop using and reporting them. There have even been attempted bans. Many thousands of papers have now been written, in all areas of research, explaining why significance tests do not work. There are too many for all to be cited here. This paper summarises the logical problems as described in over 100 of these prior pieces. It then presents a series of demonstrations showing that significance tests do not work in practice. In fact, they are more likely to produce the wrong answer than a right one. The confused use of significance testing has practical and damaging consequences for people's lives. Ending the use of significance tests is a pressing ethical issue for research. Anyone knowing the problems, as described over one hundred years, who continues to teach, use or publish significance tests is acting unethically, and knowingly risking the damage that ensues.

Suggested Citation

  • Stephen Gorard, 2016. "Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong," Sociological Research Online, , vol. 21(1), pages 102-115, February.
    • Handle: RePEc:sae:socres:v:21:y:2016:i:1:p:102-115
    DOI: 10.5153/sro.3857
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.5153/sro.3857
    Download Restriction: no
    File URL: https://libkey.io/10.5153/sro.3857?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Athena Engman, 2013. "Is there life after P>0.05? Statistical significance and quantitative sociology," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(1), pages 257-270, January.
    2. Marks R. Nester, 1996. "An Applied Statistician's Creed," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 45(4), pages 401-410, December.
    3. Andreas Schwab & Eric Abrahamson & William H. Starbuck & Fiona Fidler, 2011. "PERSPECTIVE---Researchers Should Make Thoughtful Assessments Instead of Null-Hypothesis Significance Tests," Organization Science, INFORMS, vol. 22(4), pages 1105-1120, August.
    4. Louis Guttman, 1985. "The illogic of statistical inference for cumulative science," Applied Stochastic Models and Data Analysis, John Wiley & Sons, vol. 1(1), pages 3-9.
    5. Stephen Gorard, 2003. "Understanding Probabilities and Re-Considering Traditional Research Training," Sociological Research Online, , vol. 8(1), pages 104-112, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Stephen Gorard, 2017. "Significance Testing is Still Wrong, and Damages Real Lives: A Brief Reply to Spreckelsen and Van Der Horst, and Nicholson and McCusker," Sociological Research Online, , vol. 22(2), pages 204-210, May.
    2. James Nicholson & Sean Mccusker, 2016. "Damaging the Case for Improving Social Science Methodology through Misrepresentation: Re-Asserting Confidence in Hypothesis Testing as a Valid Scientific Process," Sociological Research Online, , vol. 21(2), pages 136-147, May.
    3. Thees F Spreckelsen & Mariska Van Der Horst, 2016. "Is Banning Significance Testing the Best Way to Improve Applied Social Science Research? – Questions on Gorard (2016)," Sociological Research Online, , vol. 21(3), pages 95-105, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Katrin Zulauf & Ralf Wagner, 2023. "Countering Negotiation Power Asymmetries by Using the Adjusted Winner Algorithm," SN Operations Research Forum, Springer, vol. 4(1), pages 1-20, March.
    2. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    3. Marc Orlitzky, 2011. "Institutionalized dualism: statistical significance testing as myth and ceremony," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 22(1), pages 47-77, September.
    4. Lindsay, R. Murray, 1995. "Reconsidering the status of tests of significance: An alternative criterion of adequacy," Accounting, Organizations and Society, Elsevier, vol. 20(1), pages 35-53, January.
    5. Andrew Briggs & Richard Nixon & Simon Dixon & Simon Thompson, 2005. "Parametric modelling of cost data: some simulation evidence," Health Economics, John Wiley & Sons, Ltd., vol. 14(4), pages 421-428, April.
    6. Norman Fickel, 2001. "Sequential Regression: A Neodescriptive Approach to Multicollinearity," EERI Research Paper Series EERI_RP_2001_09, Economics and Econometrics Research Institute (EERI), Brussels.
    7. Jerker Denrell & Christina Fang & Chengwei Liu, 2015. "Perspective—Chance Explanations in the Management Sciences," Organization Science, INFORMS, vol. 26(3), pages 923-940, June.
    8. Arjen Witteloostuijn, 2020. "New-day statistical thinking: A bold proposal for a radical change in practices," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 51(2), pages 274-278, March.
    9. Gerrit Anton De Waal & Paul Knott, 2019. "Npd Tools, Thoroughness And Performance In Small Firms," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 23(06), pages 1-26, August.
    10. Le Breton-Miller, Isabelle & Miller, Danny, 2023. "Contradiction and disaggregation for family firm research," Journal of Family Business Strategy, Elsevier, vol. 14(1).
    11. Anand Desai, 2008. "Quantitative methods, economics, and or models," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(3), pages 640-669.
    12. Norman Fickel, 2000. "Sequential Regression: A Neodescriptive Approach to Multicollinearity," Econometrics 0004009, University Library of Munich, Germany.
    13. Block, Joern H. & Jaskiewicz, Peter & Miller, Danny, 2011. "Ownership versus management effects on performance in family and founder companies: A Bayesian reconciliation," Journal of Family Business Strategy, Elsevier, vol. 2(4), pages 232-245.
    14. Hannah Fraser & Tim Parker & Shinichi Nakagawa & Ashley Barnett & Fiona Fidler, 2018. "Questionable research practices in ecology and evolution," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-16, July.
    15. Schneider, Jesper W., 2013. "Caveats for using statistical significance tests in research assessments," Journal of Informetrics, Elsevier, vol. 7(1), pages 50-62.
    16. Jesper W. Schneider, 2015. "Null hypothesis significance tests. A mix-up of two different theories: the basis for widespread confusion and numerous misinterpretations," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 411-432, January.
    17. Tomas Macak, 2021. "Stability of Dependencies of Contingent Subgroups with Merged Groups: Vaccination Case Study," Mathematics, MDPI, vol. 9(22), pages 1-12, November.
    18. Peter Jaskiewicz & Joern H. Block & James G. Combs & Danny Miller, 2017. "The Effects of Founder and Family Ownership on Hired CEOs’ Incentives and Firm Performance," Entrepreneurship Theory and Practice, , vol. 41(1), pages 73-103, January.
    19. Dybro Liengaard, Benjamin, 2024. "Measurement invariance testing in partial least squares structural equation modeling," Journal of Business Research, Elsevier, vol. 177(C).
    20. Christopher Hansen & Joern Block & Matthias Neuenkirch, 2020. "Family Firm Performance Over The Business Cycle: A Meta‐Analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 476-511, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:socres:v:21:y:2016:i:1:p:102-115. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.