IDEAS home Printed from https://ideas.repec.org/a/bla/scjsta/v50y2023i3p899-914.html
   My bibliography  Save this article

Connecting simple and precise P‐values to complex and ambiguous realities (includes rejoinder to comments on “Divergence vs. decision P‐values”)

Author

Listed:
  • Sander Greenland

Abstract

Mathematics is a limited component of solutions to real‐world problems, as it expresses only what is expected to be true if all our assumptions are correct, including implicit assumptions that are omnipresent and often incorrect. Statistical methods are rife with implicit assumptions whose violation can be life‐threatening when results from them are used to set policy. Among them are that there is human equipoise or unbiasedness in data generation, management, analysis, and reporting. These assumptions correspond to levels of cooperation, competence, neutrality, and integrity that are absent more often than we would like to believe. Given this harsh reality, we should ask what meaning, if any, we can assign to the P‐values, “statistical significance” declarations, “confidence” intervals, and posterior probabilities that are used to decide what and how to present (or spin) discussions of analyzed data. By themselves, P‐values and CI do not test any hypothesis, nor do they measure the significance of results or the confidence we should have in them. The sense otherwise is an ongoing cultural error perpetuated by large segments of the statistical and research community via misleading terminology. So‐called inferential statistics can only become contextually interpretable when derived explicitly from causal stories about the real data generator (such as randomization), and can only become reliable when those stories are based on valid and public documentation of the physical mechanisms that generated the data. Absent these assurances, traditional interpretations of statistical results become pernicious fictions that need to be replaced by far more circumspect descriptions of data and model relations.

Suggested Citation

  • Sander Greenland, 2023. "Connecting simple and precise P‐values to complex and ambiguous realities (includes rejoinder to comments on “Divergence vs. decision P‐values”)," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(3), pages 899-914, September.
    • Handle: RePEc:bla:scjsta:v:50:y:2023:i:3:p:899-914
    DOI: 10.1111/sjos.12645
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/sjos.12645
    Download Restriction: no
    File URL: https://libkey.io/10.1111/sjos.12645?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sander Greenland, 2019. "Valid P-Values Behave Exactly as They Should: Some Misleading Criticisms of P-Values and Their Resolution With S-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 106-114, March.
    2. Blakeley B. McShane & David Gal & Andrew Gelman & Christian Robert & Jennifer L. Tackett, 2019. "Abandon Statistical Significance," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 235-245, March.
    3. Valentin Amrhein & David Trafimow & Sander Greenland, 2019. "Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 262-270, March.
    4. Noah N. N. van Dongen & Johnny B. van Doorn & Quentin F. Gronau & Don van Ravenzwaaij & Rink Hoekstra & Matthias N. Haucke & Daniel Lakens & Christian Hennig & Richard D. Morey & Saskia Homer & Andrew, 2019. "Multiple Perspectives on Inference for Two Simple Statistical Scenarios," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 328-339, March.
    5. Steven N. Goodman, 2019. "Why is Getting Rid of P-Values So Hard? Musings on Science and Statistics," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 26-30, March.
    6. David B. Allison & Andrew W. Brown & Brandon J. George & Kathryn A. Kaiser, 2016. "Reproducibility: A tragedy of errors," Nature, Nature, vol. 530(7588), pages 27-29, February.
    7. Valentin Amrhein & Sander Greenland & Blake McShane, 2019. "Scientists rise up against statistical significance," Nature, Nature, vol. 567(7748), pages 305-307, March.
    8. Michael Lavine, 2019. "Frequentist, Bayes, or Other?," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 312-318, March.
    9. Sander Greenland, 2001. "Sensitivity Analysis, Monte Carlo Risk Analysis, and Bayesian Uncertainty Assessment," Risk Analysis, John Wiley & Sons, vol. 21(4), pages 579-584, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    2. Arjen Witteloostuijn, 2020. "New-day statistical thinking: A bold proposal for a radical change in practices," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 51(2), pages 274-278, March.
    3. Keith R Lohse & Kristin L Sainani & J Andrew Taylor & Michael L Butson & Emma J Knight & Andrew J Vickers, 2020. "Systematic review of the use of “magnitude-based inference” in sports science and medicine," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-22, June.
    4. Smriti Kumar & Elizabeth G. Miller & Martin Mende & Maura L. Scott, 2022. "Language matters: humanizing service robots through the use of language during the COVID-19 pandemic," Marketing Letters, Springer, vol. 33(4), pages 607-623, December.
    5. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    6. Helmut Wasserbacher & Martin Spindler, 2024. "Credit Ratings: Heterogeneous Effect on Capital Structure," Papers 2406.18936, arXiv.org.
    7. Austin Chia & Margaret L. Kern, 2021. "Subjective Wellbeing and the Social Responsibilities of Business: an Exploratory Investigation of Australian Perspectives," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 16(5), pages 1881-1908, October.
    8. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    9. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
    10. Sander Greenland, 2023. "Divergence versus decision P‐values: A distinction worth making in theory and keeping in practice: Or, how divergence P‐values measure evidence even when decision P‐values do not," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(1), pages 54-88, March.
    11. Wagenmakers, Eric-Jan & Sarafoglou, Alexandra & Aarts, Sil Dr. & Albers, Casper J & Algermissen, Johannes & Bahník, Štěpán & van Dongen, Noah N'Djaye Nikolai & Hoekstra, Rink & Moreau, David & van Rav, 2021. "Toward More Transparency in Statistical Practice," MetaArXiv t93cg, Center for Open Science.
    12. Eleni Verykouki & Christos T. Nakas, 2023. "Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions," Stats, MDPI, vol. 6(2), pages 1-13, April.
    13. Johnstone, David, 2022. "Accounting research and the significance test crisis," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 89(C).
    14. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    15. repec:hal:journl:hal-04791500 is not listed on IDEAS
    16. Mia Papasideris & Scott T Leatherdale & Kate Battista & Peter A Hall, 2021. "An examination of the prospective association between physical activity and academic achievement in youth at the population level," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-15, June.
    17. Surhan Cam & Serap Palaz, 2023. "Mutual interests management with a purposive approach: Evidence from the Turkish shipyards for an amorphous impact model between (subjective) well‐being and performance," Industrial Relations Journal, Wiley Blackwell, vol. 54(1), pages 40-70, January.
    18. Michael Essman & Lindsey Smith Taillie & Tamryn Frank & Shu Wen Ng & Barry M Popkin & Elizabeth C Swart, 2021. "Taxed and untaxed beverage intake by South African young adults after a national sugar-sweetened beverage tax: A before-and-after study," PLOS Medicine, Public Library of Science, vol. 18(5), pages 1-17, May.
    19. Siddharth Sareen & Andrea Saltelli & Kjetil Rommetveit, 2020. "Ethics of quantification: illumination, obfuscation and performative legitimation," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-5, December.
    20. Zachary Van Winkle & Anette Fasang, 2021. "The complexity of employment and family life courses across 20th century Europe: More evidence for larger cross-national differences but little change across 1916‒1966 birth cohorts," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 44(32), pages 775-810.
    21. Francesco De Pretis & Barbara Osimani, 2019. "New Insights in Computational Methods for Pharmacovigilance: E-Synthesis , a Bayesian Framework for Causal Assessment," IJERPH, MDPI, vol. 16(12), pages 1-19, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:scjsta:v:50:y:2023:i:3:p:899-914. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0303-6898 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.