IDEAS home Printed from https://ideas.repec.org/a/gam/jstats/v6y2023i2p35-551d1132637.html
   My bibliography  Save this article

Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions

Author

Listed:
  • Eleni Verykouki

    (Laboratory of Biometry, Department of Agriculture Crop Production and Rural Environment, School of Agricultural Sciences, University of Thessaly, Fytokou Street, 38446 Volos, Greece
    Laboratory of Entomology and Agricultural Zoology, Department of Agriculture Crop Production and Rural Environment, School of Agricultural Sciences, University of Thessaly, Fytokou Street, 38446 Volos, Greece)

  • Christos T. Nakas

    (Laboratory of Biometry, Department of Agriculture Crop Production and Rural Environment, School of Agricultural Sciences, University of Thessaly, Fytokou Street, 38446 Volos, Greece
    Institute of Clinical Chemistry, Inselspital, Bern University Hospital, University of Bern, 3010 Bern, Switzerland)

Abstract

P -values have played a central role in the advancement of research in virtually all scientific fields; however, there has been significant controversy over their use. “The ASA president’s task force statement on statistical significance and replicability ” has provided a solid basis for resolving the quarrel, but although the significance part is clearly dealt with, the replicability part raises further discussions. Given the clear statement regarding significance, in this article, we consider the validity of p -value use for statistical inference as de facto . We briefly review the bibliography regarding the relevant controversy in recent years and illustrate how already proposed approaches, or slight adaptations thereof, can be readily implemented to address both significance and reproducibility, adding credibility to empirical study findings. The definitions used for the notions of replicability and reproducibility are also clearly described. We argue that any p -value must be reported along with its corresponding s-value followed by ( 1 − α ) % confidence intervals and the rejection replication index.

Suggested Citation

  • Eleni Verykouki & Christos T. Nakas, 2023. "Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions," Stats, MDPI, vol. 6(2), pages 1-13, April.
    • Handle: RePEc:gam:jstats:v:6:y:2023:i:2:p:35-551:d:1132637
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-905X/6/2/35/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-905X/6/2/35/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sander Greenland, 2019. "Valid P-Values Behave Exactly as They Should: Some Misleading Criticisms of P-Values and Their Resolution With S-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 106-114, March.
    2. Daniel J. Benjamin & James O. Berger, 2019. "Three Recommendations for Improving the Use of p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 186-191, March.
    3. William M. Briggs, 2017. "The Substitute for -Values," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 897-898, July.
    4. Kenneth Rice & Lingbo Ye, 2022. "Expressing Regret: A Unified View of Credible Intervals," The American Statistician, Taylor & Francis Journals, vol. 76(3), pages 248-256, July.
    5. Regina Nuzzo, 2014. "Scientific method: Statistical errors," Nature, Nature, vol. 506(7487), pages 150-152, February.
    6. Gerhard Marinell & Gabriele Steckel-Berger & Hanno Ulmer, 2012. "Not Significant: What Now?," Journal of Probability and Statistics, Hindawi, vol. 2012, pages 1-6, November.
    7. Eric B. Laber & Kerby Shedden, 2017. "Statistical Significance and the Dichotomization of Evidence: The Relevance of the ASA Statement on Statistical Significance and p-Values for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 902-904, July.
    8. Margaret Sullivan Pepe & Tianxi Cai, 2004. "The Analysis of Placement Values for Evaluating Discriminatory Measures," Biometrics, The International Biometric Society, vol. 60(2), pages 528-535, June.
    9. Browne, Richard H., 2010. "The t-Test p Value and Its Relationship to the Effect Size and P(X>Y)," The American Statistician, American Statistical Association, vol. 64(1), pages 30-33.
    10. Yi Zuo & Thomas G. Stewart & Jeffrey D. Blume, 2022. "Variable Selection With Second-Generation P-Values," The American Statistician, Taylor & Francis Journals, vol. 76(2), pages 91-101, April.
    11. Steven N. Goodman, 2019. "Why is Getting Rid of P-Values So Hard? Musings on Science and Statistics," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 26-30, March.
    12. Megan L Head & Luke Holman & Rob Lanfear & Andrew T Kahn & Michael D Jennions, 2015. "The Extent and Consequences of P-Hacking in Science," PLOS Biology, Public Library of Science, vol. 13(3), pages 1-15, March.
    13. Todd A. Kuffner & Stephen G. Walker, 2019. "Why are p-Values Controversial?," The American Statistician, Taylor & Francis Journals, vol. 73(1), pages 1-3, January.
    14. Bhattacharya B. & Habtzghi D., 2002. "Median of the p Value Under the Alternative Hypothesis," The American Statistician, American Statistical Association, vol. 56, pages 202-206, August.
    15. Sylvia Richardson, 2022. "Statistics in times of increasing uncertainty," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1471-1496, October.
    16. Margaret Pepe & Holly Janes & Gary Longton & Wendy Leisenring & Polly Newcomb, 2004. "Limitations of the Odds Ratio in Gauging the Performance of a Diagnostic or Prognostic Marker," UW Biostatistics Working Paper Series 1035, Berkeley Electronic Press.
    17. De Martini, Daniele, 2008. "Reproducibility probability estimation for testing statistical hypotheses," Statistics & Probability Letters, Elsevier, vol. 78(9), pages 1056-1061, July.
    18. David J. Hand, 2022. "Trustworthiness of statistical inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(1), pages 329-347, January.
    19. Sven-Kristjan Bormann, 2022. "A Stata implementation of second-generation p-values," Stata Journal, StataCorp LP, vol. 22(3), pages 496-520, September.
    20. William M. Goodman & Susan E. Spruill & Eugene Komaroff, 2019. "A Proposed Hybrid Effect Size Plus p-Value Criterion: Empirical Evidence Supporting its Use," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 168-185, March.
    21. Martin Bland, 2013. "Do Baseline P-Values Follow a Uniform Distribution in Randomised Trials?," PLOS ONE, Public Library of Science, vol. 8(10), pages 1-5, October.
    22. Blakeley B. McShane & David Gal, 2017. "Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 885-895, July.
    23. Haolun Shi & Guosheng Yin, 2021. "Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests," The American Statistician, Taylor & Francis Journals, vol. 75(3), pages 265-275, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin E Héroux & Janet L Taylor & Simon C Gandevia, 2015. "The Use and Abuse of Transcranial Magnetic Stimulation to Modulate Corticospinal Excitability in Humans," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-10, December.
    2. Cheng, Yuanyuan, 2023. "A method of 3R to evaluate the correlation and predictive value of variables," OSF Preprints c79tu, Center for Open Science.
    3. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    4. J. M. Bauer & L. A. Reisch, 2019. "Behavioural Insights and (Un)healthy Dietary Choices: a Review of Current Evidence," Journal of Consumer Policy, Springer, vol. 42(1), pages 3-45, March.
    5. Arjen Witteloostuijn, 2020. "New-day statistical thinking: A bold proposal for a radical change in practices," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 51(2), pages 274-278, March.
    6. Pütz, Peter & Kramer-Sunderbrink, Arne & Dreher, Robin Tim & Hoffmann, Leona & Werner, Robin, 2022. "A Proposed Hybrid Effect Size Plus p-Value Criterion. A Comment on Goodman et al. (The American Statistician, 2019)," Journal of Comments and Replications in Economics (JCRE), ZBW - Leibniz Information Centre for Economics, vol. 1(2022-4), pages 1-15.
    7. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
    8. Blakeley B. McShane & David Gal, 2017. "Rejoinder: Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 904-908, July.
    9. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.
    10. Emilyane de Oliveira Santana Amaral & Sergio Roberto Peres Line, 2021. "Current use of effect size or confidence interval analyses in clinical and biomedical research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9133-9145, November.
    11. Debashis Ghosh & Michael S. Sabel, 2022. "A Weighted Sample Framework to Incorporate External Calculators for Risk Modeling," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(3), pages 363-379, December.
    12. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.
    13. Bibhas Chakraborty, 2020. "Statistical Remedies for Medical Researchers," International Statistical Review, International Statistical Institute, vol. 88(3), pages 802-804, December.
    14. Y. Huang & M. S. Pepe, 2009. "A Parametric ROC Model-Based Approach for Evaluating the Predictiveness of Continuous Markers in Case–Control Studies," Biometrics, The International Biometric Society, vol. 65(4), pages 1133-1144, December.
    15. Aljoscha Benjamin Hwang & Guido Schuepfer & Mario Pietrini & Stefan Boes, 2021. "External validation of EPIC’s Risk of Unplanned Readmission model, the LACE+ index and SQLape as predictors of unplanned hospital readmissions: A monocentric, retrospective, diagnostic cohort study in," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-33, November.
    16. Anna-Karin Ivert & Marie Torstensson Levander & Juan Merlo, 2013. "Adolescents' Utilisation of Psychiatric Care, Neighbourhoods and Neighbourhood Socioeconomic Deprivation: A Multilevel Analysis," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    17. Abel Brodeur, Nikolai M. Cook, Anthony Heyes, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," LCERPA Working Papers am0133, Laurier Centre for Economic Research and Policy Analysis.
    18. Margaret Sullivan Pepe & Tianxi Cai & Gary Longton, 2006. "Combining Predictors for Classification Using the Area under the Receiver Operating Characteristic Curve," Biometrics, The International Biometric Society, vol. 62(1), pages 221-229, March.
    19. Gunnarsson, Björn Rafn & vanden Broucke, Seppe & Baesens, Bart & Óskarsdóttir, María & Lemahieu, Wilfried, 2021. "Deep learning for credit scoring: Do or don’t?," European Journal of Operational Research, Elsevier, vol. 295(1), pages 292-305.
    20. Jasper Brinkerink, 2023. "When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research," Entrepreneurship Theory and Practice, , vol. 47(2), pages 304-343, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jstats:v:6:y:2023:i:2:p:35-551:d:1132637. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.