IDEAS home Printed from https://ideas.repec.org/a/spr/testjl/v31y2022i3d10.1007_s11749-022-00803-4.html
   My bibliography  Save this article

Increasing the replicability for linear models via adaptive significance levels

Author

Listed:
  • D. Vélez

    (University of Puerto Rico)

  • M. E. Pérez

    (University of Puerto Rico)

  • L. R. Pericchi

    (University of Puerto Rico)

Abstract

We put forward an adaptive $$\alpha $$ α (type I error) that decreases as the information grows for hypothesis tests comparing nested linear models. A less elaborate adaptation was presented in Pérez and Pericchi (Stat Probab Lett 85:20–24, 2014) for general i.i.d. models. The calibration proposed in this paper may be interpreted as a Bayes–non-Bayes compromise, of a simple translation of a Bayes factor on frequentist terms that leads to statistical consistency, and most importantly, it is a step toward statistics that promotes replicable scientific findings.

Suggested Citation

  • D. Vélez & M. E. Pérez & L. R. Pericchi, 2022. "Increasing the replicability for linear models via adaptive significance levels," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(3), pages 771-789, September.
    • Handle: RePEc:spr:testjl:v:31:y:2022:i:3:d:10.1007_s11749-022-00803-4
    DOI: 10.1007/s11749-022-00803-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11749-022-00803-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11749-022-00803-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richter, W. -D. & Schumacher, J., 2000. "Asymptotic Expansions for Large Deviation Probabilities of Noncentral Generalized Chi-Square Distributions," Journal of Multivariate Analysis, Elsevier, vol. 75(2), pages 184-218, November.
    2. Sellke T. & Bayarri M. J. & Berger J. O., 2001. "Calibration of rho Values for Testing Precise Null Hypotheses," The American Statistician, American Statistical Association, vol. 55, pages 62-71, February.
    3. David Findley, 1991. "Counterexamples to parsimony and BIC," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 43(3), pages 505-514, September.
    4. Pérez, María-Eglée & Pericchi, Luis Raúl, 2014. "Changing statistical significance with the amount of information: The adaptive α significance level," Statistics & Probability Letters, Elsevier, vol. 85(C), pages 20-24.
    5. M. J. Bayarri & James O. Berger & Woncheol Jang & Surajit Ray & Luis R. Pericchi & Ingmar Visser, 2019. "Prior-based Bayesian information criterion," Statistical Theory and Related Fields, Taylor & Francis Journals, vol. 3(1), pages 2-13, January.
    6. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    7. Valen E. Johnson & David Rossell, 2010. "On the use of non‐local prior densities in Bayesian hypothesis tests," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(2), pages 143-170, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.
    2. Kim, Jae H., 2017. "Stock returns and investors' mood: Good day sunshine or spurious correlation?," International Review of Financial Analysis, Elsevier, vol. 52(C), pages 94-103.
    3. Kelter, Riko, 2022. "Power analysis and type I and type II error rates of Bayesian nonparametric two-sample tests for location-shifts based on the Bayes factor under Cauchy priors," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).
    4. Julia Roloff & Michael J. Zyphur, 2019. "Null Findings, Replications and Preregistered Studies in Business Ethics Research," Journal of Business Ethics, Springer, vol. 160(3), pages 609-619, December.
    5. Pathairat Pastpipatkul & Petchaluck Boonyakunakorn & Kanyaphon Phetsakda, 2020. "The Impact of Thailand’s Openness on Bilateral Trade between Thailand and Japan: Copula-Based Markov Switching Seemingly Unrelated Regression Model," Economies, MDPI, vol. 8(1), pages 1-13, January.
    6. Eugenio Melilli & Piero Veronese, 2024. "Confidence distributions and hypothesis testing," Statistical Papers, Springer, vol. 65(6), pages 3789-3820, August.
    7. Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
    8. Mayo, Deborah & Morey, Richard Donald, 2017. "A Poor Prognosis for the Diagnostic Screening Critique of Statistical Tests," OSF Preprints ps38b, Center for Open Science.
    9. Fetene B. Tekle & Dereje W. Gudicha & Jeroen K. Vermunt, 2016. "Power analysis for the bootstrap likelihood ratio test for the number of classes in latent class models," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 10(2), pages 209-224, June.
    10. Riccardo (Jack) Lucchetti & Luca Pedini, 2020. "ParMA: Parallelised Bayesian Model Averaging for Generalised Linear Models," Working Papers 2020:28, Department of Economics, University of Venice "Ca' Foscari".
    11. Gary Koop & Roberto Leon-Gonzalez & Rodney Strachan, 2008. "Bayesian inference in a cointegrating panel data model," Advances in Econometrics, in: Bayesian Econometrics, pages 433-469, Emerald Group Publishing Limited.
    12. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    13. Chatelain, Jean-Bernard & Ralf, Kirsten, 2018. "Publish and Perish: Creative Destruction and Macroeconomic Theory," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 46(2), pages 65-101.
    14. Segurado, Pedro & Gutiérrez-Cánovas, Cayetano & Ferreira, Teresa & Branco, Paulo, 2022. "Stressor gradient coverage affects interaction identification," Ecological Modelling, Elsevier, vol. 472(C).
    15. Uwe Hassler & Marc‐Oliver Pohle, 2022. "Unlucky Number 13? Manipulating Evidence Subject to Snooping," International Statistical Review, International Statistical Institute, vol. 90(2), pages 397-410, August.
    16. Gergely Ganics & Atsushi Inoue & Barbara Rossi, 2021. "Confidence Intervals for Bias and Size Distortion in IV and Local Projections-IV Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 307-324, January.
    17. Hirschauer, Norbert & Grüner, Sven & Mußhoff, Oliver & Becker, Claudia & Jantsch, Antje, 2020. "Can p-values be meaningfully interpreted without random sampling?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14, pages 71-91.
    18. Oliver Schilke & Sheen S. Levine & Olenka Kacperczyk & Lynne G. Zucker, 2019. "Call for Papers-Special Issue on Experiments in Organizational Theory," Organization Science, INFORMS, vol. 30(1), pages 232-234, February.
    19. Bachmann, Dirk & Dette, Holger, 2004. "A note on the Bickel-Rosenblatt test in autoregressive time series," Technical Reports 2004,17, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    20. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:testjl:v:31:y:2022:i:3:d:10.1007_s11749-022-00803-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.