IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0213715.html
   My bibliography  Save this article

Studentized bootstrap model-averaged tail area intervals

Author

Listed:
  • Jiaxu Zeng
  • David Fletcher
  • Peter W Dillingham
  • Christopher E Cornwall

Abstract

In many scientific studies, the underlying data-generating process is unknown and multiple statistical models are considered to describe it. For example, in a factorial experiment we might consider models involving just main effects, as well as those that include interactions. Model-averaging is a commonly-used statistical technique to allow for model uncertainty in parameter estimation. In the frequentist setting, the model-averaged estimate of a parameter is a weighted mean of the estimates from the individual models, with the weights typically being based on an information criterion, cross-validation, or bootstrapping. One approach to building a model-averaged confidence interval is to use a Wald interval, based on the model-averaged estimate and its standard error. This has been the default method in many application areas, particularly those in the life sciences. The MA-Wald interval, however, assumes that the studentized model-averaged estimate has a normal distribution, which can be far from true in practice due to the random, data-driven model weights. Recently, the model-averaged tail area Wald interval (MATA-Wald) has been proposed as an alternative to the MA-Wald interval, which only assumes that the studentized estimate from each model has a N(0, 1) or t-distribution, when that model is true. This alternative to the MA-Wald interval has been shown to have better coverage in simulation studies. However, when we have a response variable that is skewed, even these relaxed assumptions may not be valid, and use of these intervals might therefore result in poor coverage. We propose a new interval (MATA-SBoot) which uses a parametric bootstrap approach to estimate the distribution of the studentized estimate for each model, when that model is true. This method only requires that the studentized estimate from each model is approximately pivotal, an assumption that will often be true in practice, even for skewed data. We illustrate use of this new interval in the analysis of a three-factor marine global change experiment in which the response variable is assumed to have a lognormal distribution. We also perform a simulation study, based on the example, to compare the lower and upper error rates of this interval with those for existing methods. The results suggest that the MATA-SBoot interval can provide better error rates than existing intervals when we have skewed data, particularly for the upper error rate when the sample size is small.

Suggested Citation

  • Jiaxu Zeng & David Fletcher & Peter W Dillingham & Christopher E Cornwall, 2019. "Studentized bootstrap model-averaged tail area intervals," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-16, March.
  • Handle: RePEc:plo:pone00:0213715
    DOI: 10.1371/journal.pone.0213715
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0213715
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0213715&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0213715?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Paul Kabaila & A. H. Welsh & Waruni Abeysekera, 2016. "Model-Averaged Confidence Intervals," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(1), pages 35-48, March.
    2. Fletcher, David & Dillingham, Peter W., 2011. "Model-averaged confidence intervals for factorial experiments," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 3041-3048, November.
    3. Chris Chatfield, 1995. "Model Uncertainty, Data Mining and Statistical Inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(3), pages 419-444, May.
    4. Paul Lukacs & Kenneth Burnham & David Anderson, 2010. "Model selection bias and Freedman’s paradox," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 62(1), pages 117-125, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael Schomaker & Christian Heumann, 2020. "When and when not to use optimal model averaging," Statistical Papers, Springer, vol. 61(5), pages 2221-2240, October.
    2. Shaobo Jin, 2022. "Frequentist Model Averaging in Structure Equation Model With Ordinal Data," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1130-1145, September.
    3. Fletcher, David & Dillingham, Peter W., 2011. "Model-averaged confidence intervals for factorial experiments," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 3041-3048, November.
    4. Paul Kabaila & A. H. Welsh & Waruni Abeysekera, 2016. "Model-Averaged Confidence Intervals," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(1), pages 35-48, March.
    5. Shaobo Jin & Sebastian Ankargren, 2019. "Frequentist Model Averaging in Structural Equation Modelling," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 84-104, March.
    6. Turek, Daniel & Fletcher, David, 2012. "Model-averaged Wald confidence intervals," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2809-2815.
    7. Schomaker, Michael & Heumann, Christian, 2014. "Model selection and model averaging after multiple imputation," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 758-770.
    8. Claudia García-García & Catalina B. García-García & Román Salmerón, 2021. "Confronting collinearity in environmental regression models: evidence from world data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(3), pages 895-926, September.
    9. Chou, Ping & Chuang, Howard Hao-Chun & Chou, Yen-Chun & Liang, Ting-Peng, 2022. "Predictive analytics for customer repurchase: Interdisciplinary integration of buy till you die modeling and machine learning," European Journal of Operational Research, Elsevier, vol. 296(2), pages 635-651.
    10. Sai Ding & John Knight, 2011. "Why has China Grown So Fast? The Role of Physical and Human Capital Formation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 73(2), pages 141-174, April.
    11. Riccardo (Jack) Lucchetti & Luca Pedini, 2020. "ParMA: Parallelised Bayesian Model Averaging for Generalised Linear Models," Working Papers 2020:28, Department of Economics, University of Venice "Ca' Foscari".
    12. Robert Lehmann & Antje Weyh, 2016. "Forecasting Employment in Europe: Are Survey Results Helpful?," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 12(1), pages 81-117, September.
    13. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    14. Lee, Yun Shin & Scholtes, Stefan, 2014. "Empirical prediction intervals revisited," International Journal of Forecasting, Elsevier, vol. 30(2), pages 217-234.
    15. Johan Verbeeck & Martin Geroldinger & Konstantin Thiel & Andrew Craig Hooker & Sebastian Ueckert & Mats Karlsson & Arne Cornelius Bathke & Johann Wolfgang Bauer & Geert Molenberghs & Georg Zimmermann, 2023. "How to analyze continuous and discrete repeated measures in small‐sample cross‐over trials?," Biometrics, The International Biometric Society, vol. 79(4), pages 3998-4011, December.
    16. Coleman, Stephen, 2005. "Testing Theories with Qualitative and Quantitative Predictions," MPRA Paper 105171, University Library of Munich, Germany.
    17. Ewout W. Steyerberg, 2005. "Local Applicability of Clinical and Model-Based Probability Estimates," Medical Decision Making, , vol. 25(6), pages 678-680, November.
    18. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    19. Brooks, Jeremy S., 2010. "The Buddha mushroom: Conservation behavior and the development of institutions in Bhutan," Ecological Economics, Elsevier, vol. 69(4), pages 779-795, February.
    20. Ebersberger, Bernd & Galia, Fabrice & Laursen, Keld & Salter, Ammon, 2021. "Inbound Open Innovation and Innovation Performance: A Robustness Study," Research Policy, Elsevier, vol. 50(7).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0213715. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.