IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i1d10.1007_s11336-022-09899-x.html
   My bibliography  Save this article

Accurate Confidence and Bayesian Interval Estimation for Non-centrality Parameters and Effect Size Indices

Author

Listed:
  • Kaidi Kang

    (Vanderbilt University)

  • Megan T. Jones

    (Vanderbilt University)

  • Kristan Armstrong

    (Vanderbilt University Medical Center)

  • Suzanne Avery

    (Vanderbilt University Medical Center)

  • Maureen McHugo

    (Vanderbilt University Medical Center)

  • Stephan Heckers

    (Vanderbilt University Medical Center)

  • Simon Vandekar

    (Vanderbilt University)

Abstract

Reporting effect size index estimates with their confidence intervals (CIs) can be an excellent way to simultaneously communicate the strength and precision of the observed evidence. We recently proposed a robust effect size index (RESI) that is advantageous over common indices because it’s widely applicable to different types of data. Here, we use statistical theory and simulations to develop and evaluate RESI estimators and confidence/credible intervals that rely on different covariance estimators. Our results show (1) counter to intuition, the randomness of covariates reduces coverage for Chi-squared and F CIs; (2) when the variance of the estimators is estimated, the non-central Chi-squared and F CIs using the parametric and robust RESI estimators fail to cover the true effect size at the nominal level. Using the robust estimator along with the proposed nonparametric bootstrap or Bayesian (credible) intervals provides valid inference for the RESI, even when model assumptions may be violated. This work forms a unified effect size reporting procedure, such that effect sizes with confidence/credible intervals can be easily reported in an analysis of variance (ANOVA) table format.

Suggested Citation

  • Kaidi Kang & Megan T. Jones & Kristan Armstrong & Suzanne Avery & Maureen McHugo & Stephan Heckers & Simon Vandekar, 2023. "Accurate Confidence and Bayesian Interval Estimation for Non-centrality Parameters and Effect Size Indices," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 253-273, March.
    • Handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09899-x
    DOI: 10.1007/s11336-022-09899-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09899-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-022-09899-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ronald L. Wasserstein & Allen L. Schirm & Nicole A. Lazar, 2019. "Moving to a World Beyond “p," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 1-19, March.
    2. Zhiyi Zhang & Nancy Schoeps, 1997. "On robust estimation of effect size under semiparametric models," Psychometrika, Springer;The Psychometric Society, vol. 62(2), pages 201-214, June.
    3. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "Erratum to: A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 946-946, December.
    4. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 232-246, March.
    5. J. Scott Long & Jeremy Freese, 2006. "Regression Models for Categorical Dependent Variables using Stata, 2nd Edition," Stata Press books, StataCorp LP, edition 2, number long2, March.
    6. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 232-246, March.
    2. Jinsuk Yang & Qing Hao & Mahmut Yaşar, 2023. "Institutional investors and cross‐border mergers and acquisitions: The 2000–2018 period," International Review of Finance, International Review of Finance Ltd., vol. 23(3), pages 553-583, September.
    3. Tyrväinen, Timo, 1991. "Unions, wages and employment: evidence from Finland," Bank of Finland Research Discussion Papers 16/1991, Bank of Finland.
    4. MacKinnon, J G, 1989. "Heteroskedasticity-Robust Tests for Structural Change," Empirical Economics, Springer, vol. 14(2), pages 77-92.
    5. Gu, Chen & Kurov, Alexander & Wolfe, Marketa Halova, 2018. "Relief Rallies after FOMC Announcements as a Resolution of Uncertainty," Journal of Empirical Finance, Elsevier, vol. 49(C), pages 1-18.
    6. Harvey, David I. & van Dijk, Dick, 2006. "Sample size, lag order and critical values of seasonal unit root tests," Computational Statistics & Data Analysis, Elsevier, vol. 50(10), pages 2734-2751, June.
    7. Hayk Khachatryan & Alicia Rihn & Ben Campbell & Bridget Behe & Charles Hall, 2018. "How do consumer perceptions of “local†production benefits influence their visual attention to state marketing programs?," Agribusiness, John Wiley & Sons, Ltd., vol. 34(2), pages 390-406, March.
    8. Marc Poitras, 2004. "The Impact of Macroeconomic Announcements on Stock Prices: In Search of State Dependence," Southern Economic Journal, John Wiley & Sons, vol. 70(3), pages 549-565, January.
    9. Gregory Thompson & Jeffrey Brown & Torsha Bhattacharya, 2012. "What Really Matters for Increasing Transit Ridership: Understanding the Determinants of Transit Ridership Demand in Broward County, Florida," Urban Studies, Urban Studies Journal Limited, vol. 49(15), pages 3327-3345, November.
    10. Goncalves, Silvia & Kilian, Lutz, 2004. "Bootstrapping autoregressions with conditional heteroskedasticity of unknown form," Journal of Econometrics, Elsevier, vol. 123(1), pages 89-120, November.
    11. Rafael González-Val, 2011. "Deviations from Zipf’s Law for American Cities," Urban Studies, Urban Studies Journal Limited, vol. 48(5), pages 1017-1035, April.
    12. Kerri Brick & Martine Visser & Justine Burns, 2012. "Risk Aversion: Experimental Evidence from South African Fishing Communities," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 94(1), pages 133-152.
    13. Kazunobu Hayakawa & Kiyoyasu Tanaka & Yasushi Ueki, 2013. "Transport Modal Choice by Multinational Firms: Firm-level Evidence from Southeast Asia," International Economic Journal, Taylor & Francis Journals, vol. 27(4), pages 609-623, December.
    14. Seung-Whan Choi & James A. Piazza, 2017. "Foreign Military Interventions and Suicide Attacks," Journal of Conflict Resolution, Peace Science Society (International), vol. 61(2), pages 271-297, February.
    15. Marius BRÜLHART & Federico TRIONFETTI, 1999. "Home-Biased Demand and International Specialisation : A Test of Trade Theories," Cahiers de Recherches Economiques du Département d'économie 9918, Université de Lausanne, Faculté des HEC, Département d’économie.
    16. Barbara Wolfe & Robert Haveman & Karen Pence & Jonathan Schwabish, 2007. "Do youth nonmarital childbearing choices reflect income and relationship expectations?," Journal of Population Economics, Springer;European Society for Population Economics, vol. 20(1), pages 73-100, February.
    17. Clara Berridge & Yuanjin Zhou & Julie M. Robillard & Jeffrey Kaye, 2023. "AI Companion Robot Data Sharing: Preferences of an Online Cohort and Policy Implications," Journal of Elder Policy, John Wiley & Sons, vol. 2(3), pages 19-54, June.
    18. Mikael Svensson & Fredrik Nilsson & Karl Arnberg, 2015. "Reimbursement Decisions for Pharmaceuticals in Sweden: The Impact of Disease Severity and Cost Effectiveness," PharmacoEconomics, Springer, vol. 33(11), pages 1229-1236, November.
    19. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    20. Dixon, Huw D. & Grimme, Christian, 2022. "State-dependent or time-dependent pricing? New evidence from a monthly firm-level survey: 1980–2017," European Economic Review, Elsevier, vol. 150(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09899-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.