IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i1d10.1007_s11336-022-09899-x.html
   My bibliography  Save this article

Accurate Confidence and Bayesian Interval Estimation for Non-centrality Parameters and Effect Size Indices

Author

Listed:
  • Kaidi Kang

    (Vanderbilt University)

  • Megan T. Jones

    (Vanderbilt University)

  • Kristan Armstrong

    (Vanderbilt University Medical Center)

  • Suzanne Avery

    (Vanderbilt University Medical Center)

  • Maureen McHugo

    (Vanderbilt University Medical Center)

  • Stephan Heckers

    (Vanderbilt University Medical Center)

  • Simon Vandekar

    (Vanderbilt University)

Abstract

Reporting effect size index estimates with their confidence intervals (CIs) can be an excellent way to simultaneously communicate the strength and precision of the observed evidence. We recently proposed a robust effect size index (RESI) that is advantageous over common indices because it’s widely applicable to different types of data. Here, we use statistical theory and simulations to develop and evaluate RESI estimators and confidence/credible intervals that rely on different covariance estimators. Our results show (1) counter to intuition, the randomness of covariates reduces coverage for Chi-squared and F CIs; (2) when the variance of the estimators is estimated, the non-central Chi-squared and F CIs using the parametric and robust RESI estimators fail to cover the true effect size at the nominal level. Using the robust estimator along with the proposed nonparametric bootstrap or Bayesian (credible) intervals provides valid inference for the RESI, even when model assumptions may be violated. This work forms a unified effect size reporting procedure, such that effect sizes with confidence/credible intervals can be easily reported in an analysis of variance (ANOVA) table format.

Suggested Citation

  • Kaidi Kang & Megan T. Jones & Kristan Armstrong & Suzanne Avery & Maureen McHugo & Stephan Heckers & Simon Vandekar, 2023. "Accurate Confidence and Bayesian Interval Estimation for Non-centrality Parameters and Effect Size Indices," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 253-273, March.
    • Handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09899-x
    DOI: 10.1007/s11336-022-09899-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09899-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-022-09899-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ronald L. Wasserstein & Allen L. Schirm & Nicole A. Lazar, 2019. "Moving to a World Beyond “p," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 1-19, March.
    2. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "Erratum to: A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 946-946, December.
    3. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    4. Zhiyi Zhang & Nancy Schoeps, 1997. "On robust estimation of effect size under semiparametric models," Psychometrika, Springer;The Psychometric Society, vol. 62(2), pages 201-214, June.
    5. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 232-246, March.
    6. J. Scott Long & Jeremy Freese, 2006. "Regression Models for Categorical Dependent Variables using Stata, 2nd Edition," Stata Press books, StataCorp LP, edition 2, number long2, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simon Vandekar & Ran Tao & Jeffrey Blume, 2020. "A Robust Effect Size Index," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 232-246, March.
    2. Jinsuk Yang & Qing Hao & Mahmut Yaşar, 2023. "Institutional investors and cross‐border mergers and acquisitions: The 2000–2018 period," International Review of Finance, International Review of Finance Ltd., vol. 23(3), pages 553-583, September.
    3. Tyrväinen, Timo, 1991. "Unions, wages and employment: evidence from Finland," Bank of Finland Research Discussion Papers 16/1991, Bank of Finland.
    4. MacKinnon, J G, 1989. "Heteroskedasticity-Robust Tests for Structural Change," Empirical Economics, Springer, vol. 14(2), pages 77-92.
    5. Gu, Chen & Kurov, Alexander & Wolfe, Marketa Halova, 2018. "Relief Rallies after FOMC Announcements as a Resolution of Uncertainty," Journal of Empirical Finance, Elsevier, vol. 49(C), pages 1-18.
    6. Marc Poitras, 2004. "The Impact of Macroeconomic Announcements on Stock Prices: In Search of State Dependence," Southern Economic Journal, John Wiley & Sons, vol. 70(3), pages 549-565, January.
    7. Gregory Thompson & Jeffrey Brown & Torsha Bhattacharya, 2012. "What Really Matters for Increasing Transit Ridership: Understanding the Determinants of Transit Ridership Demand in Broward County, Florida," Urban Studies, Urban Studies Journal Limited, vol. 49(15), pages 3327-3345, November.
    8. Goncalves, Silvia & Kilian, Lutz, 2004. "Bootstrapping autoregressions with conditional heteroskedasticity of unknown form," Journal of Econometrics, Elsevier, vol. 123(1), pages 89-120, November.
    9. Kerri Brick & Martine Visser & Justine Burns, 2012. "Risk Aversion: Experimental Evidence from South African Fishing Communities," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 94(1), pages 133-152.
    10. Seung-Whan Choi & James A. Piazza, 2017. "Foreign Military Interventions and Suicide Attacks," Journal of Conflict Resolution, Peace Science Society (International), vol. 61(2), pages 271-297, February.
    11. Clara Berridge & Yuanjin Zhou & Julie M. Robillard & Jeffrey Kaye, 2023. "AI Companion Robot Data Sharing: Preferences of an Online Cohort and Policy Implications," Journal of Elder Policy, John Wiley & Sons, vol. 2(3), pages 19-54, June.
    12. Mikael Svensson & Fredrik Nilsson & Karl Arnberg, 2015. "Reimbursement Decisions for Pharmaceuticals in Sweden: The Impact of Disease Severity and Cost Effectiveness," PharmacoEconomics, Springer, vol. 33(11), pages 1229-1236, November.
    13. Dixon, Huw D. & Grimme, Christian, 2022. "State-dependent or time-dependent pricing? New evidence from a monthly firm-level survey: 1980–2017," European Economic Review, Elsevier, vol. 150(C).
    14. Melanie Lefevre, 2011. "Willingness-to-pay for Local Milk-based Dairy Product in Senegal," CREPP Working Papers 1108, Centre de Recherche en Economie Publique et de la Population (CREPP) (Research Center on Public and Population Economics) HEC-Management School, University of Liège.
    15. Erik Stam & Roy Thurik & Peter van der Zwan, 2010. "Entrepreneurial exit in real and imagined markets," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 19(4), pages 1109-1139, August.
    16. Angrist, Josh & Lavy, Victor, 2002. "The Effect of High School Matriculation Awards: Evidence from Randomized Trials," CEPR Discussion Papers 3827, C.E.P.R. Discussion Papers.
    17. Miriam Marcén & Marina Morales, 2019. "Live together: does culture matter?," Review of Economics of the Household, Springer, vol. 17(2), pages 671-713, June.
    18. Koetter, Michael & Krause, Thomas & Tonzer, Lena, 2019. "Delay determinants of European Banking Union implementation," European Journal of Political Economy, Elsevier, vol. 58(C), pages 1-20.
    19. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    20. Bruno Amable, 2009. "The Differentiation of Social Demands in Europe. The Social Basis of the European Models of Capitalism," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 91(3), pages 391-426, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09899-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.