IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v42y2017i5p499-530.html
   My bibliography  Save this article

Statistical Power for Causally Defined Indirect Effects in Group-Randomized Trials With Individual-Level Mediators

Author

Listed:
  • Benjamin Kelcey

    (University of Cincinnati)

  • Nianbo Dong

    (University of Missouri)

  • Jessaca Spybrook

    (Western Michigan University)

  • Kyle Cox

    (University of Cincinnati)

Abstract

Designs that facilitate inferences concerning both the total and indirect effects of a treatment potentially offer a more holistic description of interventions because they can complement “what works†questions with the comprehensive study of the causal connections implied by substantive theories. Mapping the sensitivity of designs to detect these effects is of critical importance because it directly governs the types of evidence researchers can bring to bear on theories of action under realistic sample sizes. In this study, we develop closed-form expressions to estimate the variance of and the power to detect causally defined indirect effects in two-level group-randomized studies examining individual-level mediators (i.e., 2-1-1 mediation). We formulate our approach within the purview of typical multilevel mediation models and anchor their interpretation in the potential outcomes framework. The results provide power analysis formulas that reduce calculations to simple functions of the primary path coefficients (e.g., treatment–mediator and mediator–outcome relationships) and common summary statistics (e.g., intraclass correlation coefficients). Probing these formulas suggests that group-randomized designs can be well powered to detect indirect effects when carefully planned. The power formulas are implemented in the PowerUp software ( causalevaluation.org ).

Suggested Citation

  • Benjamin Kelcey & Nianbo Dong & Jessaca Spybrook & Kyle Cox, 2017. "Statistical Power for Causally Defined Indirect Effects in Group-Randomized Trials With Individual-Level Mediators," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 499-530, October.
    • Handle: RePEc:sae:jedbes:v:42:y:2017:i:5:p:499-530
    DOI: 10.3102/1076998617695506
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998617695506
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998617695506?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hong, Guanglei & Raudenbush, Stephen W., 2006. "Evaluating Kindergarten Retention Policy: A Case Study of Causal Inference for Multilevel Observational Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 901-910, September.
    2. repec:mpr:mprres:7090 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xu Qin & Jonah Deutsch & Guanglei Hong, 2021. "Unpacking Complex Mediation Mechanisms And Their Heterogeneity Between Sites In A Job Corps Evaluation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(1), pages 158-190, January.
    2. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    3. Chiba, Yasutaka, 2012. "A note on bounds for the causal infectiousness effect in vaccine trials," Statistics & Probability Letters, Elsevier, vol. 82(7), pages 1422-1429.
    4. Youmi Suk & Kyung T. Han, 2024. "A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 151-172, April.
    5. Lindsay C. Page, 2012. "Understanding the Impact of Career Academy Attendance," Evaluation Review, , vol. 36(2), pages 99-132, April.
    6. Soojin Park & Peter M. Steiner & David Kaplan, 2018. "Identification and Sensitivity Analysis for Average Causal Mediation Effects with Time-Varying Treatments and Mediators: Investigating the Underlying Mechanisms of Kindergarten Retention Policy," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 298-320, June.
    7. Nianbo Dong & Mark W. Lipsey, 2018. "Can Propensity Score Analysis Approximate Randomized Experiments Using Pretest and Demographic Information in Pre-K Intervention Research?," Evaluation Review, , vol. 42(1), pages 34-70, February.
    8. Nianbo Dong & Elizabeth A. Stuart & David Lenis & Trang Quynh Nguyen, 2020. "Using Propensity Score Analysis of Survey Data to Estimate Population Average Treatment Effects: A Case Study Comparing Different Methods," Evaluation Review, , vol. 44(1), pages 84-108, February.
    9. Weicong Lyu & Jee-Seon Kim & Youmi Suk, 2023. "Estimating Heterogeneous Treatment Effects Within Latent Class Multilevel Models: A Bayesian Approach," Journal of Educational and Behavioral Statistics, , vol. 48(1), pages 3-36, February.
    10. Tadao Hoshino & Takahide Yanagi, 2023. "Randomization Test for the Specification of Interference Structure," Papers 2301.05580, arXiv.org, revised Dec 2023.
    11. Tyler J. VanderWeele, 2010. "Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data," Sociological Methods & Research, , vol. 38(4), pages 515-544, May.
    12. Peter Z. Schochet, 2013. "Student Mobility, Dosage, and Principal Stratification in School-Based RCTs," Journal of Educational and Behavioral Statistics, , vol. 38(4), pages 323-354, August.
    13. Silvia Noirjean & Marco Mariani & Alessandra Mattei & Fabrizia Mealli, 2020. "Exploiting network information to disentangle spillover effects in a field experiment on teens' museum attendance," Papers 2011.11023, arXiv.org, revised May 2022.
    14. Md Saiful Islam & Md Sarowar Morshed & Gary J Young & Md Noor-E-Alam, 2019. "Robust policy evaluation from large-scale observational studies," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-19, October.
    15. Arpino, Bruno & Mattei, Alessandra, 2013. "Assessing the Impact of Financial Aids to Firms: Causal Inference in the presence of Interference," MPRA Paper 51795, University Library of Munich, Germany.
    16. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    17. Luke Keele & Rocío Titiunik, 2018. "Geographic Natural Experiments with Interference: The Effect of All-Mail Voting on Turnout in Colorado," CESifo Economic Studies, CESifo Group, vol. 64(2), pages 127-149.
    18. Arpino, Bruno & Mealli, Fabrizia, 2011. "The specification of the propensity score in multilevel observational studies," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1770-1780, April.
    19. Yongyun Shin & Stephen W. Raudenbush, 2011. "The Causal Effect of Class Size on Academic Achievement," Journal of Educational and Behavioral Statistics, , vol. 36(2), pages 154-185, April.
    20. Rigdon, Joseph & Hudgens, Michael G., 2015. "Exact confidence intervals in the presence of interference," Statistics & Probability Letters, Elsevier, vol. 105(C), pages 130-135.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:42:y:2017:i:5:p:499-530. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.