IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v34y2019i2d10.1007_s00180-018-0852-5.html
   My bibliography  Save this article

Power comparison for propensity score methods

Author

Listed:
  • Byeong Yeob Choi

    (University of Texas Health Science Center)

  • Chen-Pin Wang

    (University of Texas Health Science Center)

  • Joel Michalek

    (University of Texas Health Science Center)

  • Jonathan Gelfond

    (University of Texas Health Science Center)

Abstract

We compared four propensity score (PS) methods using simulations: maximum likelihood (ML), generalized boosting models (GBM), covariate balancing propensity scores (CBPS), and generalized additive models (GAM). Although these methods have been shown to perform better than the ML in estimating causal treatment effects, no comparison has been conducted in terms of type I error and power, and the impact of treatment exposure prevalence on PS methods has not been studied. In order to fill these gaps, we considered four simulation scenarios differing by the complexity of a propensity score model and a range of exposure prevalence. Propensity score weights were estimated using the ML, CBPS and GAM of logistic regression and the GBM. We used these propensity weights to estimate the average treatment effect among treated on a binary outcome. Simulations showed that (1) the CBPS was generally superior across the four scenarios studied in terms of type I error, power and mean squared error; (2) the GBM and the GAM were less biased than the CBPS and the ML under complex models; (3) the ML performed well when treatment exposure is rare.

Suggested Citation

  • Byeong Yeob Choi & Chen-Pin Wang & Joel Michalek & Jonathan Gelfond, 2019. "Power comparison for propensity score methods," Computational Statistics, Springer, vol. 34(2), pages 743-761, June.
  • Handle: RePEc:spr:compst:v:34:y:2019:i:2:d:10.1007_s00180-018-0852-5
    DOI: 10.1007/s00180-018-0852-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-018-0852-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00180-018-0852-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kosuke Imai & Marc Ratkovic, 2014. "Covariate balancing propensity score," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 243-263, January.
    2. Brian K Lee & Justin Lessler & Elizabeth A Stuart, 2011. "Weight Trimming and Propensity Score Weighting," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-6, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Byeong Yeob Choi, 2021. "Instrumental variable estimation of truncated local average treatment effects," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-12, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.
    2. Tenglong Li & Jordan Lawson, 2021. "A generalized bootstrap procedure of the standard error and confidence interval estimation for inverse probability of treatment weighting," Papers 2109.00171, arXiv.org.
    3. Byeong Yeob Choi, 2021. "Instrumental variable estimation of truncated local average treatment effects," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-12, April.
    4. Martin Cousineau & Vedat Verter & Susan A. Murphy & Joelle Pineau, 2022. "Estimating causal effects with optimization-based methods: A review and empirical comparison," Papers 2203.00097, arXiv.org.
    5. Noémi Kreif & Richard Grieve & Iván Díaz & David Harrison, 2015. "Evaluation of the Effect of a Continuous Treatment: A Machine Learning Approach with an Application to Treatment for Traumatic Brain Injury," Health Economics, John Wiley & Sons, Ltd., vol. 24(9), pages 1213-1228, September.
    6. Turner, Alex J. & Fichera, Eleonora & Sutton, Matt, 2021. "The effects of in-utero exposure to influenza on mental health and mortality risk throughout the life-course," Economics & Human Biology, Elsevier, vol. 43(C).
    7. Zichen Deng & Maarten Lindeboom, 2021. "Early-life Famine Exposure, Hunger Recall and Later-life Health," Tinbergen Institute Discussion Papers 21-054/V, Tinbergen Institute.
    8. Baron, Opher & Callen, Jeffrey L. & Segal, Dan, 2023. "Does the bullwhip matter economically? A cross-sectional firm-level analysis," International Journal of Production Economics, Elsevier, vol. 259(C).
    9. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.
    10. Mortimer, Duncan & Harris, Anthony & Wijnands, Jasper S. & Stevenson, Mark, 2021. "Persistence or reversal? The micro-effects of time-varying financial penalties," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 72-86.
    11. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    12. Glover, Steven & Jones, Sam, 2019. "Can commercial farming promote rural dynamism in sub-Saharan Africa? Evidence from Mozambique," World Development, Elsevier, vol. 114(C), pages 110-121.
    13. Jaiswal, Sreeja & Balietti, Anca & Schäffer, Daniel, 2023. "Environmental Protection and Labor Market Composition," Working Papers 0736, University of Heidelberg, Department of Economics.
    14. Caloffi, Annalisa & Freo, Marzia & Ghinoi, Stefano & Mariani, Marco & Rossi, Federica, 2022. "Assessing the effects of a deliberate policy mix: The case of technology and innovation advisory services and innovation vouchers," Research Policy, Elsevier, vol. 51(6).
    15. Nazmul Islam & Natalie E. Sheils & Megan S. Jarvis & Kenneth Cohen, 2022. "Comparative effectiveness over time of the mRNA-1273 (Moderna) vaccine and the BNT162b2 (Pfizer-BioNTech) vaccine," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    16. Sourafel Girma & Holger Görg, 2022. "Productivity effects of processing and ordinary export market entry: A time‐varying treatments approach," Review of International Economics, Wiley Blackwell, vol. 30(3), pages 836-853, August.
    17. Wendy Chan, 2018. "Applications of Small Area Estimation to Generalization With Subclassification by Propensity Scores," Journal of Educational and Behavioral Statistics, , vol. 43(2), pages 182-224, April.
    18. Meyer, Maximilian & Hulke, Carolin & Kamwi, Jonathan & Kolem, Hannah & Börner, Jan, 2022. "Spatially heterogeneous effects of collective action on environmental dependence in Namibia’s Zambezi region," World Development, Elsevier, vol. 159(C).
    19. Oyenubi, Adeola & Kollamparambil, Umakrishnan, 2023. "Does noncompliance with COVID-19 regulations impact the depressive symptoms of others?," Economic Modelling, Elsevier, vol. 120(C).
    20. Bonnie E. Shook‐Sa & Michael G. Hudgens, 2022. "Power and sample size for observational studies of point exposure effects," Biometrics, The International Biometric Society, vol. 78(1), pages 388-398, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:34:y:2019:i:2:d:10.1007_s00180-018-0852-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.