IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v78y2022i1p202-213.html
   My bibliography  Save this article

New multivariate tests for assessing covariate balance in matched observational studies

Author

Listed:
  • Hao Chen
  • Dylan S. Small

Abstract

We propose new tests for assessing whether covariates in a treatment group and matched control group are balanced in observational studies. The tests exhibit high power under a wide range of multivariate alternatives, some of which existing tests have little power for. The asymptotic permutation null distributions of the proposed tests are studied and the P‐values calculated through the asymptotic results work well in simulation studies, facilitating the application of the test to large data sets. The tests are illustrated in a study of the effect of smoking on blood lead levels. The proposed tests are implemented in an R package BalanceCheck.

Suggested Citation

  • Hao Chen & Dylan S. Small, 2022. "New multivariate tests for assessing covariate balance in matched observational studies," Biometrics, The International Biometric Society, vol. 78(1), pages 202-213, March.
    • Handle: RePEc:bla:biomet:v:78:y:2022:i:1:p:202-213
    DOI: 10.1111/biom.13395
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13395
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13395?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kosuke Imai & Marc Ratkovic, 2014. "Covariate balancing propensity score," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 243-263, January.
    2. Czajka, John L, et al, 1992. "Projecting from Advance Data Using Propensity Modeling: An Application to Income and Tax Statistics," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(2), pages 117-131, April.
    3. Jesse Y. Hsu & Dylan S. Small, 2013. "Calibrating Sensitivity Analyses to Observed Covariates in Observational Studies," Biometrics, The International Biometric Society, vol. 69(4), pages 803-811, December.
    4. Paul R. Rosenbaum, 2005. "An exact distribution‐free test comparing two multivariate distributions based on adjacency," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(4), pages 515-530, September.
    5. Heller, Ruth & Rosenbaum, Paul R. & Small, Dylan S., 2010. "Using the Cross-Match Test to Appraise Covariate Balance in Matched Pairs," The American Statistician, American Statistical Association, vol. 64(4), pages 299-309.
    6. Kosuke Imai & Gary King & Elizabeth A. Stuart, 2008. "Misunderstandings between experimentalists and observationalists about causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(2), pages 481-502, April.
    7. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    8. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    9. Mozer, Reagan & Miratrix, Luke & Kaufman, Aaron Russell & Jason Anastasopoulos, L., 2020. "Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality," Political Analysis, Cambridge University Press, vol. 28(4), pages 445-468, October.
    10. Sekhon, Jasjeet S., 2011. "Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i07).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bo Zhang, 2023. "Efficient algorithms for building representative matched pairs with enhanced generalizability," Biometrics, The International Biometric Society, vol. 79(4), pages 3981-3997, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adeola Oyenubi & Martin Wittenberg, 2021. "Does the choice of balance-measure matter under genetic matching?," Empirical Economics, Springer, vol. 61(1), pages 489-502, July.
    2. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    3. Lenz, Gabriel & Sahn, Alexander, 2017. "Achieving Statistical Significance with Covariates and without Transparency," MetaArXiv s42ba, Center for Open Science.
    4. Jasjeet Singh Sekhon & Richard D. Grieve, 2012. "A matching method for improving covariate balance in cost‐effectiveness analyses," Health Economics, John Wiley & Sons, Ltd., vol. 21(6), pages 695-714, June.
    5. Wang-Sheng Lee, 2013. "Propensity score matching and variations on the balancing test," Empirical Economics, Springer, vol. 44(1), pages 47-80, February.
    6. Alessi, Lucia & Battiston, Stefano & Kvedaras, Virmantas, 2024. "Over with carbon? Investors’ reaction to the Paris Agreement and the US withdrawal," Journal of Financial Stability, Elsevier, vol. 71(C).
    7. Ragni, Alessandra & Ippolito, Daniel & Masci, Chiara, 2024. "Assessing the impact of hybrid teaching on students’ academic performance via multilevel propensity score-based techniques," Socio-Economic Planning Sciences, Elsevier, vol. 92(C).
    8. Nicole E. Pashley & Luke W. Miratrix, 2021. "Insights on Variance Estimation for Blocked and Matched Pairs Designs," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 271-296, June.
    9. Alexander G. Nikolaev & Sheldon H. Jacobson & Wendy K. Tam Cho & Jason J. Sauppe & Edward C. Sewell, 2013. "Balance Optimization Subset Selection (BOSS): An Alternative Approach for Causal Inference with Observational Data," Operations Research, INFORMS, vol. 61(2), pages 398-412, April.
    10. Frölich, Markus & Huber, Martin & Wiesenfarth, Manuel, 2017. "The finite sample performance of semi- and non-parametric estimators for treatment effects and policy evaluation," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 91-102.
    11. Wichman, Casey J. & Ferraro, Paul J., 2017. "A cautionary tale on using panel data estimators to measure program impacts," Economics Letters, Elsevier, vol. 151(C), pages 82-90.
    12. Bob Blankenberger & Sophia Gehlhausen Anderson & Eric Lichtenberger, 2021. "Improving Institutional Evaluation Methods: Comparing Three Evaluations Using PSM, Exact and Coarsened Exact Matching," Research in Higher Education, Springer;Association for Institutional Research, vol. 62(8), pages 1248-1275, December.
    13. Berenger Djoumessi Tiague, 2023. "Floods, Agricultural Production, and Household Welfare: Evidence from Tanzania," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 85(2), pages 341-384, June.
    14. Alessi, Lucia & Battiston, Stefano & Kvedaras, Virmantas, 2021. "Over with carbon? Investors' reaction to the Paris Agreement and the US withdrawal," JRC Working Papers in Economics and Finance 2021-12, Joint Research Centre, European Commission.
    15. Lundberg, Ian & Brand, Jennie E. & Jeon, Nanum, 2022. "Researcher reasoning meets computational capacity: Machine learning for social science," SocArXiv s5zc8, Center for Open Science.
    16. Ben Weidmann & Luke Miratrix, 2021. "Lurking Inferential Monsters? Quantifying Selection Bias In Evaluations Of School Programs," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 964-986, June.
    17. Francesco Lamperti & Roberto Mavilia & Simona Castellini, 2017. "The role of Science Parks: a puzzle of growth, innovation and R&D investments," The Journal of Technology Transfer, Springer, vol. 42(1), pages 158-183, February.
    18. Marco Morucci & Md. Noor-E-Alam & Cynthia Rudin, 2022. "A Robust Approach to Quantifying Uncertainty in Matching Problems of Causal Inference," INFORMS Joural on Data Science, INFORMS, vol. 1(2), pages 156-171, October.
    19. Margaret E. Roberts & Brandon M. Stewart & Richard A. Nielsen, 2020. "Adjusting for Confounding with Text Matching," American Journal of Political Science, John Wiley & Sons, vol. 64(4), pages 887-903, October.
    20. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:78:y:2022:i:1:p:202-213. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.