IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i9p1547-d808559.html
   My bibliography  Save this article

Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model

Author

Listed:
  • Daniel Garcia Iglesias

    (Arrhythmia Unit, Cardiology Department, Hospital Universitario Central de Asturias, 33011 Oviedo, Spain
    Instituto de Investigación Sanitaria del Principado de Asturias, 33011 Oviedo, Spain)

Abstract

Propensity Score Matching (PSM) is a useful method to reduce the impact of Treatment-Selection Bias in the estimation of causal effects in observational studies. After matching, the PSM significantly reduces the sample under investigation, which may lead to other possible biases (due to overfitting, excess of covariation or a reduced number of observations). In this sense, we wanted to analyze the behavior of this PSM compared with other widely used methods to deal with non-comparable groups, such as the Multivariate Regression Model (MRM). Monte Carlo Simulations are made to construct groups with different effects in order to compare the behavior of PSM and MRM estimating these effects. In addition, the Treatment Selection Bias reduction for the PSM is calculated. With the PSM a reduction in the Treatment Selection Bias is achieved (0.983 [0.982, 0.984]), with a reduction in the Relative Real Treatment Effect Estimation Error (0.216 [0.2, 0.232]), but despite this bias reduction and estimation error reduction, the MRM reduces this estimation error significantly more than the PSM (0.539 [0.522, 0.556], p < 0.001). In addition, the PSM leads to a 30% reduction in the sample. This loss of information derived from the matching process may lead to another not known bias and thus to the inaccuracy of the effect estimation compared with the MRM.

Suggested Citation

  • Daniel Garcia Iglesias, 2022. "Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model," Mathematics, MDPI, vol. 10(9), pages 1-8, May.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/9/1547/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/9/1547/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ho, Daniel & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2011. "MatchIt: Nonparametric Preprocessing for Parametric Causal Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i08).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Emmanuel Owusu Oppong & Baorong Yu & Bruvine Orchidée Mazonga Mfoutou, 2024. "The effect of microinsurance on the financial resilience of low-income households in Ghana: evidence from a propensity score matching analysis," The Geneva Papers on Risk and Insurance - Issues and Practice, Palgrave Macmillan;The Geneva Association, vol. 49(3), pages 474-500, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chervier, Colas & Le Velly, Gwenolé & Ezzine-de-Blas, Driss, 2019. "When the Implementation of Payments for Biodiversity Conservation Leads to Motivation Crowding-out: A Case Study From the Cardamoms Forests, Cambodia," Ecological Economics, Elsevier, vol. 156(C), pages 499-510.
    2. Moritz Flubacher & George Sheldon & Adrian Müller, 2015. "Comparison of the Economic Performance between Organic and Conventional Dairy Farms in the Swiss Mountain Region Using Matching and Stochastic Frontier Analysis," Journal of Socio-Economics in Agriculture (Until 2015: Yearbook of Socioeconomics in Agriculture), Swiss Society for Agricultural Economics and Rural Sociology, vol. 7(1), pages 76-84.
    3. Akoh Fabien Yao & Maxime Sèbe & Laura Recuero Virto & Abdelhak Nassiri & Hervé Dumez, 2024. "The effect of LNG bunkering on port competitiveness using multilevel data analysis [L'effet du soutage par GNL sur la compétitivité des ports à l'aide de l'analyse de données à plusieurs niveaux]," Post-Print hal-04611804, HAL.
    4. Rigdon, Joseph & Berkowitz, Seth A. & Seligman, Hilary K. & Basu, Sanjay, 2017. "Re-evaluating associations between the Supplemental Nutrition Assistance Program participation and body mass index in the context of unmeasured confounders," Social Science & Medicine, Elsevier, vol. 192(C), pages 112-124.
    5. Rongtao Jiang & Stephanie Noble & Matthew Rosenblatt & Wei Dai & Jean Ye & Shu Liu & Shile Qi & Vince D. Calhoun & Jing Sui & Dustin Scheinost, 2024. "The brain structure, inflammatory, and genetic mechanisms mediate the association between physical frailty and depression," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    6. Finocchiaro Castro, Massimo & Guccio, Calogero & Rizzo, Ilde, 2023. "How "one-size-fits-all" public works contract does it better? An assessment of infrastructure provision in Italy," EconStor Preprints 270729, ZBW - Leibniz Information Centre for Economics.
    7. Lo, A. W.-T. & Houston, D., 2018. "How do compact, accessible, and walkable communities promote gender equality in spatial behavior?," Journal of Transport Geography, Elsevier, vol. 68(C), pages 42-54.
    8. Stjepan Srhoj & Michael Lapinski & Janette Walde, 2019. "Size matters? Impact evaluation of business development grants on SME performance," Working Papers 2019-14, Faculty of Economics and Statistics, Universität Innsbruck.
    9. Jiaming Zeng & Michael F. Gensheimer & Daniel L. Rubin & Susan Athey & Ross D. Shachter, 2022. "Uncovering interpretable potential confounders in electronic medical records," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    10. Dong, Hongwei, 2017. "Rail-transit-induced gentrification and the affordability paradox of TOD," Journal of Transport Geography, Elsevier, vol. 63(C), pages 1-10.
    11. Peter Psathas, 2024. "Top of the Class Assessing the Credit Performance of Graduates from Secured Credit Card Programs," Consumer Finance Institute discussion papers DP24-02, Federal Reserve Bank of Philadelphia.
    12. Chen, Shanting & Mallory, Allen B., 2021. "The effect of racial discrimination on mental and physical health: A propensity score weighting approach," Social Science & Medicine, Elsevier, vol. 285(C).
    13. Katie Devenish & Sébastien Desbureaux & Simon Willcock & Julia P. G. Jones, 2022. "On track to achieve no net loss of forest at Madagascar’s biggest mine," Nature Sustainability, Nature, vol. 5(6), pages 498-508, June.
    14. Changjun Gu & Pei Zhao & Qiong Chen & Shicheng Li & Lanhui Li & Linshan Liu & Yili Zhang, 2020. "Forest Cover Change and the Effectiveness of Protected Areas in the Himalaya since 1998," Sustainability, MDPI, vol. 12(15), pages 1-24, July.
    15. Häusler, Mara-Magdalena & Zabel, Astrid, 2024. "Sites side by side: Can an agglomeration bonus with an adjacency rule connect agri-environmental sites?," Ecological Economics, Elsevier, vol. 224(C).
    16. Leite, Walter & Zhang, Huibin & collier, zachary & Chawla, Kamal & , l.kong@ufl.edu & Lee, Yongseok & Quan, Jia & Soyoye, Olushola, 2024. "Machine Learning for Propensity Score Estimation: A Systematic Review and Reporting Guidelines," OSF Preprints gmrk7, Center for Open Science.
    17. Arne Lauer & Samantha L. Speroni & Myoung Choi & Xiao Da & Christine Duncan & Siobhan McCarthy & Vijai Krishnan & Cole A. Lusk & David Rohde & Mikkel Bo Hansen & Jayashree Kalpathy-Cramer & Daniel J. , 2023. "Hematopoietic stem-cell gene therapy is associated with restored white matter microvascular function in cerebral adrenoleukodystrophy," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    18. Maekawa, Wakako, 2024. "United Nations peacekeeping operations and multilateral foreign aid: Credibility of good governance," World Development, Elsevier, vol. 176(C).
    19. Mei-Cheng Wang & Yuxin Zhu, 2022. "Bias correction via outcome reassignment for cross-sectional data with binary disease outcome," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 659-674, October.
    20. Schneeberger, Andres R. & Huber, Christian G. & Lang, Undine E. & Muenzenmaier, Kristina H. & Castille, Dorothy & Jaeger, Matthias & Seixas, Azizi & Sowislo, Julia & Link, Bruce G., 2017. "Effects of assisted outpatient treatment and health care services on psychotic symptoms," Social Science & Medicine, Elsevier, vol. 175(C), pages 152-160.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.