IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i9p1547-d808559.html
   My bibliography  Save this article

Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model

Author

Listed:
  • Daniel Garcia Iglesias

    (Arrhythmia Unit, Cardiology Department, Hospital Universitario Central de Asturias, 33011 Oviedo, Spain
    Instituto de Investigación Sanitaria del Principado de Asturias, 33011 Oviedo, Spain)

Abstract

Propensity Score Matching (PSM) is a useful method to reduce the impact of Treatment-Selection Bias in the estimation of causal effects in observational studies. After matching, the PSM significantly reduces the sample under investigation, which may lead to other possible biases (due to overfitting, excess of covariation or a reduced number of observations). In this sense, we wanted to analyze the behavior of this PSM compared with other widely used methods to deal with non-comparable groups, such as the Multivariate Regression Model (MRM). Monte Carlo Simulations are made to construct groups with different effects in order to compare the behavior of PSM and MRM estimating these effects. In addition, the Treatment Selection Bias reduction for the PSM is calculated. With the PSM a reduction in the Treatment Selection Bias is achieved (0.983 [0.982, 0.984]), with a reduction in the Relative Real Treatment Effect Estimation Error (0.216 [0.2, 0.232]), but despite this bias reduction and estimation error reduction, the MRM reduces this estimation error significantly more than the PSM (0.539 [0.522, 0.556], p < 0.001). In addition, the PSM leads to a 30% reduction in the sample. This loss of information derived from the matching process may lead to another not known bias and thus to the inaccuracy of the effect estimation compared with the MRM.

Suggested Citation

  • Daniel Garcia Iglesias, 2022. "Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model," Mathematics, MDPI, vol. 10(9), pages 1-8, May.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/9/1547/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/9/1547/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ho, Daniel & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2011. "MatchIt: Nonparametric Preprocessing for Parametric Causal Inference," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i08).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liao, Chuan & Jung, Suhyun & Brown, Daniel G. & Agrawal, Arun, 2024. "Does land tenure change accelerate deforestation? A matching-based four-country comparison," Ecological Economics, Elsevier, vol. 215(C).
    2. Quynh-Lam Tran & Gregorio Benitez & Fadi Shehadeh & Matthew Kaczynski & Eleftherios Mylonakis, 2022. "Clinical Outcomes Associated with SARS-CoV-2 Co-Infection with Rhinovirus and Adenovirus in Adults—A Retrospective Matched Cohort Study," IJERPH, MDPI, vol. 20(1), pages 1-13, December.
    3. Meyer, Maximilian & Hulke, Carolin & Kamwi, Jonathan & Kolem, Hannah & Börner, Jan, 2022. "Spatially heterogeneous effects of collective action on environmental dependence in Namibia’s Zambezi region," World Development, Elsevier, vol. 159(C).
    4. Chervier, Colas & Le Velly, Gwenolé & Ezzine-de-Blas, Driss, 2019. "When the Implementation of Payments for Biodiversity Conservation Leads to Motivation Crowding-out: A Case Study From the Cardamoms Forests, Cambodia," Ecological Economics, Elsevier, vol. 156(C), pages 499-510.
    5. Mansaray, Alhassan & Coleman, Simeon & Ataullah, Ali & Sirichand, Kavita, 2021. "Residual government ownership in public-private partnership projects," Journal of Government and Economics, Elsevier, vol. 4(C).
    6. Moritz Flubacher & George Sheldon & Adrian Müller, 2015. "Comparison of the Economic Performance between Organic and Conventional Dairy Farms in the Swiss Mountain Region Using Matching and Stochastic Frontier Analysis," Journal of Socio-Economics in Agriculture (Until 2015: Yearbook of Socioeconomics in Agriculture), Swiss Society for Agricultural Economics and Rural Sociology, vol. 7(1), pages 76-84.
    7. Finocchiaro Castro, Massimo & Guccio, Calogero & Rizzo, Ilde, 2023. "“One-size-fits-all” public works contract does it better? An assessment of infrastructure provision in Italy," Journal of Policy Modeling, Elsevier, vol. 45(5), pages 994-1014.
    8. Ferentinos, Konstantinos & Gibberd, Alex & Guin, Benjamin, 2021. "Climate policy and transition risk in the housing market," Bank of England working papers 918, Bank of England.
    9. Altman, Micah & Fox, John & Jackman, Simon & Zeileis, Achim, 2011. "An Introduction to the Special Volume on "Political Methodology"," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i01).
    10. Marcos-Martinez, Raymundo & Measham, Thomas G. & Fleming-Muñoz, David A., 2019. "Economic impacts of early unconventional gas mining: Lessons from the coal seam gas industry in New South Wales, Australia," Energy Policy, Elsevier, vol. 125(C), pages 338-346.
    11. Akoh Fabien Yao & Maxime Sèbe & Laura Recuero Virto & Abdelhak Nassiri & Hervé Dumez, 2024. "The effect of LNG bunkering on port competitiveness using multilevel data analysis [L'effet du soutage par GNL sur la compétitivité des ports à l'aide de l'analyse de données à plusieurs niveaux]," Post-Print hal-04611804, HAL.
    12. Carazza, Luís & Silveira Neto, Raul da Mota, 2021. "Evaluating the Regional Expansion of Brazil’s Federal System of Vocational and Technological Education," Revista Brasileira de Estudos Regionais e Urbanos, Associação Brasileira de Estudos Regionais e Urbanos (ABER), vol. 15(2), pages 212-246.
    13. Rigdon, Joseph & Berkowitz, Seth A. & Seligman, Hilary K. & Basu, Sanjay, 2017. "Re-evaluating associations between the Supplemental Nutrition Assistance Program participation and body mass index in the context of unmeasured confounders," Social Science & Medicine, Elsevier, vol. 192(C), pages 112-124.
    14. Wang-Ly, Nathan & Newell, Ben R., 2022. "Allowing early access to retirement savings: Lessons from Australia," Economic Analysis and Policy, Elsevier, vol. 75(C), pages 716-733.
    15. Rongtao Jiang & Stephanie Noble & Matthew Rosenblatt & Wei Dai & Jean Ye & Shu Liu & Shile Qi & Vince D. Calhoun & Jing Sui & Dustin Scheinost, 2024. "The brain structure, inflammatory, and genetic mechanisms mediate the association between physical frailty and depression," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    16. Finocchiaro Castro, Massimo & Guccio, Calogero & Rizzo, Ilde, 2023. "How "one-size-fits-all" public works contract does it better? An assessment of infrastructure provision in Italy," EconStor Preprints 270729, ZBW - Leibniz Information Centre for Economics.
    17. Lo, A. W.-T. & Houston, D., 2018. "How do compact, accessible, and walkable communities promote gender equality in spatial behavior?," Journal of Transport Geography, Elsevier, vol. 68(C), pages 42-54.
    18. Dai, Ziyi & Liu, Haobing & Rodgers, Michael O. & Guensler, Randall, 2022. "Electric vehicle market potential and associated energy and emissions reduction benefits," Applied Energy, Elsevier, vol. 322(C).
    19. Petros Andrikopoulos & Judith Aron-Wisnewsky & Rima Chakaroun & Antonis Myridakis & Sofia K. Forslund & Trine Nielsen & Solia Adriouch & Bridget Holmes & Julien Chilloux & Sara Vieira-Silva & Gwen Fal, 2023. "Evidence of a causal and modifiable relationship between kidney function and circulating trimethylamine N-oxide," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    20. Stjepan Srhoj & Michael Lapinski & Janette Walde, 2019. "Size matters? Impact evaluation of business development grants on SME performance," Working Papers 2019-14, Faculty of Economics and Statistics, Universität Innsbruck.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:9:p:1547-:d:808559. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.