IDEAS home Printed from https://ideas.repec.org/a/taf/jnlbes/v34y2016i1p139-160.html
   My bibliography  Save this article

The Finite Sample Performance of Estimators for Mediation Analysis Under Sequential Conditional Independence

Author

Listed:
  • Martin Huber
  • Michael Lechner
  • Giovanni Mellace

Abstract

Using a comprehensive simulation study based on empirical data, this article investigates the finite sample properties of different classes of parametric and semiparametric estimators of (natural) direct and indirect causal effects used in mediation analysis under sequential conditional independence assumptions. The estimators are based on regression, inverse probability weighting, and combinations thereof. Our simulation design uses a large population of Swiss jobseekers and considers variations of several features of the data-generating process (DGP) and the implementation of the estimators that are of practical relevance. We find that no estimator performs uniformly best (in terms of root mean squared error) in all simulations. Overall, so-called “g-computation” dominates. However, differences between estimators are often (but not always) minor in the various setups and the relative performance of the methods often (but not always) varies with the features of the DGP.

Suggested Citation

  • Martin Huber & Michael Lechner & Giovanni Mellace, 2016. "The Finite Sample Performance of Estimators for Mediation Analysis Under Sequential Conditional Independence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(1), pages 139-160, January.
    • Handle: RePEc:taf:jnlbes:v:34:y:2016:i:1:p:139-160
    DOI: 10.1080/07350015.2015.1017644
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/07350015.2015.1017644
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/07350015.2015.1017644?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Stefanie Behncke & Markus Frölich & Michael Lechner, 2010. "A Caseworker Like Me - Does The Similarity Between The Unemployed and Their Caseworkers Increase Job Placements?," Economic Journal, Royal Economic Society, vol. 120(549), pages 1430-1459, December.
    2. Arun Advani & Tymon Słoczyński, 2013. "Mostly harmless simulations? On the internal validity of empirical Monte Carlo studies," CeMMAP working papers 64/13, Institute for Fiscal Studies.
    3. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    4. Zheng Wenjing & van der Laan Mark J., 2012. "Targeted Maximum Likelihood Estimation of Natural Direct Effects," The International Journal of Biostatistics, De Gruyter, vol. 8(1), pages 1-40, January.
    5. Thomas R. Ten Have & Marshall M. Joffe & Kevin G. Lynch & Gregory K. Brown & Stephen A. Maisto & Aaron T. Beck, 2007. "Causal Mediation Analyses with Rank Preserving Models," Biometrics, The International Biometric Society, vol. 63(3), pages 926-934, September.
    6. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    7. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    8. Andrew Gelman & Guido Imbens, 2013. "Why ask Why? Forward Causal Inference and Reverse Causal Questions," NBER Working Papers 19614, National Bureau of Economic Research, Inc.
    9. Flores, Carlos A. & Flores-Lagunes, Alfonso, 2009. "Identification and Estimation of Causal Mechanisms and Net Effects of a Treatment under Unconfoundedness," IZA Discussion Papers 4237, Institute of Labor Economics (IZA).
    10. Newey, Whitney K., 1984. "A method of moments interpretation of sequential estimators," Economics Letters, Elsevier, vol. 14(2-3), pages 201-206.
    11. van der Laan Mark J. & Rubin Daniel, 2006. "Targeted Maximum Likelihood Learning," The International Journal of Biostatistics, De Gruyter, vol. 2(1), pages 1-40, December.
    12. Shakeeb Khan & Elie Tamer, 2010. "Irregular Identification, Support Conditions, and Inverse Weight Estimation," Econometrica, Econometric Society, vol. 78(6), pages 2021-2042, November.
    13. Raymond Hicks & Dustin Tingley, 2011. "Causal mediation analysis," Stata Journal, StataCorp LP, vol. 11(4), pages 605-619, December.
    14. Martin Huber & Michael Lechner & Giovanni Mellace, 2017. "Why Do Tougher Caseworkers Increase Employment? The Role of Program Assignment as a Causal Mechanism," The Review of Economics and Statistics, MIT Press, vol. 99(1), pages 180-183, March.
    15. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    16. Jeffrey M. Albert & Suchitra Nelson, 2011. "Generalized Causal Mediation Analysis," Biometrics, The International Biometric Society, vol. 67(3), pages 1028-1038, September.
    17. Imai, Kosuke & Yamamoto, Teppei, 2013. "Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments," Political Analysis, Cambridge University Press, vol. 21(2), pages 141-171, April.
    18. Huber, Martin, 2012. "Identifying causal mechanisms in experiments (primarily) based on inverse probability weighting," Economics Working Paper Series 1213, University of St. Gallen, School of Economics and Political Science, revised May 2013.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Arun Advani & Toru Kitagawa & Tymon Słoczyński, 2019. "Mostly harmless simulations? Using Monte Carlo studies for estimator selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(6), pages 893-910, September.
    2. Lombardi, Stefano & van den Berg, Gerard J. & Vikström, Johan, 2020. "Empirical Monte Carlo evidence on estimation of Timing-of-Events models," Working Paper Series 2020:26, IFAU - Institute for Evaluation of Labour Market and Education Policy, revised 05 Jan 2021.
    3. Frölich, Markus & Huber, Martin & Wiesenfarth, Manuel, 2017. "The finite sample performance of semi- and non-parametric estimators for treatment effects and policy evaluation," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 91-102.
    4. Mellace, Giovanni & Pasquini, Alessandra, 2019. "Identify More, Observe Less: Mediation Analysis: Mediation Analysis Synthetic Control," Discussion Papers on Economics 12/2019, University of Southern Denmark, Department of Economics.
    5. Dr. Daniel Kasser Tee, 2022. "Mediating Effect of Customer Satisfaction on the Relationship between Service Quality and Customer Loyalty in the Ghana Banking Industry," International Journal of Research and Scientific Innovation, International Journal of Research and Scientific Innovation (IJRSI), vol. 9(4), pages 17-26, April.
    6. Giovanni Mellace & Alessandra Pasquini, 2022. "Mediation Analysis Synthetic Control," Temi di discussione (Economic working papers) 1389, Bank of Italy, Economic Research and International Relations Area.
    7. Advani, Arun & Sloczynski, Tymon, 2013. "Mostly Harmless Simulations? On the Internal Validity of Empirical Monte Carlo Studies," IZA Discussion Papers 7874, Institute of Labor Economics (IZA).
    8. Hugo Bodory & Lorenzo Camponovo & Martin Huber & Michael Lechner, 2020. "The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 183-200, January.
    9. Hugo Bodory & Martin Huber & Michael Lechner, 2024. "The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates," Computational Economics, Springer;Society for Computational Economics, vol. 64(4), pages 2053-2078, October.
    10. Giovanni Mellace & Alessandra Pasquini, 2019. "Identify More, Observe Less: Mediation Analysis Synthetic Control," CEIS Research Paper 474, Tor Vergata University, CEIS, revised 20 Nov 2019.
    11. Bijwaard, Govert & Alessie, Rob & Angelini, Viola, 2018. "The Effect of Early Life Health on Later Life Home Care Use: The Mediating Role of Household Composition," IZA Discussion Papers 11729, Institute of Labor Economics (IZA).
    12. Stefano Lombardi & Gerard J. van den Berg & Johan Vikström, 2024. "Empirical Monte Carlo evidence on estimation of timing-of-events models," Econometric Reviews, Taylor & Francis Journals, vol. 44(1), pages 90-118, September.
    13. Bellani, Luna & Bia, Michela, 2017. "The Long-Run Impact of Childhood Poverty and the Mediating Role of Education," IZA Discussion Papers 10677, Institute of Labor Economics (IZA).
    14. Stephen Whelan, 2017. "Does homeownership affect education outcomes?," IZA World of Labor, Institute of Labor Economics (IZA), pages 342-342, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin Huber & Michael Lechner & Giovanni Mellace, 2017. "Why Do Tougher Caseworkers Increase Employment? The Role of Program Assignment as a Causal Mechanism," The Review of Economics and Statistics, MIT Press, vol. 99(1), pages 180-183, March.
    2. Helmut Farbmacher & Martin Huber & Lukáš Lafférs & Henrika Langen & Martin Spindler, 2022. "Causal mediation analysis with double machine learning [Mediation analysis via potential outcomes models]," The Econometrics Journal, Royal Economic Society, vol. 25(2), pages 277-300.
    3. Martin Huber, 2015. "Causal Pitfalls in the Decomposition of Wage Gaps," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 179-191, April.
    4. Hsu Yu-Chin & Huber Martin & Lai Tsung-Chih, 2019. "Nonparametric estimation of natural direct and indirect effects based on inverse probability weighting," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-20, January.
    5. Martin Huber & Yu‐Chin Hsu & Ying‐Ying Lee & Layal Lettry, 2020. "Direct and indirect effects of continuous treatments based on generalized propensity score weighting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(7), pages 814-840, November.
    6. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    7. Frölich, Markus & Huber, Martin & Wiesenfarth, Manuel, 2017. "The finite sample performance of semi- and non-parametric estimators for treatment effects and policy evaluation," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 91-102.
    8. Viviana Celli, 2022. "Causal mediation analysis in economics: Objectives, assumptions, models," Journal of Economic Surveys, Wiley Blackwell, vol. 36(1), pages 214-234, February.
    9. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    10. Markus Frölich & Martin Huber, 2017. "Direct and indirect treatment effects–causal chains and mediation analysis with instrumental variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1645-1666, November.
    11. Wunsch, Conny & Strobl, Renate, 2018. "Identification of Causal Mechanisms Based on Between-Subject Double Randomization Designs," IZA Discussion Papers 11626, Institute of Labor Economics (IZA).
    12. Hugo Bodory & Lorenzo Camponovo & Martin Huber & Michael Lechner, 2020. "The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 183-200, January.
    13. Huber, Martin, 2012. "Identifying causal mechanisms in experiments (primarily) based on inverse probability weighting," Economics Working Paper Series 1213, University of St. Gallen, School of Economics and Political Science, revised May 2013.
    14. Michael Lechner & Anthony Strittmatter, 2019. "Practical procedures to deal with common support problems in matching estimation," Econometric Reviews, Taylor & Francis Journals, vol. 38(2), pages 193-207, February.
    15. Hugo Bodory & Martin Huber & Lukáš Lafférs, 2022. "Evaluating (weighted) dynamic treatment effects by double machine learning [Identification of causal effects using instrumental variables]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 628-648.
    16. Martin Huber & Michael Lechner & Anthony Strittmatter, 2018. "Direct and indirect effects of training vouchers for the unemployed," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(2), pages 441-463, February.
    17. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.
    18. Ferman, Bruno, 2021. "Matching estimators with few treated and many control observations," Journal of Econometrics, Elsevier, vol. 225(2), pages 295-307.
    19. Martin Huber & Lukáš Lafférs, 2022. "Bounds on direct and indirect effects under treatment/mediator endogeneity and outcome attrition," Econometric Reviews, Taylor & Francis Journals, vol. 41(10), pages 1141-1163, November.
    20. Michael Lechner & Jana Mareckova, 2022. "Modified Causal Forest," Papers 2209.03744, arXiv.org.

    More about this item

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlbes:v:34:y:2016:i:1:p:139-160. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UBES20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.