IDEAS home Printed from https://ideas.repec.org/p/iab/iabdpa/202029.html
   My bibliography  Save this paper

TippingSens: An R Shiny Application to Facilitate Sensitivity Analysis for Causal Inference Under Confounding

Author

Listed:
  • Haensch, Anna-Carolina

    (Institute for Employment Research (IAB), Nuremberg, Germany ; GESIS Insitute for the Social Sciences)

  • Drechsler, Jörg

    (Institute for Employment Research (IAB), Nuremberg, Germany)

  • Bernhard, Sarah

    (Institute for Employment Research (IAB), Nuremberg, Germany)

Abstract

"Most strategies for causal inference based on quasi-experimental or observational data critically rely on the assumption of unconfoundedness. If this assumption is suspect, sensitivity analysis can be a viable tool to evaluate the impact of confounding on the analysis of interest. One of the earliest proposals for such a sensitivity analysis was suggested by Rosenbaum/ Rubin (1983). However, while it is straightforward to obtain estimates for the causal effect under specific assumptions regarding an unobserved confounder, conducting a full sensitivity analysis based on a range of parameter settings is unwieldy based on the simple forking tables which Rosenbaum and Rubin used. To tackle the multiple parameter problem of the Rosenbaum-Rubin approach, we present an interactive R Shiny application called TippingSens, which visualizes the impact of various parameter settings on the estimated causal effect. Borrowing from the literature on tipping point analysis, the flexible app facilitates manipulating all parameters simultaneously. We demonstrate the usefulness of our app by conducting a sensitivity analysis for a quasi-experiment measuring the effect of vocational training programs on unemployed men. The online supplement accompanying this paper provides a step-by-step introduction to the app using the original illustrative example from Rosenbaum/Rubin (1983)." (Author's abstract, IAB-Doku) ((en))

Suggested Citation

  • Haensch, Anna-Carolina & Drechsler, Jörg & Bernhard, Sarah, 2020. "TippingSens: An R Shiny Application to Facilitate Sensitivity Analysis for Causal Inference Under Confounding," IAB-Discussion Paper 202029, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
  • Handle: RePEc:iab:iabdpa:202029
    as

    Download full text from publisher

    File URL: https://doku.iab.de/discussionpapers/2020/dp2920.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    2. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    3. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    4. Mullahy, John & Sindelar, Jody, 1996. "Employment, unemployment, and problem drinking," Journal of Health Economics, Elsevier, vol. 15(4), pages 409-434, August.
    5. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    6. Patrick Bayer & Stephen L. Ross & Giorgio Topa, 2008. "Place of Work and Place of Residence: Informal Hiring Networks and Labor Market Outcomes," Journal of Political Economy, University of Chicago Press, vol. 116(6), pages 1150-1196, December.
    7. Richard Blundell & Amanda Gosling & Hidehiko Ichimura & Costas Meghir, 2007. "Changes in the Distribution of Male and Female Wages Accounting for Employment Composition Using Bounds," Econometrica, Econometric Society, vol. 75(2), pages 323-363, March.
    8. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    9. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    10. Petra E. Todd & Jeffrey A. Smith, 2001. "Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods," American Economic Review, American Economic Association, vol. 91(2), pages 112-118, May.
    11. Fredriksson, Peter & Johansson, Per, 2008. "Dynamic Treatment Assignment," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 435-445.
    12. Paul R. Rosenbaum, 1999. "Reduced Sensitivity to Hidden Bias at Upper Quantiles in Observational Studies with Dilated Treatment Effects," Biometrics, The International Biometric Society, vol. 55(2), pages 560-564, June.
    13. James J. Heckman & Jora Stixrud & Sergio Urzua, 2006. "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 411-482, July.
    14. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    15. Ziggy MacDonald & Michael A. Shields, 2004. "Does problem drinking affect employment? Evidence from England," Health Economics, John Wiley & Sons, Ltd., vol. 13(2), pages 139-155, February.
    16. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    17. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    18. Charles F. Manski & John V. Pepper, 2009. "More on monotone instrumental variables," Econometrics Journal, Royal Economic Society, vol. 12(s1), pages 200-216, January.
    19. Marion Devaux & Franco Sassi, 2015. "The Labour Market Impacts of Obesity, Smoking, Alcohol Use and Related Chronic Diseases," OECD Health Working Papers 86, OECD Publishing.
    20. Dorner, Matthias & Heining, Jörg & Jacobebbinghaus, Peter & Seth, Stefan, 2010. "Sample of Integrated Labour Market Biographies (SIAB) 1975-2008," FDZ Methodenreport 201009_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    21. Changhui Kang, 2011. "Family Size and Educational Investments in Children: Evidence from Private Tutoring Expenditures in South Korea," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 73(1), pages 59-78, February.
    22. Paul R. Rosenbaum, 2007. "Sensitivity Analysis for m-Estimates, Tests, and Confidence Intervals in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 63(2), pages 456-464, June.
    23. Stefanie Hof, 2014. "Does private tutoring work? The effectiveness of private tutoring: a nonparametric bounds analysis," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 347-366, August.
    24. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 74-90, January.
    25. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    26. Dorner, Matthias & Heining, Jörg & Jacobebbinghaus, Peter & Seth, Stefan, 2010. "Sample of Integrated Labour Market Biographies (SIAB) 1975-2008," FDZ Datenreport. Documentation on Labour Market Data 201001_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    27. Gerrit Mueller & Erik Plug, 2006. "Estimating the Effect of Personality on Male and Female Earnings," ILR Review, Cornell University, ILR School, vol. 60(1), pages 3-22, October.
    28. Pannenberg, Markus, 2010. "Risk attitudes and reservation wages of unemployed workers: Evidence from panel data," Economics Letters, Elsevier, vol. 106(3), pages 223-226, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    3. Jorge Rodríguez & Fernando Saltiel & Sergio Urzúa, 2022. "Dynamic treatment effects of job training," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 242-269, March.
    4. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    5. Tobias Brändle & Lukas Fervers, 2021. "Give it Another Try: What are the Effects of a Job Creation Scheme Especially Designed for Hard-to-Place Workers?," Journal of Labor Research, Springer, vol. 42(3), pages 382-417, December.
    6. Hagen, Tobias, 2016. "Econometric Evaluation of a Placement Coaching Program for Recipients of Disability Insurance Benefits in Switzerland," VfS Annual Conference 2016 (Augsburg): Demographic Change 145736, Verein für Socialpolitik / German Economic Association.
    7. Sianesi, Barbara, 2017. "Evidence of randomisation bias in a large-scale social experiment: The case of ERA," Journal of Econometrics, Elsevier, vol. 198(1), pages 41-64.
    8. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    9. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    10. Thomas Kruppe & Julia Lang, 2018. "Labour market effects of retraining for the unemployed: the role of occupations," Applied Economics, Taylor & Francis Journals, vol. 50(14), pages 1578-1600, March.
    11. Stefanie Hof, 2014. "Does private tutoring work? The effectiveness of private tutoring: a nonparametric bounds analysis," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 347-366, August.
    12. Sarah Bernhard & Thomas Kruppe, 2012. "Effectiveness of Further Vocational Training in Germany – Empirical Findings for Persons Receiving Means-tested Unemployment Benefits," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 132(4), pages 501-526.
    13. Matthew A Masten & Alexandre Poirier, 2023. "Choosing exogeneity assumptions in potential outcome models," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 327-349.
    14. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
    15. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    16. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    17. Cockx, Bart & Lechner, Michael & Bollens, Joost, 2023. "Priority to unemployed immigrants? A causal machine learning evaluation of training in Belgium," Labour Economics, Elsevier, vol. 80(C).
    18. Tsunao Okumura & Emiko Usui, 2014. "Concave‐monotone treatment response and monotone treatment selection: With an application to the returns to schooling," Quantitative Economics, Econometric Society, vol. 5, pages 175-194, March.
    19. Jens Ruhose & Stephan L. Thomsen & Insa Weilage, 2018. "The Wider Benefits of Adult Learning: Work-Related Training and Social Capital," CESifo Working Paper Series 7268, CESifo.
    20. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.

    More about this item

    Keywords

    Determinanten ; Erhebungsmethode ; Experiment ; Intervention ; Kausalanalyse ; Kennzahlen ; Methode ; Änderung ; Arbeitslose ; Validität ; Weiterbildung ; Wirkungsforschung;
    All these keywords.

    JEL classification:

    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iab:iabdpa:202029. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: IAB, Geschäftsbereich Wissenschaftliche Fachinformation und Bibliothek (email available below). General contact details of provider: https://edirc.repec.org/data/iabbbde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.