IDEAS home Printed from https://ideas.repec.org/a/eee/labeco/v15y2008i4p604-618.html
   My bibliography  Save this article

Varying biases in matching estimates: Evidence from two randomised job search training experiments

Author

Listed:
  • Hämäläinen, Kari
  • Uusitalo, Roope
  • Vuori, Jukka

Abstract

We compare various matching estimators with the results from two randomised field experiments that evaluate the employment effects of job search training programmes. We find that commonly used non-experimental matching estimators tend to overestimate the programme effects, especially in the first experiment in which participation in the programme is voluntary. In the second experiment, where caseworkers assign unemployed persons to the training programme, the matching methods produce estimates that are close to the experimental results.

Suggested Citation

  • Hämäläinen, Kari & Uusitalo, Roope & Vuori, Jukka, 2008. "Varying biases in matching estimates: Evidence from two randomised job search training experiments," Labour Economics, Elsevier, vol. 15(4), pages 604-618, August.
    • Handle: RePEc:eee:labeco:v:15:y:2008:i:4:p:604-618
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0927-5371(08)00033-X
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Gerard J. van den Berg & Bas van der Klaauw, 2006. "Counseling And Monitoring Of Unemployed Workers: Theory And Evidence From A Controlled Social Experiment," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 47(3), pages 895-936, August.
    3. Peter Dolton & Donal O'Neill, 2002. "The Long-Run Effects of Unemployment Monitoring and Work-Search Programs: Experimental Evidence from the United Kingdom," Journal of Labor Economics, University of Chicago Press, vol. 20(2), pages 381-403, Part.
    4. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2006. "Evaluating the Differential Effects of Alternative Welfare-to-Work Training Components: A Reanalysis of the California GAIN Program," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 521-566, July.
    5. Alberto Abadie & David Drukker & Jane Leber Herr & Guido W. Imbens, 2004. "Implementing matching estimators for average treatment effects in Stata," Stata Journal, StataCorp LP, vol. 4(3), pages 290-311, September.
    6. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2004. "Evaluating the Employment Impact of a Mandatory Job Search Program," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 569-606, June.
    7. Roberto Agodini & Mark Dynarski, 2004. "Are Experiments the Only Option? A Look at Dropout Prevention Programs," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 180-194, February.
    8. Charles Michalopoulos & Howard S. Bloom & Carolyn J. Hill, 2004. "Can Propensity-Score Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs?," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 156-179, February.
    9. Edwin Leuven & Barbara Sianesi, 2003. "PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing," Statistical Software Components S432001, Boston College Department of Economics, revised 01 Feb 2018.
    10. Gerard van den Berg & Bas van der Klaauw, 2000. "Counseling and Monitoring of Unemployed Workers: Theory and Evidence from a Social Experiment," Econometric Society World Congress 2000 Contributed Papers 0972, Econometric Society.
    11. Howard S. Bloom & Larry L. Orr & Stephen H. Bell & George Cave & Fred Doolittle & Winston Lin & Johannes M. Bos, 1997. "The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 549-576.
    12. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    13. Roberto Agodini & Mark Dynarski, "undated". "Are Experiments the Only Option? A Look at Dropout Prevention Programs," Mathematica Policy Research Reports 51241adbf9fa4a26add6d54c5, Mathematica Policy Research.
    14. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    2. Ana Gorenca & Theodhor Kuro, 2018. "The Sensitivity of the Effect of on-the-Job Training on Employment Outcomes in Experimental and Non-Experimental Settings," Acta Universitatis Danubius. OEconomica, Danubius University of Galati, issue 14(5), pages 141-153, OCTOBER.
    3. Bando, Rosangela & Li, Xia, 2014. "The Effect of In-Service Teacher Training on Student Learning of English as a Second Language," IDB Publications (Working Papers) 6596, Inter-American Development Bank.
    4. Rosangela Bando & Xia Li, 2014. "The Effect of In-Service Teacher Training on Student Learning of English as a Second Language," IDB Publications (Working Papers) 86173, Inter-American Development Bank.
    5. Nekby, Lena, 2008. "Active labor market programs for the integration of youths and immigrants into the labor market: the Nordic experience," Macroeconomía del Desarrollo 5441, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    3. Sudhanshu Handa & John A. Maluccio, 2010. "Matching the Gold Standard: Comparing Experimental and Nonexperimental Evaluation Techniques for a Geographically Targeted Program," Economic Development and Cultural Change, University of Chicago Press, vol. 58(3), pages 415-447, April.
    4. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    5. Andrew P. Jaciw, 2016. "Assessing the Accuracy of Generalized Inferences From Comparison Group Studies Using a Within-Study Comparison Approach," Evaluation Review, , vol. 40(3), pages 199-240, June.
    6. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    7. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
    8. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    9. Biewen, Martin & Fitzenberger, Bernd & Osikominu, Aderonke & Waller, Marie, 2007. "Which Program for Whom? Evidence on the Comparative Effectiveness of Public Sponsored Training Programs in Germany," ZEW Discussion Papers 07-042, ZEW - Leibniz Centre for European Economic Research.
    10. Centeno, Luis & Centeno, Mário & Novo, Álvaro A., 2009. "Evaluating job-search programs for old and young individuals: Heterogeneous impact on unemployment duration," Labour Economics, Elsevier, vol. 16(1), pages 12-25, January.
    11. Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
    12. Kristen Harknett, 2006. "Does Receiving an Earnings Supplement Affect Union Formation? Estimating Effects for Program Participants Using Propensity Score Matching," Evaluation Review, , vol. 30(6), pages 741-778, December.
    13. Stephan Thomsen, 2009. "Job Search Assistance Programs in Europe: Evaluation Methods and Recent Empirical Findings," FEMM Working Papers 09018, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    14. Gonzalo Nunez-Chaim & Henry G. Overman & Capucine Riom, 2024. "Does subsidising business advice improve firm performance? Evidence from a large RCT," CEP Discussion Papers dp1977, Centre for Economic Performance, LSE.
    15. Robin Jacob & Marie-Andree Somers & Pei Zhu & Howard Bloom, 2016. "The Validity of the Comparative Interrupted Time Series Design for Evaluating the Effect of School-Level Interventions," Evaluation Review, , vol. 40(3), pages 167-198, June.
    16. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    17. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    18. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    19. Bukowski, Maciej & Lewandowski, Piotr & Koloch, Grzegorz & Baranowska, Anna & Magda, Iga & Szydlowski, Arkadiusz & Bober, Magda & Bieliński, Jacek & Zawistowski, Julian & Sarzalska, Malgorzata, 2008. "Employment in Poland 2007: Security on flexible labour market," MPRA Paper 14284, University Library of Munich, Germany.
    20. Sauermann, Jan & Stenberg, Anders, 2020. "Assessing Selection Bias in Non-Experimental Estimates of the Returns to Workplace Training," IZA Discussion Papers 13789, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    C93 Job search training Field experiments Matching Unemployment;

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:labeco:v:15:y:2008:i:4:p:604-618. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/labeco .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.