IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v32y2007i2p491-528.html
   My bibliography  Save this article

Evaluating multi-treatment programs: theory and evidence from the U.S. Job Training Partnership Act experiment

Author

Listed:
  • Miana Plesca
  • Jeffrey Smith

Abstract

This paper considers the evaluation of programs that offer multiple treatments to their participants. Our theoretical discussion outlines the tradeoffs associated with evaluating the program as a whole versus separately evaluating the various individual treatments. Our empirical analysis considers the value of disaggregating multi-treatment programs using data from the U.S. National Job Training Partnership Act Study. This study includes both experimental data, which serve as a benchmark, and non-experimental data. The JTPA experiment divides the program into three treatment “streams” centered on different services. Unlike previous work that analyzes the program as a whole, we analyze the streams separately. Despite our relatively small sample sizes, our findings illustrate the potential for valuable insights into program operation and impact to get lost when aggregating treatments. In addition, we show that many of the lessons drawn from analyzing JTPA as a single treatment carry over to the individual treatment streams.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Miana Plesca & Jeffrey Smith, 2007. "Evaluating multi-treatment programs: theory and evidence from the U.S. Job Training Partnership Act experiment," Empirical Economics, Springer, vol. 32(2), pages 491-528, May.
    • Handle: RePEc:spr:empeco:v:32:y:2007:i:2:p:491-528
    DOI: 10.1007/s00181-006-0095-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00181-006-0095-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s00181-006-0095-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jeffrey Smith & Jeremy Lise & Shannon N. Seitz, 2003. "Equilibrium Policy Experiments And The Evaluation Of Social Programs," Working Paper 1012, Economics Department, Queen's University.
    2. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    3. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    4. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    5. Michael Gerfin & Michael Lechner, 2002. "A Microeconometric Evaluation of the Active Labour Market Policy in Switzerland," Economic Journal, Royal Economic Society, vol. 112(482), pages 854-893, October.
    6. Lechner, Michael & Smith, Jeffrey, 2007. "What is the value added by caseworkers?," Labour Economics, Elsevier, vol. 14(2), pages 135-151, April.
    7. Bernd Fitzenberger & Stefan Speckesser, 2007. "Employment effects of the provision of specific professional skills and techniques in Germany," Empirical Economics, Springer, vol. 32(2), pages 529-573, May.
    8. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.
    9. Alberto Abadie & Guido W. Imbens, 2008. "On the Failure of the Bootstrap for Matching Estimators," Econometrica, Econometric Society, vol. 76(6), pages 1537-1557, November.
    10. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    11. Markus Frölich, 2006. "A Note on Parametric and Nonparametric Regression in the Presence of Endogenous Control Variables," University of St. Gallen Department of Economics working paper series 2006 2006-11, Department of Economics, University of St. Gallen.
    12. James Heckman & Lance Lochner & Christopher Taber, 1998. "Explaining Rising Wage Inequality: Explanations With A Dynamic General Equilibrium Model of Labor Earnings With Heterogeneous Agents," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(1), pages 1-58, January.
    13. Card, David & Sullivan, Daniel G, 1988. "Measuring the Effect of Subsidized Training Programs on Movements in and out of Employment," Econometrica, Econometric Society, vol. 56(3), pages 497-530, May.
    14. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peralta, M. Alexandra & Swinton, Scott M., 2013. "Impact Assessment with Opt-in Treatments: Evidence from a rural development project in Nicaragua," 2013 Annual Meeting, August 4-6, 2013, Washington, D.C. 150988, Agricultural and Applied Economics Association.
    2. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    3. José Ignacio García Pérez & David Troncoso Ponce, 2011. "La introducción de la renta graria: una evaluación del impacto sobre la oferta de trabajo en Andalucía y Extremadura," Hacienda Pública Española / Review of Public Economics, IEF, vol. 199(4), pages 53-80, December.
    4. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
    5. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    6. Vincenzo Caponi & Miana Plesca, 2009. "Post-secondary education in Canada: can ability bias explain the earnings gap between college and university graduates?," Canadian Journal of Economics, Canadian Economics Association, vol. 42(3), pages 1100-1131, August.
    7. Carlos A. Flores & Oscar A. Mitnik, 2013. "Comparing Treatments across Labor Markets: An Assessment of Nonexperimental Multiple-Treatment Strategies," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1691-1707, December.
    8. Chen, Wenjie, 2011. "The effect of investor origin on firm performance: Domestic and foreign direct investment in the United States," Journal of International Economics, Elsevier, vol. 83(2), pages 219-228, March.
    9. Núria Rodríguez-Planas & Benus Jacob, 2010. "Evaluating active labor market programs in Romania," Empirical Economics, Springer, vol. 38(1), pages 65-84, February.
    10. Lee, Ying-Ying, 2018. "Efficient propensity score regression estimators of multivalued treatment effects for the treated," Journal of Econometrics, Elsevier, vol. 204(2), pages 207-222.
    11. Galdo, Jose & Chong, Alberto, 2012. "Does the quality of public-sponsored training programs matter? Evidence from bidding processes data," Labour Economics, Elsevier, vol. 19(6), pages 970-986.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    2. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    3. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    4. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    5. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    6. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    7. Christian Volpe Martincus & Jerónimo Carballo, 2012. "Export promotion activities in developing countries: What kind of trade do they promote?," The Journal of International Trade & Economic Development, Taylor & Francis Journals, vol. 21(4), pages 539-578, June.
    8. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    9. Jespersen, Svend T. & Munch, Jakob R. & Skipper, Lars, 2008. "Costs and benefits of Danish active labour market programmes," Labour Economics, Elsevier, vol. 15(5), pages 859-884, October.
    10. Deborah A. Cobb‐Clark & Thomas Crossley, 2003. "Econometrics for Evaluations: An Introduction to Recent Developments," The Economic Record, The Economic Society of Australia, vol. 79(247), pages 491-511, December.
    11. Quỳnh Tiên Nguyên Lê & Morgan S. Polikoff, 2021. "Do English Language Development Curriculum Materials Matter for Students’ English Proficiency?," SAGE Open, , vol. 11(3), pages 21582440211, July.
    12. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    13. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    14. Christian Volpe Martincus & Jerónimo Carballo, 2012. "Export promotion activities in developing countries: What kind of trade do they promote?," The Journal of International Trade & Economic Development, Taylor & Francis Journals, vol. 21(4), pages 539-578, June.
    15. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    16. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    17. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    18. Olivier Dagnelie & Philippe Lemay‐Boucher, 2012. "Rosca Participation in Benin: A Commitment Issue," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 74(2), pages 235-252, April.
    19. Frölich, Markus & Lechner, Michael, 2010. "Exploiting Regional Treatment Intensity for the Evaluation of Labor Market Policies," Journal of the American Statistical Association, American Statistical Association, vol. 105(491), pages 1014-1029.
    20. Christian Volpe Martincus & Jerónimo Carballo, 2010. "Is Export Promotion Effective in Developing Countries? Firm-Level Evidence on the Intensive and Extensive Margins of Exports," IDB Publications (Working Papers) 36763, Inter-American Development Bank.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:32:y:2007:i:2:p:491-528. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.