IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v53y2016icp31-45.html
   My bibliography  Save this article

Teacher quality differences between teacher preparation programs: How big? How reliable? Which programs are different?

Author

Listed:
  • von Hippel, Paul T.
  • Bellows, Laura
  • Osborne, Cynthia
  • Lincove, Jane Arnold
  • Mills, Nick

Abstract

Sixteen US states have begun to hold teacher preparation programs (TPPs) accountable for teacher quality, where quality is estimated by teacher value-added to student test scores. Yet it is not easy to identify TPPs whose teachers are substantially better or worse than average. True teacher quality differences between TPPs are small; estimated differences are not very reliable; and when many TPPs are compared, multiple comparisons increase the danger of misclassifying ordinary TPPs as good or bad. Using a large and diverse dataset from Texas, we evaluate statistical methods for estimating teacher quality differences between TPPs. The most convincing estimates come from a value-added model where confidence intervals are widened by the inclusion of teacher random effects (or teacher clustering in large TPPs) and further widened by the Bonferroni correction for multiple comparisons. Using these confidence intervals, it is rarely possible to tell which TPPs, if any, are better or worse than average. The potential benefits of TPP accountability may be too small to balance the risk that a proliferation of noisy TPP estimates will encourage arbitrary and ineffective policy actions.

Suggested Citation

  • von Hippel, Paul T. & Bellows, Laura & Osborne, Cynthia & Lincove, Jane Arnold & Mills, Nick, 2016. "Teacher quality differences between teacher preparation programs: How big? How reliable? Which programs are different?," Economics of Education Review, Elsevier, vol. 53(C), pages 31-45.
    • Handle: RePEc:eee:ecoedu:v:53:y:2016:i:c:p:31-45
    DOI: 10.1016/j.econedurev.2016.05.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775716302503
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2016.05.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Guido W. Imbens & Michal Kolesár, 2016. "Robust Standard Errors in Small Samples: Some Practical Advice," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 701-712, October.
    2. Papay, John P. & Kraft, Matthew A., 2015. "Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement," Journal of Public Economics, Elsevier, vol. 130(C), pages 105-119.
    3. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    4. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    5. Mariesa Herrmann & Elias Walsh & Eric Isenberg & Alexandra Resch, 2013. "Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels," Mathematica Policy Research Reports 2b140369be0242ac83eeb5b0a, Mathematica Policy Research.
    6. Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Woolrdige, 2014. "Can Value-Added Measures of Teacher Performance Be Trusted?," Education Finance and Policy, MIT Press, vol. 10(1), pages 117-156, November.
    7. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    8. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    9. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    10. Goldhaber, Dan & Liddle, Stephanie & Theobald, Roddy, 2013. "The gateway to the profession: Assessing teacher preparation programs based on student achievement," Economics of Education Review, Elsevier, vol. 34(C), pages 29-44.
    11. Koedel, Cory, 2009. "An empirical analysis of teacher spillover effects in secondary school," Economics of Education Review, Elsevier, vol. 28(6), pages 682-692, December.
    12. Wiswall, Matthew, 2013. "The dynamics of teacher quality," Journal of Public Economics, Elsevier, vol. 100(C), pages 61-78.
    13. repec:tpr:journl:edfpol:v:10:y:2015:i:4:p:508-534 is not listed on IDEAS
    14. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    15. Kata Mihaly & Daniel McCaffrey & Tim R. Sass & J. R. Lockwood, 2013. "Where You Come From or Where You Go? Distinguishing Between School Quality and the Effectiveness of Teacher Preparation Program Graduates," Education Finance and Policy, MIT Press, vol. 8(4), pages 459-493, October.
    16. Green, Donald P. & Vavreck, Lynn, 2008. "Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches," Political Analysis, Cambridge University Press, vol. 16(2), pages 138-152, April.
    17. repec:mpr:mprres:7748 is not listed on IDEAS
    18. John P. Conley & Ali Sina Onder, 2014. "The Research Productivity of New PhDs in Economics: The Surprisingly High Non-success of the Successful," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 205-216, Summer.
    19. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    20. Cory Koedel & Eric Parsons & Michael Podgursky & Mark Ehlert, 2015. "Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs?," Education Finance and Policy, MIT Press, vol. 10(4), pages 508-534, October.
    21. Rumberger, Russell W. & Thomas, Scott L., 1993. "The economic returns to college major, quality and performance: A multilevel analysis of recent graduates," Economics of Education Review, Elsevier, vol. 12(1), pages 1-19, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    2. von Hippel, Paul T. & Bellows, Laura, 2018. "How much does teacher quality vary across teacher preparation programs? Reanalyses from six states," Economics of Education Review, Elsevier, vol. 64(C), pages 298-312.
    3. Backes, Ben & Goldhaber, Dan & Cade, Whitney & Sullivan, Kate & Dodson, Melissa, 2018. "Can UTeach? Assessing the relative effectiveness of STEM teachers," Economics of Education Review, Elsevier, vol. 64(C), pages 184-198.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    2. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    3. Cory Koedel & Eric Parsons & Michael Podgursky & Mark Ehlert, 2015. "Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs?," Education Finance and Policy, MIT Press, vol. 10(4), pages 508-534, October.
    4. Dhushyanth Raju, 2017. "Public School Teacher Management in Sri Lanka," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 18(1), pages 39-63, March.
    5. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    6. Wedel, Katharina, 2021. "Instruction time and student achievement: The moderating role of teacher qualifications," Economics of Education Review, Elsevier, vol. 85(C).
    7. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    8. Goldhaber, Dan & Krieg, John & Theobald, Roddy, 2020. "Effective like me? Does having a more productive mentor improve the productivity of mentees?," Labour Economics, Elsevier, vol. 63(C).
    9. Filmer,Deon P. & Nahata,Vatsal & Sabarwal,Shwetlena, 2021. "Preparation, Practice, and Beliefs : A Machine Learning Approach to Understanding Teacher Effectiveness," Policy Research Working Paper Series 9847, The World Bank.
    10. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.
    11. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    12. Jacob, Brian A. & Rockoff, Jonah E. & Taylor, Eric S. & Lindy, Benjamin & Rosen, Rachel, 2018. "Teacher applicant hiring and teacher performance: Evidence from DC public schools," Journal of Public Economics, Elsevier, vol. 166(C), pages 81-97.
    13. von Hippel, Paul T. & Bellows, Laura, 2018. "How much does teacher quality vary across teacher preparation programs? Reanalyses from six states," Economics of Education Review, Elsevier, vol. 64(C), pages 298-312.
    14. Zheng, Qi & Xie, Xin & Ye, Xiaoyang & Wei, Yi, 2024. "Do teachers colleges prepare more effective teachers? Evidence from a top school district in China," China Economic Review, Elsevier, vol. 87(C).
    15. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    16. Stacy, Brian, 2014. "Ranking Teachers when Teacher Value-Added is Heterogeneous Across Students," EconStor Preprints 104743, ZBW - Leibniz Information Centre for Economics.
    17. James G. MacKinnon, 2019. "How cluster‐robust inference is changing applied econometrics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 52(3), pages 851-881, August.
    18. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    19. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.
    20. Helen F. Ladd & Lucy C. Sorensen, 2017. "Returns to Teacher Experience: Student Achievement and Motivation in Middle School," Education Finance and Policy, MIT Press, vol. 12(2), pages 241-279, Spring.

    More about this item

    Keywords

    Accountability; Teacher training; Heterogeneity;
    All these keywords.

    JEL classification:

    • I12 - Health, Education, and Welfare - - Health - - - Health Behavior
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:53:y:2016:i:c:p:31-45. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.