IDEAS home Printed from https://ideas.repec.org/a/tpr/edfpol/v15y2020i3p397-427.html
   My bibliography  Save this article

Can Video Technology Improve Teacher Evaluations? An Experimental Study

Author

Listed:
  • Thomas J. Kane

    (Harvard Graduate School of Education Cambridge, MA 02138)

  • David Blazar

    (College of Education University of Maryland College Park, MD 20742)

  • Hunter Gehlbach

    (School of Education Johns Hopkins University 2800 N. Charles St. Baltimore, MD 21218)

  • Miriam Greenberg

    (Center for Education Policy Research Harvard Graduate School of Education Cambridge, MA 02138)

  • David M. Quinn

    (Rossier School of Education University of Southern California Los Angeles, CA 90089)

  • Daniel Thal

    (Mathematica Cambridge, MA 02139)

Abstract

Teacher evaluation reform has been among the most controversial education reforms in recent years. It also is one of the costliest in terms of the time teachers and principals must spend on classroom observations. We conducted a randomized field trial at four sites to evaluate whether substituting teacher-collected videos for in-person observations could improve the value of teacher observations for teachers, administrators, or students. Relative to teachers in the control group who participated in standard in-person observations, teachers in the video-based treatment group reported that post-observation meetings were more “supportive†and they were more able to identify a specific practice they changed afterward. Treatment principals were able to shift their observation work to noninstructional times. The program also substantially increased teacher retention. Nevertheless, the intervention did not improve students’ academic achievement or self-reported classroom experiences, either in the year of the intervention or for the next cohort of students. Following from the literature on observation and feedback cycles in low-stakes settings, we hypothesize that to improve student outcomes schools may need to pair video feedback with more specific supports for desired changes in practice.

Suggested Citation

  • Thomas J. Kane & David Blazar & Hunter Gehlbach & Miriam Greenberg & David M. Quinn & Daniel Thal, 2020. "Can Video Technology Improve Teacher Evaluations? An Experimental Study," Education Finance and Policy, MIT Press, vol. 15(3), pages 397-427, Summer.
  • Handle: RePEc:tpr:edfpol:v:15:y:2020:i:3:p:397-427
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/edfp_a_00289
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lauren Sartain & Matthew P. Steinberg, 2016. "Teachers’ Labor Market Responses to Performance Evaluation Reform: Experimental Evidence from Chicago Public Schools," Journal of Human Resources, University of Wisconsin Press, vol. 51(3), pages 615-655.
    2. Ben Ost, 2014. "How Do Teachers Improve? The Relative Importance of Specific and General Human Capital," American Economic Journal: Applied Economics, American Economic Association, vol. 6(2), pages 127-151, April.
    3. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    4. Eric S. Taylor & John H. Tyler, 2012. "The Effect of Evaluation on Teacher Performance," American Economic Review, American Economic Association, vol. 102(7), pages 3628-3651, December.
    5. repec:tpr:journl:edfpol:v:10:y:2015:i:4:p:535-572 is not listed on IDEAS
    6. Blazar, David, 2015. "Effective teaching in elementary mathematics: Identifying classroom practices that support student achievement," Economics of Education Review, Elsevier, vol. 48(C), pages 16-29.
    7. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    8. Matthew P. Steinberg & Lauren Sartain, 2015. "Does Teacher Evaluation Improve School Performance? Experimental Evidence from Chicago's Excellence in Teaching Project," Education Finance and Policy, MIT Press, vol. 10(4), pages 535-572, October.
    9. Kraft, Matthew A. & Brunner, Eric J. & Dougherty, Shaun M. & Schwegman, David J., 2020. "Teacher accountability reforms and the supply and quality of new teachers," Journal of Public Economics, Elsevier, vol. 188(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Valentina Dagienė & Eglė Jasutė & Vladimiras Dolgopolovas, 2021. "Professional Development of In-Service Teachers: Use of Eye Tracking for Language Classes, Case Study," Sustainability, MDPI, vol. 13(22), pages 1-24, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pope, Nolan G., 2019. "The effect of teacher ratings on teacher performance," Journal of Public Economics, Elsevier, vol. 172(C), pages 84-110.
    2. Matthew P. Steinberg & Morgaen L. Donaldson, 2016. "The New Educational Accountability: Understanding the Landscape of Teacher Evaluation in the Post-NCLB Era," Education Finance and Policy, MIT Press, vol. 11(3), pages 340-359, Summer.
    3. John P. Papay & Eric S. Taylor & John H. Tyler & Mary Laski, 2016. "Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data," NBER Working Papers 21986, National Bureau of Economic Research, Inc.
    4. Liebowitz, David D., 2021. "Teacher evaluation for accountability and growth: Should policy treat them as complements or substitutes?," Labour Economics, Elsevier, vol. 71(C).
    5. Jacob, Brian A. & Rockoff, Jonah E. & Taylor, Eric S. & Lindy, Benjamin & Rosen, Rachel, 2018. "Teacher applicant hiring and teacher performance: Evidence from DC public schools," Journal of Public Economics, Elsevier, vol. 166(C), pages 81-97.
    6. Murphy, Richard & Weinhardt, Felix & Wyness, Gill, 2021. "Who teaches the teachers? A RCT of peer-to-peer observation and feedback in 181 schools," Economics of Education Review, Elsevier, vol. 82(C).
    7. Papay, John P. & Kraft, Matthew A., 2015. "Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement," Journal of Public Economics, Elsevier, vol. 130(C), pages 105-119.
    8. Bobby Chung & Jian Zou, 2021. "Teacher Licensing, Teacher Supply, and Student Achievement: Nationwide Implementation of edTPA," Working Papers 2021-039, Human Capital and Economic Opportunity Working Group.
    9. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    10. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    11. David Blazar & Matthew A. Kraft, 2015. "Teacher and Teaching Effects on Students' Academic Behaviors and Mindsets," Mathematica Policy Research Reports c1c4216bd08f408fb6bdf16a3, Mathematica Policy Research.
    12. Cory Koedel & Jiaxi Li & Matthew G. Springer & Li Tan, 2016. "The Impact of Performance Ratings on Job Satisfaction for Public School Teachers," Working Papers 1617, Department of Economics, University of Missouri.
    13. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    14. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    15. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.
    16. Lombardi, María, 2019. "Is the remedy worse than the disease? The impact of teacher remediation on teacher and student performance in Chile," Economics of Education Review, Elsevier, vol. 73(C).
    17. Cory Koedel & Jiaxi Li & Matthew G. Springer & Li Tan, 2018. "Teacher Performance Ratings and Professional Improvement," Working Papers 1808, Department of Economics, University of Missouri.
    18. Cory Koedel & Jiaxi Li, 2016. "The Efficiency Implications Of Using Proportional Evaluations To Shape The Teaching Workforce," Contemporary Economic Policy, Western Economic Association International, vol. 34(1), pages 47-62, January.
    19. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    20. Kraft, Matthew A. & Brunner, Eric J. & Dougherty, Shaun M. & Schwegman, David J., 2020. "Teacher accountability reforms and the supply and quality of new teachers," Journal of Public Economics, Elsevier, vol. 188(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:edfpol:v:15:y:2020:i:3:p:397-427. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kelly McDougall (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.