IDEAS home Printed from https://ideas.repec.org/a/tpr/edfpol/v12y2017i3p369-395.html
   My bibliography  Save this article

Assessing Principals’ Assessments: Subjective Evaluations of Teacher Effectiveness in Low- and High-Stakes Environments

Author

Listed:
  • Jason A. Grissom

    (Peabody College Vanderbilt University Nashville, TN 37203-5721)

  • Susanna Loeb

    (Center for Education Policy Analysis Stanford University Stanford, CA 94305-3084)

Abstract

Teacher effectiveness varies substantially, yet principals’ evaluations of teachers often fail to differentiate performance among teachers. We offer new evidence on principals’ subjective evaluations of their teachers’ effectiveness using two sources of data from a large, urban district: principals’ high-stakes personnel evaluations of teachers, and their low-stakes assessments of a subsample of those teachers provided to the researchers. We find that principals’ evaluations of teachers are quite positive whether the stakes are high or low, but the low-stakes evaluations show substantially more use of lower rating categories, and many teachers rated ineffective on the low-stakes assessment receive “effective” or “highly effective” high-stakes ratings. Teacher characteristics, such as experience, partially explain the discrepancy between the two scores. Also, despite the fact that principals overwhelmingly assign teachers to the two highest rating categories on the high-stakes evaluation, their high- and low-stakes ratings show similar correlations with teacher value-added measures.

Suggested Citation

  • Jason A. Grissom & Susanna Loeb, 2017. "Assessing Principals’ Assessments: Subjective Evaluations of Teacher Effectiveness in Low- and High-Stakes Environments," Education Finance and Policy, MIT Press, vol. 12(3), pages 369-395, Summer.
  • Handle: RePEc:tpr:edfpol:v:12:y:2017:i:3:p:369-395
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/EDFP_a_00210
    Download Restriction: Access to PDF is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Steven Glazerman & Susanna Loeb & Dan Goldhaber & Douglas Staiger & Stephen Raudenbush & Grover Whitehurst, "undated". "Evaluating Teachers: The Important Role of Value-Added," Mathematica Policy Research Reports 503e125ed5244bb39586aafe1, Mathematica Policy Research.
    2. Bentley W. MacLeod, 2003. "Optimal Contracting with Subjective Evaluation," American Economic Review, American Economic Association, vol. 93(1), pages 216-240, March.
    3. Lauren Sartain & Matthew P. Steinberg, 2016. "Teachers’ Labor Market Responses to Performance Evaluation Reform: Experimental Evidence from Chicago Public Schools," Journal of Human Resources, University of Wisconsin Press, vol. 51(3), pages 615-655.
    4. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    5. Lewis, Jeffrey B. & Linzer, Drew A., 2005. "Estimating Regression Models in Which the Dependent Variable Is Based on Estimates," Political Analysis, Cambridge University Press, vol. 13(4), pages 345-364.
    6. Darling-Hammond, Linda & Amrein-Beardsley, Audrey & Haertel, Edward & Rothstein, Jesse, 2012. "Evaluating Teacher Evaluation," Department of Economics, Working Paper Series qt63n8q20q, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    7. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    8. Brian A. Jacob & Lars Lefgren, 2008. "Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education," Journal of Labor Economics, University of Chicago Press, vol. 26(1), pages 101-136.
    9. Brian A. Jacob & Lars Lefgren, 2005. "Principals as Agents: Subjective Performance Measurement in Education," NBER Working Papers 11463, National Bureau of Economic Research, Inc.
    10. Jason A. Grissom & Lael R. Keiser, 2011. "A supervisor like me: Race, representation, and the satisfaction and turnover decisions of public sector employees," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(3), pages 557-580, June.
    11. Jonah E. Rockoff & Douglas O. Staiger & Thomas J. Kane & Eric S. Taylor, 2012. "Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools," American Economic Review, American Economic Association, vol. 102(7), pages 3184-3213, December.
    12. Harris, Douglas N. & Sass, Tim R., 2014. "Skills, productivity and the evaluation of teacher performance," Economics of Education Review, Elsevier, vol. 40(C), pages 183-204.
    13. repec:mpr:mprres:6829 is not listed on IDEAS
    14. Jacob, Brian A. & Lefgren, Lars, 2005. "Principals as Agents: Subjective Performance Measures in Education," Working Paper Series rwp05-040, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    2. Jason A. Grissom & Brendan Bartanen, 2022. "Potential Race and Gender Biases in High‐Stakes Teacher Observations," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(1), pages 131-161, January.
    3. Cory Koedel & Jiaxi Li & Matthew G. Springer & Li Tan, 2018. "Teacher Performance Ratings and Professional Improvement," Working Papers 1808, Department of Economics, University of Missouri.
    4. Liebowitz, David D., 2021. "Teacher evaluation for accountability and growth: Should policy treat them as complements or substitutes?," Labour Economics, Elsevier, vol. 71(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    2. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    3. Hanley Chiang & Stephen Lipscomb & Brian Gill, 2014. "Is School Value Added Indicative of Principal Quality?," Mathematica Policy Research Reports a11ab111ac3a497885e0a736f, Mathematica Policy Research.
    4. Hanley Chiang & Stephen Lipscomb & Brian Gill, 2016. "Is School Value Added Indicative of Principal Quality?," Education Finance and Policy, MIT Press, vol. 11(3), pages 283-309, Summer.
    5. Cassandra M. Guarino & Michelle Maxfield & Mark D. Reckase & Paul N. Thompson & Jeffrey M. Wooldridge, 2015. "An Evaluation of Empirical Bayes’s Estimation of Value-Added Teacher Performance Measures," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 190-222, April.
    6. Goel, Deepti & Barooah, Bidisha, 2018. "Drivers of Student Performance: Evidence from Higher Secondary Public Schools in Delhi," GLO Discussion Paper Series 231, Global Labor Organization (GLO).
    7. Pope, Nolan G., 2019. "The effect of teacher ratings on teacher performance," Journal of Public Economics, Elsevier, vol. 172(C), pages 84-110.
    8. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    9. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    10. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    11. David Blazar, 2018. "Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students," Education Finance and Policy, MIT Press, vol. 13(3), pages 281-309, Summer.
    12. Matthew A. Kraft, 2015. "Teacher Layoffs, Teacher Quality, and Student Achievement: Evidence from a Discretionary Layoff Policy," Education Finance and Policy, MIT Press, vol. 10(4), pages 467-507, October.
    13. Elizabeth Dhuey & Justin Smith, 2014. "How important are school principals in the production of student achievement?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 47(2), pages 634-663, May.
    14. repec:hal:spmain:info:hdl:2441/1jgbspo1909q48svne93o55rca is not listed on IDEAS
    15. Alexis Le Chapelain, 2014. "Market for education and student achievement [Marché de l’éducation et réussite scolaire]," SciencePo Working papers tel-03510563, HAL.
    16. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    17. Marc Steeg & Sander Gerritsen, 2016. "Teacher Evaluations and Pupil Achievement Gains: Evidence from Classroom Observations," De Economist, Springer, vol. 164(4), pages 419-443, December.
    18. repec:spo:wpmain:info:hdl:2441/1jgbspo1909q48svne93o55rca is not listed on IDEAS
    19. Alexis Le Chapelain, 2014. "Market for education and student achievement [Marché de l’éducation et réussite scolaire]," SciencePo Working papers Main tel-03510563, HAL.
    20. Harris, Douglas N. & Sass, Tim R., 2014. "Skills, productivity and the evaluation of teacher performance," Economics of Education Review, Elsevier, vol. 40(C), pages 183-204.
    21. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    22. Jenifer Ruiz-Valenzuela & Camille Terrier & Clémentine Van Effenterre, 2017. "Effectiveness of CEOs in the public sector: evidence from further education institutions," CVER Briefing Notes 005, Centre for Vocational Education Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:edfpol:v:12:y:2017:i:3:p:369-395. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kelly McDougall (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.