IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v35y2010i3p253-279.html
   My bibliography  Save this article

A Model for Teacher Effects From Longitudinal Data Without Assuming Vertical Scaling

Author

Listed:
  • Louis T. Mariano
  • Daniel F. McCaffrey
  • J. R. Lockwood

Abstract

There is an increasing interest in using longitudinal measures of student achievement to estimate individual teacher effects. Current multivariate models assume each teacher has a single effect on student outcomes that persists undiminished to all future test administrations (complete persistence [CP]) or can diminish with time but remains perfectly correlated (variable persistence [VP]). However, when state assessments do not use a vertical scale or the evolution of the mix of topics present across a sequence of vertically aligned assessments changes as students advance in school, these assumptions of persistence may not be consistent with the achievement data. We develop the “generalized persistence†(GP) model, a Bayesian multivariate model for estimating teacher effects that accommodates longitudinal data that are not vertically scaled by allowing less than perfect correlation of a teacher’s effects across test administrations. We illustrate the model using mathematics assessment data.

Suggested Citation

  • Louis T. Mariano & Daniel F. McCaffrey & J. R. Lockwood, 2010. "A Model for Teacher Effects From Longitudinal Data Without Assuming Vertical Scaling," Journal of Educational and Behavioral Statistics, , vol. 35(3), pages 253-279, June.
    • Handle: RePEc:sae:jedbes:v:35:y:2010:i:3:p:253-279
    DOI: 10.3102/1076998609346967
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998609346967
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998609346967?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    2. Thomas Cornelissen, 2008. "The Stata command felsdvreg to fit a linear model with two high-dimensional fixed effects," Stata Journal, StataCorp LP, vol. 8(2), pages 170-189, June.
    3. W. B. Smith & R. R. Hocking, 1972. "Wishart Variate Generator," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 21(3), pages 341-345, November.
    4. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    5. Schacter, John & Thum, Yeow Meng, 2004. "Paying for high- and low-quality teaching," Economics of Education Review, Elsevier, vol. 23(4), pages 411-430, August.
    6. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    7. repec:pri:edures:25ers.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jennifer Broatch & Sharon Lohr, 2012. "Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes," Journal of Educational and Behavioral Statistics, , vol. 37(2), pages 256-277, April.
    2. Derek C. Briggs & Ben Domingue, 2013. "The Gains From Vertical Scaling," Journal of Educational and Behavioral Statistics, , vol. 38(6), pages 551-576, December.
    3. Karl, Andrew T. & Yang, Yan & Lohr, Sharon L., 2013. "Efficient maximum likelihood estimation of multiple membership linear mixed models, with an application to educational value-added assessments," Computational Statistics & Data Analysis, Elsevier, vol. 59(C), pages 13-27.
    4. Andrew T. Karl & Yan Yang & Sharon L. Lohr, 2013. "A Correlated Random Effects Model for Nonignorable Missing Data in Value-Added Assessment of Teacher Effects," Journal of Educational and Behavioral Statistics, , vol. 38(6), pages 577-603, December.
    5. Minjeong Jeon & Sophia Rabe-Hesketh, 2012. "Profile-Likelihood Approach for Estimating Generalized Linear Mixed Models With Factor Structures," Journal of Educational and Behavioral Statistics, , vol. 37(4), pages 518-542, August.
    6. Karl, Andrew T. & Yang, Yan & Lohr, Sharon L., 2014. "Computation of maximum likelihood estimates for multiresponse generalized linear mixed models with non-nested, correlated random effects," Computational Statistics & Data Analysis, Elsevier, vol. 73(C), pages 146-162.
    7. Brendan Houng & Moshe Justman, 2013. "Comparing Least-Squares Value-Added Analysis and Student Growth Percentile Analysis for Evaluating Student Progress and Estimating School Effects," Melbourne Institute Working Paper Series wp2013n07, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    8. Moshe Justman & Brendan Houng, 2013. "A Comparison Of Two Methods For Estimating School Effects And Tracking Student Progress From Standardized Test Scores," Working Papers 1316, Ben-Gurion University of the Negev, Department of Economics.
    9. Stephen Lipscomb & Bing-ru Teh & Brian Gill & Hanley Chiang & Antoniya Owens, "undated". "Teacher and Principal Value-Added: Research Findings and Implementation Practices," Mathematica Policy Research Reports b024faae6179407da5b887263, Mathematica Policy Research.
    10. Savitsky, Terrance & Paddock, Susan, 2014. "Bayesian Semi- and Non-Parametric Models for Longitudinal Data with Multiple Membership Effects in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 57(i03).
    11. repec:mpr:mprres:6941 is not listed on IDEAS
    12. Vosters, Kelly N. & Guarino, Cassandra M. & Wooldridge, Jeffrey M., 2018. "Understanding and evaluating the SAS® EVAAS® Univariate Response Model (URM) for measuring teacher effectiveness," Economics of Education Review, Elsevier, vol. 66(C), pages 191-205.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    2. Papay, John P. & Kraft, Matthew A., 2015. "Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement," Journal of Public Economics, Elsevier, vol. 130(C), pages 105-119.
    3. Ingo E. Isphording & Ulf Zölitz, 2020. "The value of a peer," ECON - Working Papers 342, Department of Economics - University of Zurich.
    4. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    5. Heather C. Hill, 2009. "Evaluating value-added models: A validity argument approach," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 28(4), pages 700-709.
    6. Gary Henry & Roderick Rose & Doug Lauen, 2014. "Are value-added models good enough for teacher evaluations? Assessing commonly used models with simulated and actual data," Investigaciones de Economía de la Educación volume 9, in: Adela García Aracil & Isabel Neira Gómez (ed.), Investigaciones de Economía de la Educación 9, edition 1, volume 9, chapter 20, pages 383-405, Asociación de Economía de la Educación.
    7. repec:pri:cepsud:170rothstein is not listed on IDEAS
    8. Stephen Lipscomb & Hanley Chiang & Brian Gill, "undated". "Value-Added Estimates for Phase 1 of the Pennsylvania Teacher and Principal Evaluation Pilot," Mathematica Policy Research Reports 88151979a8a6469582f1bce32, Mathematica Policy Research.
    9. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    10. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    11. Rockoff, Jonah E. & Speroni, Cecilia, 2011. "Subjective and objective evaluations of teacher effectiveness: Evidence from New York City," Labour Economics, Elsevier, vol. 18(5), pages 687-696, October.
    12. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    13. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    14. Jian Zhang & Songqing Jin & Wei Si, 2020. "Incentivising teachers? Evaluating the incentive effect of China’s teacher performance‐based compensation reform in rural China," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 64(1), pages 171-188, January.
    15. Serena Canaan & Antoine Deeb & Pierre Mouganie, 2022. "Adviser Value Added and Student Outcomes: Evidence from Randomly Assigned College Advisers," American Economic Journal: Economic Policy, American Economic Association, vol. 14(4), pages 151-191, November.
    16. Eric S. Taylor & John H. Tyler, 2011. "The Effect of Evaluation on Performance: Evidence from Longitudinal Student Achievement Data of Mid-career Teachers," NBER Working Papers 16877, National Bureau of Economic Research, Inc.
    17. Tim R. Sass, 2015. "Licensure and Worker Quality: A Comparison of Alternative Routes to Teaching," Journal of Law and Economics, University of Chicago Press, vol. 58(1), pages 1-35.
    18. Goulas Sofoklis & Megalokonomou Rigissa, 2020. "Marathon, Hurdling, or Sprint? The Effects of Exam Scheduling on Academic Performance," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 20(2), pages 1-36, April.
    19. Hermann, Zoltán & Horváth, Hedvig, 2022. "Tanári eredményesség és tanár-diák összepárosítás az általános iskolákban. Empirikus mintázatok három magyarországi tankerület adatai alapján [Teacher effectiveness and teacher-student matching in ," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(11), pages 1377-1406.
    20. Zheng, Lei & Qi, Xiang & Zhang, Chongjiu, 2023. "Can improvements in teacher quality reduce the cognitive gap between urban and rural students in China?," International Journal of Educational Development, Elsevier, vol. 100(C).
    21. Lavy, Victor & Megalokonomou, Rigissa, 2024. "Alternative Measures of Teachers' Value Added and Impact on Short and Long-Term Outcomes: Evidence from Random Assignment," IZA Discussion Papers 17121, Institute of Labor Economics (IZA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:35:y:2010:i:3:p:253-279. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.