IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/9847.html
   My bibliography  Save this paper

Preparation, Practice, and Beliefs : A Machine Learning Approach to Understanding Teacher Effectiveness

Author

Listed:
  • Filmer,Deon P.
  • Nahata,Vatsal
  • Sabarwal,Shwetlena

Abstract

This paper uses machine learning methods to identify key predictors of teacher effectiveness,proxied by student learning gains linked to a teacher over an academic year. Conditional inference forests and theleast absolute shrinkage and selection operator are applied to matched student-teacher data for math and Kiswahili fromgrades 2 and 3 in 392 schools across Tanzania. These two machine learning methods produce consistent results andoutperform standard ordinary least squares in out-of-sample prediction by 14–24 percent. As in previous research,commonly used teacher covariates like teacher gender, education, experience, and so forth are not good predictorsof teacher effectiveness. Instead, teacher practice (what teachers do, measured through classroom observations andstudent surveys) and teacher beliefs (measured through teacher surveys) emerge as much more important. Overall,teacher covariates are stronger predictors of teacher effectiveness in math than in Kiswahili. Teacher beliefsthat they can help disadvantaged and struggling studentslearn (for math) and they have good relationships within schools (for Kiswahili), teacher practice of providingwritten feedback and reviewing key concepts at the end of class (for math), and spending extra time with strugglingstudents (for Kiswahili) are highly predictive of teacher effectiveness. As is teacher preparation on how to teachfoundational topics (for both Math and Kiswahili). These results demonstrate the need to pay more systematicattention to teacher preparation, practice, and beliefs in teacher research and policy.

Suggested Citation

  • Filmer,Deon P. & Nahata,Vatsal & Sabarwal,Shwetlena, 2021. "Preparation, Practice, and Beliefs : A Machine Learning Approach to Understanding Teacher Effectiveness," Policy Research Working Paper Series 9847, The World Bank.
  • Handle: RePEc:wbk:wbrwps:9847
    as

    Download full text from publisher

    File URL: http://documents.worldbank.org/curated/en/737101636989604541/pdf/Preparation-Practice-and-Beliefs-A-Machine-Learning-Approach-to-Understanding-Teacher-Effectiveness.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Steven G. Rivkin & Eric A. Hanushek & John F. Kain, 2005. "Teachers, Schools, and Academic Achievement," Econometrica, Econometric Society, vol. 73(2), pages 417-458, March.
    2. Cruz-Aguayo, Yyannú & Ibarrarán, Pablo & Schady, Norbert, 2017. "Do tests applied to teachers predict their effectiveness?," Economics Letters, Elsevier, vol. 159(C), pages 108-111.
    3. Aaron Chalfin & Oren Danieli & Andrew Hillis & Zubin Jelveh & Michael Luca & Jens Ludwig & Sendhil Mullainathan, 2016. "Productivity and Selection of Human Capital with Machine Learning," American Economic Review, American Economic Association, vol. 106(5), pages 124-127, May.
    4. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    5. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    6. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    7. Melissa Adelman & Francisco Haimovich & Andres Ham & Emmanuel Vazquez, 2018. "Predicting school dropout with administrative data: new evidence from Guatemala and Honduras," Education Economics, Taylor & Francis Journals, vol. 26(4), pages 356-372, July.
    8. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    9. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    10. Filmer,Deon P. & Molina,Ezequiel & Wane,Waly, 2020. "Identifying Effective Teachers : Lessons from Four Classroom Observation Tools," Policy Research Working Paper Series 9365, The World Bank.
    11. Andrew Bacher-Hicks & Mark J. Chin & Thomas J. Kane & Douglas O. Staiger, 2017. "An Evaluation of Bias in Three Measures of Teacher Quality: Value-Added, Classroom Observations, and Student Surveys," NBER Working Papers 23478, National Bureau of Economic Research, Inc.
    12. Azam, Mehtabul & Kingdon, Geeta Gandhi, 2015. "Assessing teacher quality in India," Journal of Development Economics, Elsevier, vol. 117(C), pages 74-83.
    13. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    14. Athey, Susan & Imbens, Guido W., 2019. "Machine Learning Methods Economists Should Know About," Research Papers 3776, Stanford University, Graduate School of Business.
    15. Brian A. Jacob & Lars Lefgren, 2005. "Principals as Agents: Subjective Performance Measurement in Education," NBER Working Papers 11463, National Bureau of Economic Research, Inc.
    16. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    17. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    18. Molina,Ezequiel & Fatima,Syeda Farwa & Ho,Andrew Dean Y C & Melo Hurtado,Carolina Eugenia & Wilichowksi,Tracy & Pushparatnam,Adelle, 2018. "Measuring Teaching Practices at Scale : Results from the Development and Validation of the Teach Classroom Observation Tool," Policy Research Working Paper Series 8653, The World Bank.
    19. Dario Sansone, 2019. "Beyond Early Warning Indicators: High School Dropout and Machine Learning," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 81(2), pages 456-485, April.
    20. Paolo Brunori & Paul Hufe & Daniel Gerszon Mahler, 2017. "The Roots of Inequality: Estimating Inequality of Opportunity from Regression Trees," Working Papers - Economics wp2017_18.rdf, Universita' degli Studi di Firenze, Dipartimento di Scienze per l'Economia e l'Impresa.
    21. Sabarwal,Shwetlena & Abu-Jawdeh,Malek, 2018. "What teachers believe : mental models about accountability, absenteeism, and student learning," Policy Research Working Paper Series 8454, The World Bank.
    22. Susan Athey & Guido W. Imbens, 2019. "Machine Learning Methods That Economists Should Know About," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 685-725, August.
    23. Gérard Biau & Erwan Scornet, 2016. "A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 197-227, June.
    24. Fritz Schiltz & Chiara Masci & Tommaso Agasisti & Daniel Horn, 2018. "Using regression tree ensembles to model interaction effects: a graphical approach," Applied Economics, Taylor & Francis Journals, vol. 50(58), pages 6341-6354, December.
    25. Jonah E. Rockoff, 2004. "The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data," American Economic Review, American Economic Association, vol. 94(2), pages 247-252, May.
    26. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    27. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    28. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    29. Isaac Mbiti & Mauricio Romero & Youdi Schipper, 2023. "Designing Effective Teacher Performance Pay Programs: Experimental Evidence from Tanzania," The Economic Journal, Royal Economic Society, vol. 133(653), pages 1968-2000.
    30. Gérard Biau & Erwan Scornet, 2016. "Rejoinder on: A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 264-268, June.
    31. Natalie Bau & Jishnu Das, 2020. "Teacher Value Added in a Low-Income Country," American Economic Journal: Economic Policy, American Economic Association, vol. 12(1), pages 62-96, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marina Bassi & Lelys Dinarte-Diaz & Maria Marta Ferreyra & Sergio Urzua, 2023. "What Makes a Program Good? Evidence from Short-Cycle Higher Education Programs in Five Developing Countries," CESifo Working Paper Series 10255, CESifo.
    2. Dinarte-Diaz, Lelys & Ferreyra, Maria Marta & Urzua, Sergio & Bassi, Marina, 2023. "What makes a program good? Evidence from short-cycle higher education programs in five developing countries," World Development, Elsevier, vol. 169(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Blazar, 2018. "Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students," Education Finance and Policy, MIT Press, vol. 13(3), pages 281-309, Summer.
    2. Dan Goldhaber & Cyrus Grout & Nick Huntington-Klein, 2017. "Screen Twice, Cut Once: Assessing the Predictive Validity of Applicant Selection Tools," Education Finance and Policy, MIT Press, vol. 12(2), pages 197-223, Spring.
    3. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    4. Dhushyanth Raju, 2017. "Public School Teacher Management in Sri Lanka," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 18(1), pages 39-63, March.
    5. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    6. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    7. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    8. Jacob, Brian A. & Rockoff, Jonah E. & Taylor, Eric S. & Lindy, Benjamin & Rosen, Rachel, 2018. "Teacher applicant hiring and teacher performance: Evidence from DC public schools," Journal of Public Economics, Elsevier, vol. 166(C), pages 81-97.
    9. Goel, Deepti & Barooah, Bidisha, 2018. "Drivers of Student Performance: Evidence from Higher Secondary Public Schools in Delhi," GLO Discussion Paper Series 231, Global Labor Organization (GLO).
    10. Wedel, Katharina, 2021. "Instruction time and student achievement: The moderating role of teacher qualifications," Economics of Education Review, Elsevier, vol. 85(C).
    11. Marc Steeg & Sander Gerritsen, 2016. "Teacher Evaluations and Pupil Achievement Gains: Evidence from Classroom Observations," De Economist, Springer, vol. 164(4), pages 419-443, December.
    12. Araujo P., Maria Daniela & Quis, Johanna Sophie, 2021. "Parents can tell! Evidence on classroom quality differences in German primary schools," BERG Working Paper Series 172, Bamberg University, Bamberg Economic Research Group.
    13. Araujo P., María Daniela & Quis, Johanna Sophie, 2021. "Teacher Effects in Germany: Evidence from Elementary School," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242457, Verein für Socialpolitik / German Economic Association.
    14. Murphy, Richard & Weinhardt, Felix & Wyness, Gill, 2021. "Who teaches the teachers? A RCT of peer-to-peer observation and feedback in 181 schools," Economics of Education Review, Elsevier, vol. 82(C).
    15. Goldhaber, Dan & Krieg, John & Theobald, Roddy, 2020. "Effective like me? Does having a more productive mentor improve the productivity of mentees?," Labour Economics, Elsevier, vol. 63(C).
    16. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    17. Azam, Mehtabul & Kingdon, Geeta Gandhi, 2015. "Assessing teacher quality in India," Journal of Development Economics, Elsevier, vol. 117(C), pages 74-83.
    18. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    19. Melinda Adnot & Thomas Dee & Veronica Katz & James Wyckoff, 2016. "Teacher Turnover, Teacher Quality, and Student Achievement in DCPS," NBER Working Papers 21922, National Bureau of Economic Research, Inc.
    20. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:9847. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.