IDEAS home Printed from https://ideas.repec.org/p/sza/wpaper/wpapers246.html
   My bibliography  Save this paper

The impact of study guides on “matric” performance: Evidence from a randomised experiment

Author

Listed:
  • Stephen Taylor

    (Department of Economics and Department of Basic Education)

  • Patricia Watson

    (Department of Basic Education)

Abstract

Most international literature on the impact of textbooks on educational achievement suggests that this is a relatively cost-effective intervention. However, recent experimental evidence from developing countries has called this into question, suggesting that resources alone are unlikely to impact on performance and that changes in school organisation, pedagogical methods or incentives facing teachers are more effective. South African studies, using observational data, typically show weak associations between achievement and additional resources, though in some studies textbooks emerge as an exception. Some argue that school management is a key mediating variable. This paper evaluates the impact of providing study guides to pupils shortly before their secondary school leaving examination (the “matric” exam). From a sampling frame of 318 schools in the Mpumalanga province, 79 schools were randomly selected to receive study guides, leaving 239 control schools. These study guides were developed by the National Department of Basic Education and distributed to treatment schools for four subjects – accounting, economics, geography and life sciences – resulting in four distinct treatments per school. The impact of the study guides was estimated using matric results from 2011 (baseline) and 2012 (endline). The accounting and economics guides did not have a significant impact on performance. However, the geography and life sciences guides improved scores in those subjects by approximately two percentage points. Treatment heterogeneity was apparent for geography where students in better-performing schools gained more from the guides than students in low-performing schools. This may relate to other studies suggesting that additional school resources matter conditionally upon overall school functionality, particularly management. A simulation indicated that distributing the geography and life science at scale could increase the overall matric pass rate by roughly one percentage point. A cost-benefit analysis calculating the standard deviations of impact on test scores per $100 spent indicates that this intervention is amongst the most cost-effective of educational interventions internationally that have been tested using randomised experiments. Possible reasons why the guides were effective in two subjects but not the other two are discussed.

Suggested Citation

  • Stephen Taylor & Patricia Watson, 2015. "The impact of study guides on “matric” performance: Evidence from a randomised experiment," Working Papers 13/2015, Stellenbosch University, Department of Economics.
  • Handle: RePEc:sza:wpaper:wpapers246
    as

    Download full text from publisher

    File URL: https://www.ekon.sun.ac.za/wpapers/2015/wp132015/wp-13-2015.pdf
    File Function: First version, 2015
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1235-1264.
    2. Pritchett, Lant & Beatty, Amanda, 2012. "The Negative Consequences of Overambitious Curricula in Developing Countries," Working Paper Series rwp12-035, Harvard University, John F. Kennedy School of Government.
    3. Servaas van der Berg, 2006. "How effective are poor schools? Poverty and educational outcomes in South Africa," Working Papers 06/2006, Stellenbosch University, Department of Economics.
    4. Nicholas Spaull & Janeli Kotze, 2014. "Starting Behind and Staying Behind in South Africa: The case of insurmountable learning deficits in mathematics," Working Papers 27/2014, Stellenbosch University, Department of Economics.
    5. Sylvie Moulin & Michael Kremer & Paul Glewwe, 2009. "Many Children Left Behind? Textbooks and Test Scores in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 112-135, January.
    6. Abhijit V. Banerjee & Rukmini Banerji & Esther Duflo & Rachel Glennerster & Stuti Khemani, 2010. "Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India," American Economic Journal: Economic Policy, American Economic Association, vol. 2(1), pages 1-30, February.
    7. Lant Pritchett and Amanda Beatty, 2012. "The Negative Consequences of Overambitious Curricula in Developing Countries - Working Paper 293," Working Papers 293, Center for Global Development.
    8. Nic Spaull, 2011. "A Preliminary Analysis of SACMEQ III South Africa," Working Papers 11/2011, Stellenbosch University, Department of Economics.
    9. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    10. Spaull, Nicholas & Kotze, Janeli, 2015. "Starting behind and staying behind in South Africa," International Journal of Educational Development, Elsevier, vol. 41(C), pages 13-24.
    11. Eric A. Hanushek, 2003. "The Failure of Input-Based Schooling Policies," Economic Journal, Royal Economic Society, vol. 113(485), pages 64-98, February.
    12. Martin Gustafsson, 2007. "Using The Hierarchical Linear Model To Understand School Production In South Africa," South African Journal of Economics, Economic Society of South Africa, vol. 75(1), pages 84-98, March.
    13. Nicola Branson & Julia Garlick & David Lam & Murray Leibbrandt, 2012. "Education and Inequality: The South African Case," SALDRU Working Papers 75, Southern Africa Labour and Development Research Unit, University of Cape Town.
    14. Debra L. Shepherd, 2011. "Constraints to school effectiveness: what prevents poor schools from delivering results?," Working Papers 05/2011, Stellenbosch University, Department of Economics.
    15. Stephen Taylor, 2011. "Uncovering indicators of effective school management in South Africa using the National School Effectiveness Study," Working Papers 10/2011, Stellenbosch University, Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Martin Gustafsson & Stephen Taylor, 2013. "Treating schools to a new administration. The impact of South Africa’s 2005 provincial boundary changes on school performance," Working Papers 28/2013, Stellenbosch University, Department of Economics.
    2. Grace Bridgman, 2020. "Correspondence between mathematics and mathematical literacy scores: an analysis from 2010 to 2018," Working Papers 03/2020, Stellenbosch University, Department of Economics.
    3. Martin Gustafsson & Stephen Taylor, 2016. "Treating schools to a new administration: Evidence from South Africa of the impact of better practices in the system-level administration of schools," Working Papers 05/2016, Stellenbosch University, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vegas, E & Ganimian, A. J., 2013. "Theory and Evidence on Teacher Policies in Developed and Developing Countries," Working Paper 104291, Harvard University OpenScholar.
    2. Servaas van der Berg & Gabrielle Wills & Rebecca Selkirk & Charles Adams & Chris van Wyk, 2019. "The cost of repetition in South Africa," Working Papers 13/2019, Stellenbosch University, Department of Economics.
    3. Gabrielle Wills & Servaas van der Berg, 2018. "Measuring leadership and management and their linkages with literacy in rural and township primary schools in South Africa," Working Papers 21/2018, Stellenbosch University, Department of Economics.
    4. Servaas van der Berg, 2015. "What the Annual National Assessments can tell us about learning deficits over the education system and the school career year," Working Papers 18/2015, Stellenbosch University, Department of Economics.
    5. Heleen Hofmeyr, 2019. "Performance Beyond Expectations: Academic Resilience in South Africa," Working Papers 19/2019, Stellenbosch University, Department of Economics.
    6. Oscar David Marcenaro‐Gutierrez & Luis Alejandro Lopez‐Agudo, 2020. "Does Teacher Subject Knowledge Contribute to Student Academic Performance in Developing and Least Developed Countries?," South African Journal of Economics, Economic Society of South Africa, vol. 88(3), pages 267-297, September.
    7. Nakajima, Maki & Kijima, Yoko & Otsuka, Keijiro, 2018. "Is the learning crisis responsible for school dropout? A longitudinal study of Andhra Pradesh, India," International Journal of Educational Development, Elsevier, vol. 62(C), pages 245-253.
    8. Dumisani Hompashe, 2018. "Instructional leadership and academic performance: Eastern Cape educators’ perceptions and quantitative evidence," Working Papers 13/2018, Stellenbosch University, Department of Economics.
    9. Elizabeth Beasley & Elise Huillery, 2017. "Willing but Unable? Short-term Experimental Evidence on Parent Empowerment and School Quality," The World Bank Economic Review, World Bank, vol. 31(2), pages 531-552.
    10. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    11. Maruyama, Takao & Kurosaki, Takashi, 2021. "Do remedial activities using math workbooks improve student learning? Empirical evidence from scaled-up interventions in Niger," World Development, Elsevier, vol. 148(C).
    12. Jones, Sam, 2016. "How does classroom composition affect learning outcomes in Ugandan primary schools?," International Journal of Educational Development, Elsevier, vol. 48(C), pages 66-78.
    13. Elizabeth Beasley & Elise Huillery, 2015. "Willing but Unable: Short-Term Experimental Evidence on Parent Empowerment and School Quality," SciencePo Working papers hal-03460075, HAL.
    14. Nadel, Sara & Pritchett, Lant, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Paper Series rwp16-041, Harvard University, John F. Kennedy School of Government.
    15. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    16. Eduard Marinov, 2019. "The 2019 Nobel Prize in Economics," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 6, pages 78-116.
    17. Abhijit V. Banerjee & Esther Duflo, 2010. "Giving Credit Where It Is Due," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 61-80, Summer.
    18. Islam, Asad, 2019. "Parent–teacher meetings and student outcomes: Evidence from a developing country," European Economic Review, Elsevier, vol. 111(C), pages 273-304.
    19. Asmus Zoch, 2017. "The effect of neighbourhoods and school quality on education and labour market outcomes in South Africa," Working Papers 08/2017, Stellenbosch University, Department of Economics.
    20. León, Gianmarco & Valdivia, Martín, 2015. "Inequality in school resources and academic achievement: Evidence from Peru," International Journal of Educational Development, Elsevier, vol. 40(C), pages 71-84.

    More about this item

    Keywords

    South Africa; education; educational achievement; study guides; Randomised Controlled Trial;
    All these keywords.

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sza:wpaper:wpapers246. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Melt van Schoor (email available below). General contact details of provider: https://edirc.repec.org/data/desunza.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.