Disentangling the language effect in South African schools: Measuring the impact of ‘language of assessment’ in grade 3 literacy and numeracy

The aim of this paper is to exploit an unusual occurrence whereby a large group of South African grade 3 students were tested twice, 1 month apart, on the same test in different languages. Using a simplified difference-in-difference methodology, it becomes possible to identify the causal impact of writing a test in English when English is not a student’s home language for 3402 students. The article aims to address the extent to which language factors (relative to non-language factors) can explain the high levels of underperformance in reading and mathematics in South Africa. I find that the language of assessment effect is between 0.3 and 0.7 standard deviations in literacy and 0 and 0.3 standard deviations in numeracy. This is approximately 1–2 years worth of learning in literacy and 0–1 year worth of learning in numeracy. By contrast, the size of the composite effect of home background and school quality is roughly 4 years worth of learning for both numeracy (1.2 standard deviations) and literacy (1.15 standard deviations). These results clearly show that the ‘language effect’ should be seen within the broader context of a generally dysfunctional schooling system. They further stress the importance of the quality of instruction, not only the language of learning and assessment. The fact that the literacy and numeracy achievement of South African children is so low in grade 3 (prior to any language switch to English in grade 4) should give pause to those who argue that language is the most important factor in determining achievement, or lack thereof, in South Africa.