Institutions, implementation, and program effectiveness: Evidence from a randomized evaluation of computer-assisted learning in rural China

There is limited evidence on the degree to which differences in implementation among institutions matter for program effectiveness. To examine this question, we conducted an experiment in rural China in which public schools were randomly assigned to one of three treatments: a computer-assisted learning program (CAL) implemented by a government agency, the same program implemented by an NGO, and a pure control. Results show that compared to the pure control condition and unlike the NGO program, the government program did not improve student achievement. Analyzing impacts along the causal chain, we find that government officials were more likely to substitute CAL for regular instruction (contrary to protocol) and less likely to directly monitor program progress. Correlational analyses suggest that these differences in program implementation were responsible for the lack of impacts.