Impact evaluations: a tool for accountability? Lessons from experience at Agence Française de D�veloppement

This paper relates the Agence Française de D�veloppement's experience with respect to impact evaluations. Our purpose is to assess the extent to which such studies, when designed before an actual programme implementation, can provide the type of summative evidence that donors often seek for when commissioning such studies towards accountability purpose. Specifically, we rely on three large-scale randomised control trials, and scrutinise their capacity to answer questions related to the programme's impact: does the evaluated intervention correspond to the programme's typical implementation conditions? Is the intervention's impact evaluated on the programme's typical beneficiaries? Can the impact evaluation assess the programme's fulfilment of its stated objectives? We conclude that experimental studies should be promoted to answer the type of ‘tunnel’ questions characterised by a limited number of well-specified homogeneous inputs, a tried and tested process, a short and external events-proof causal chain, a large and stable participation, and a set of measurable outcomes in the short run. While a number of such questions exist and are very much worth studying experimentally to inform future development policies, few development interventions themselves satisfy these requirements, and impact evaluations are thus limited in their capacity to provide summative assessment of impact for donors' accountability use.