Machine Learning in Evaluative Synthesis

This resource discusses the use of machine learning (ML) techniques in evaluation research and their potential to automate the process of extracting and classifying large amounts of texts. ML methods can accelerate the process of extracting and classifying content in evaluation research provided that practitioners train the extraction tool properly. In practical terms, such an approach can offer evaluators a powerful analytical tool for a range of evaluative purposes, for example, for better understanding the various determinants of project success, potential challenges to project implementation, and practical lessons for future projects, among others. With the above goal in mind, the paper provides an overview of ML and discusses relevant applications in the field of evaluation. This is sup­ported by the case of the Finance and Private Sector Evaluation Unit of the Independent Evalu­ation Group as an example to illustrate the benefits of ML for text classification in evaluation. The paper concludes by offering a summary of the results of this experiment and a brief discussion of potential next steps.