What can we learn from sharing experience about evaluation practices?

Many resources are available about state-of-the-art or best examples of evaluations and evaluation guidelines, like in the IEPEC and IEPPEC proceedings. However, what do we know about daily evaluation practices? Is it always easy to find examples of evaluations about a given country, type of policy instrument, etc.? Are evaluations used to improve policies? And finally, how could evaluation practices be improved? This is the kind of issues that the European project EPATEE aims at tackling, focusing on impact evaluations. About 60 stakeholders from 13 European countries were interviewed or surveyed to better know their priorities about evaluation issues, how they would define the level of evaluation practices in their country and which barriers might impede effective evaluation practices. In parallel, references were collected and coded to build a knowledge base gathering already more than 170 evaluation reports, papers or guidebooks. Case studies are analyzing more than 20 evaluations to provide concrete and detailed experience feedback about why evaluation is used, how it is performed, what difficulties are encountered, etc. The objective of the project is not to provide an exhaustive or representative picture of the evaluation practices in Europe, but to gather and develop materials that can be used as a basis for experience sharing activities, as well as to develop an online tool box that will make these resources available in a user-friendly way. The key assumption of the project is that concrete examples and guidance can help overcome barriers that currently limit the use of evaluation. Stakeholders' feedback indeed shows that evaluation can be a very effective tool to improve policies, thereby achieving higher energy savings at lower costs. This paper presents the results of the first phase of the project, focusing on the main conclusions from the stakeholders' survey, the knowledge base, and the case studies. Feedbacks gathered remind usual no-brainers (e.g., anticipating data collection). It also shows that evaluation is not only a technical issue but that organizational issues (e.g., cooperation between institutions) are critical as well. Learning by doing can help tackle some of the issues (e.g., optimizing data collection), but some issues remain difficult challenges (e.g., getting robust results about net impacts).