Past performance does not guarantee future results: lessons from the evaluation of research units in Portugal

Research units in Portugal undergo a formal evaluation process based on peer review which is the basis for distributing funding from the national research council. This article analyzes the evaluation results and asks how good they are at predicting future research performance. Better research evaluations mean the institution receives more funding, so the key question is to what extent research evaluations are able to predict future performance as measured by bibliometric indicators. We use data from the peer evaluation of units in 2007–08, and analyze how well it is able to predict the results of a bibliometric study of the units’ Web of Science publications in the period 2007–10. We found that, in general, units that had better peer ratings, and thus more funding, as well as an increased capacity to attract extra funding, were not necessarily those that ended up producing more excellent research. The results provide an empirical contribution to the discussion regarding whether science can be measured and how, and reinforce the importance of evaluations where the use of quantitative data is defined and the differences between areas are accounted for. This analysis provides a snapshot of Portugal's recent scientific performance. Chemistry and physics are among the subfields with higher output and impact, which agrees with a traditional preferential funding of these areas. Institutions also excel in areas that may be assuming an increased relevance (Plant Sciences, Food Science and Technology, Neurosciences and other health-related subfields), which should be taken into account when implementing future science policies.