Research Evaluation as a Policy Design Tool: Mapping Approaches Across a Set of Case Studies

Abstract

This paper provides an overview of research evaluation practices across countries. The main aim is to investigate whether research assessment is implemented and to see to what extent its results are used to revise policy strategies, identify new research priorities, allocate financial resources or enhance public understanding of R&D. The paper addresses a set of cases studies, four within Europe (UK, Finland, Italy, and Spain) and two outside (US and Japan). Each case study provides an outline of the strategies devised to improve the domestic science system; offers a map of the main actors of science policy and introduces the main performers of research assessment. A short overview of how evaluation is approached at European level is also given. The study shows that approaches vary significantly from case to case and that it is not always possible to identify a clear research evaluation framework. In some cases, new strategies have been devised to improve the research system and the process of renovation has affected the structure and the role of research assessment. Overall, official documents across countries emphasise that research evaluation is not a means in itself, and call on its use as a policy design tool. However, very few cases of "management by results" can be identified. The success of research evaluation practice is always tied to strong cultural support and it is where research assessment meets with reluctance and mistrust that it yields no fruit. The absence of an "evaluative culture" is the main obstacle to an efficient research evaluation system.

Keywords: Research Evaluation Systems, Management by Results, Evaluative Culture, Research Policy, Policy Planning