Research Evaluation as a Policy Design Tool: Mapping Approaches across a Set of Case Studies

This paper provides an overview of research evaluation practices across countries. The main aim is to investigate whether research assessment is implemented and to see to what extent its results are used to revise policy strategies, identify new research priorities, allocate financial resources or enhance public understanding of R&D. The paper addresses a set of cases studies, four within Europe (UK, Finland, Italy, and Spain) and two outside (US and Japan). Each case study provides an outline of the strategies devised to improve the domestic science system; offers a map of the main actors of science policy and introduces the main performers of research assessment. A short overview of how evaluation is approached at European level is also given. The study shows that approaches vary significantly from case to case and that it is not always possible to identify a clear research evaluation framework. In some cases, new strategies have been devised to improve the research system and the process of renovation has affected the structure and the role of research assessment. Overall, official documents across countries emphasise that research evaluation is not a means in itself, and call on its use as a policy design tool. However, very few cases of “management by results” can be identified. The success of research evaluation practice is always tied to strong cultural support and it is where research assessment meets with reluctance and mistrust that it yields no fruit. The absence of an “evaluative culture” is the main obstacle to an efficient research evaluation system.Research Evaluation Systems, Management by Results, Evaluative Culture, Research Policy, Policy Planning

Scientific Advice to Public Policy-Making

A feature of policy-making today is its dependence on scientific advice to deliver public policies that are robust, credible, and effective. This paper discusses how policy-making profits from scientific advice in areas where science and technology are significant. Particular attention is given to issues holding a high level of uncertainty, either because of inherent variability, because science is incomplete or controversial, or because data are inadequate to support a definitive answer. First, we analyse the social context that characterises the relationship between science and policy-making, with a focus on the decrease of public confidence in politicians and scientists. Second, we compare three different sets of guidelines on the collection and use of expertise in policy-making (issued by the UK, Canada and the European Commission, respectively) and identify two different approaches to scientific advice in policy-making. Third, based on a set of cross-national and multi-disciplinary case studies, we look at how the relationship between science and policy-making works in practice and propose a set of recommendations towards the establishment of a more robust and effective policy-making process.Scientific advice, Policy-making, Expertise

Research Evaluation as a Policy Design Tool: Mapping Approaches across a Set of Case Studies

This paper provides an overview of research evaluation practices across countries. The main aim is to investigate whether research assessment is implemented and to see to what extent its results are used to revise policy strategies, identify new research priorities, allocate financial resources or enhance public understanding of R&D. The paper addresses a set of cases studies, four within Europe (UK, Finland, Italy, and Spain) and two outside (US and Japan). Each case study provides an outline of the strategies devised to improve the domestic science system; offers a map of the main actors of science policy and introduces the main performers of research assessment. A short overview of how evaluation is approached at European level is also given. The study shows that approaches vary significantly from case to case and that it is not always possible to identify a clear research evaluation framework. In some cases, new strategies have been devised to improve the research system and the process of renovation has affected the structure and the role of research assessment. Overall, official documents across countries emphasise that research evaluation is not a means in itself, and call on its use as a policy design tool. However, very few cases of "management by results" can be identified. The success of research evaluation practice is always tied to strong cultural support and it is where research assessment meets with reluctance and mistrust that it yields no fruit. The absence of an "evaluative culture" is the main obstacle to an efficient research evaluation system