Publication and patent analysis of European researchers in the field of production technology and manufacturing systems

This paper develops a structured comparison among a sample of European researchers in the field of Production Technology and Manufacturing Systems, on the basis of scientific publications and patents. Researchers are evaluated and compared by a variegated set of indicators concerning (1) the output of individual researchers and (2) that of groups of researchers from the same country. While not claiming to be exhaustive, the results of this preliminary study provide a rough indication of the publishing and patenting activity of researchers in the field of interest, identifying (dis)similarities between different countries. Of particular interest is a proposal for aggregating analysis results by means of maps based on publication and patent indicators. A large amount of empirical data are presented and discusse

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

Research assessment in the humanities: problems and challenges

Research assessment is going to play a new role in the governance of universities and research institutions. Evaluation of results is evolving from a simple tool for resource allocation towards policy design. In this respect "measuring" implies a different approach to quantitative aspects as well as to an estimation of qualitative criteria that are difficult to define. Bibliometrics became so popular, in spite of its limits, just offering a simple solution to complex problems. The theory behind it is not so robust but available results confirm this method as a reasonable trade off between costs and benefits. Indeed there are some fields of science where quantitative indicators are very difficult to apply due to the lack of databases and data, in few words the credibility of existing information. Humanities and social sciences (HSS) need a coherent methodology to assess research outputs but current projects are not very convincing. The possibility of creating a shared ranking of journals by the value of their contents at either institutional, national or European level is not enough as it is raising the same bias as in the hard sciences and it does not solve the problem of the various types of outputs and the different, much longer time of creation and dissemination. The web (and web 2.0) represents a revolution in the communication of research results mainly in the HSS, and also their evaluation has to take into account this change. Furthermore, the increase of open access initiatives (green and gold road) offers a large quantity of transparent, verifiable data structured according to international standards that allow comparability beyond national limits and above all is independent from commercial agents. The pilot scheme carried out at the university of Milan for the Faculty of Humanities demonstrated that it is possible to build quantitative, on average more robust indicators, that could provide a proxy of research production and productiivity even in the HSS

Indicators of university-industry knowledge transfer performance and their implications for universities: evidence from the UK’s HE-BCI survey

Focusing on the measurement of universities’ performance in knowledge transfer, we outline some critical issues connected with the choice of appropriate indicators: in particular, we argue that, in order to allow universities to correctly represent their knowledge transfer performance, indicators should include a variety of knowledge transfer activities, reflect a variety of impacts, allow comparability between institutions, and avoid the creation of perverse behavioural incentives. To illustrate these issues empirically, we discuss the case of the United Kingdom’s Higher Education –Business and Community Interaction (HE-BCI) survey. We show that the indicators used to measure and reward universities’ engagement in knowledge transfer are not fully comprehensive, they are better suited to capture the impact of certain types of activities than others and they are influenced by institutional strategies and characteristics rather than simply reflecting different performances. The conclusions explore some promising directions to address some of these problems

Development and selection of operational management strategies to achieve policy objectives

Since the reform of the EU Common Fisheries Policy in 2002, effort has been devoted to addressing the governance, scientific, social and economic issues required to introduce an ecosystem approach to fisheries management (EAFM) in Europe. Fisheries management needs to support the three pillars of sustainability (ecological, social and economic) and Fisheries Ecosystem Plans (FEPs) have been developed as a tool to assist managers considering the ecological, social and economic implications of their decision. Building upon previous studies (e.g. the FP5-funded European Fisheries Ecosystem Plan project), the core concept of the Making the European Fisheries Ecosystem Plan Operational (MEFEPO) project is to deliver operational frameworks (FEPs) for three regional seas. The project focus is on how best to make current institutional frameworks responsive to an EAFM at regional and pan-European levels in accordance with the principles of good governance. The regional seas selected for the project are the North Sea (NS), North Western Waters (NWW) and South Western Waters (SWW) RAC regions. The aim of this work package (WP5) was to develop operational objectives to achieve the ecological objectives identified for the 3 regional seas in WP2. This report describes the development and implementation of a transparent and formal process that should lead to identification of the “best” operational management strategies for an EAFM, based on sound scientific information and stakeholder involvement (e.g. regional industry groups, citizen groups, managers and other interest groups)

Developments in the educational effectiveness research programme

Educational effectiveness as a research programme moved from an input-output paradigm to an input-process-output paradigm and, in view of the fact that so-called contextual school effectiveness is gaining in importance, this might be more properly termed a context-input-process-output-based approach. The aim of this introductory chapter is to put the state of the art of educational effectiveness research into perspective by summarizing the most important developments in output measurement, the identification of relevant input-, process- and contextual conditions and the causal modeling of these categories. Specific consideration is given to the improvement of substantive multi-level models of educational effectiveness and to available theories that could help to reveal the explanatory mechanisms behind these models

Protocols for Scholarly Communication

CERN, the European Organization for Nuclear Research, has operated an institutional preprint repository for more than 10 years. The repository contains over 850,000 records of which more than 450,000 are full-text OA preprints, mostly in the field of particle physics, and it is integrated with the library's holdings of books, conference proceedings, journals and other grey literature. In order to encourage effective propagation and open access to scholarly material, CERN is implementing a range of innovative library services into its document repository: automatic keywording, reference extraction, collaborative management tools and bibliometric tools. Some of these services, such as user reviewing and automatic metadata extraction, could make up an interesting testbed for future publishing solutions and certainly provide an exciting environment for e-science possibilities. The future protocol for scientific communication should naturally guide authors towards OA publication and CERN wants to help reach a full open access publishing environment for the particle physics community and the related sciences in the next few years.Comment: 8 pages, to appear in Library and Information Systems in Astronomy