Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

The structural role of the core literature in history

The intellectual landscapes of the humanities are mostly uncharted territory. Little is known on the ways published research of humanist scholars defines areas of intellectual activity. An open question relates to the structural role of core literature: highly cited sources, naturally playing a disproportionate role in the definition of intellectual landscapes. We introduce four indicators in order to map the structural role played by core sources into connecting different areas of the intellectual landscape of citing publications (i.e. communities in the bibliographic coupling network). All indicators factor out the influence of degree distributions by internalizing a null configuration model. By considering several datasets focused on history, we show that two distinct structural actions are performed by the core literature: a global one, by connecting otherwise separated communities in the landscape, or a local one, by rising connectivity within communities. In our study, the global action is mainly performed by small sets of scholarly monographs, reference works and primary sources, while the rest of the core, and especially most journal articles, acts mostly locally

A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments

Making visible the invisible through the analysis of acknowledgements in the humanities

Purpose: Science is subject to a normative structure that includes how the contributions and interactions between scientists are rewarded. Authorship and citations have been the key elements within the reward system of science, whereas acknowledgements, despite being a well-established element in scholarly communication, have not received the same attention. This paper aims to put forward the bearing of acknowledgements in the humanities to bring to the foreground contributions and interactions that, otherwise, would remain invisible through traditional indicators of research performance. Design/methodology/approach: The study provides a comprehensive framework to understanding acknowledgements as part of the reward system with a special focus on its value in the humanities as a reflection of intellectual indebtedness. The distinctive features of research in the humanities are outlined and the role of acknowledgements as a source of contributorship information is reviewed to support these assumptions. Findings: Peer interactive communication is the prevailing support thanked in the acknowledgements of humanities, so the notion of acknowledgements as super-citations can make special sense in this area. Since single-authored papers still predominate as publishing pattern in this domain, the study of acknowledgements might help to understand social interactions and intellectual influences that lie behind a piece of research and are not visible through authorship. Originality/value: Previous works have proposed and explored the prevailing acknowledgement types by domain. This paper focuses on the humanities to show the role of acknowledgements within the reward system and highlight publication patterns and inherent research features which make acknowledgements particularly interesting in the area as reflection of the socio-cognitive structure of research.Comment: 14 page

Measuring the vitality and effectiveness within social sciences and humanities research. an attempt in Italian LIS studies

This study constitutes the beginning of a PhD research in Library and information science at Sapienza University of Rome, aimed at investigating the level of internationalisation, vitality and effectivenessof Italian Library and information studies. The paper describes the methodology of search in international citational database, Web of science and Scopus, and in Google scholar for any scholarly work published by Italian LIS tenured researchers and university professors. The results in WoS and Scopus are illustrated and compared with a first collection of data within Italian similar field of Social sciences and humanities (SSH). The reliability of the data collected in citation database within the areas of Social sciences and humanities and their significance are limited; also the comparison with Italian historical and paleographic fields seems to confirms that a substantial impact of SSH non-Anglophone scientific production within citational database is influenced by a number of factors (number of indexed core journals, language of publication, level of coverage of journals, etc.). On the other hand, Google scholar could be a valuable tool for humanistic fields, but it is necessary to deal with the comple xity of the analysis and the problems of homonyms. Therefore, these first research results made it clear that, in order to investigate the presence and impact of Italian LIS studies, a correct use and organization of quantitative data is essential