Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

Library Catalog Analysis and Library Holdings Counts: origins, methodological issues and application to the field of Informetrics

Unrevised version to be published in "Evaluative informetrics – the art of metrics based research assessment. Festschrift in honour of Henk F. Moed" , edited by Cinzia Daraio and Wolfgang Glänzel.In 2009, Torres-Salinas & Moed proposed the use of library catalogs to analyze the impact and dissemination of academic books in different ways. Library Catalog Analysis (LCA) can be defined as the application of bibliometric techniques to a set of online library catalogs in order to describe quantitatively a scientific-scholarly field on the basis of published book titles. The aim of the present chapter is to conduct an in-depth analysis of major scientific contributions since the birth of LCA in order to determine the state of the art of this research topic. Hence, our specific objectives are: 1) to discuss the original purposes of library holdings 2) to present correlations between library holdings and altmetrics indicators and interpret their feasible meanings 3) to analyze the principal sources of information 4) to use WorldCat Identities to identify the principal authors and works in the field of Informetrics

Google Scholar as a Citation Database for Non-bibliometric Areas: The EVA Project Results

In this chapter, we present the EVA (Extraction, Validation, and Analysis) project and related results about the use of Google Scholar as web database for calculation of citation indexes in non-bibliometric scientific areas, such as social sciences and humanities. The results of the EVA project are presented on a case-study about the publication records retrieved from Google Scholar for a dataset of Italian academic researchers belonging to non-bibliometric scientific areas. The evaluation results against the Elsevier-Scopus citation database are also discussed

Do social sciences and humanities behave like life and hard sciences?

The quantitative evaluation of Social Science and Humanities (SSH) and the investigation of the existing similarities between SSH and Life and Hard Sciences (LHS) represent the forefront of scientometrics research. We analyse the scientific production of the universe of Italian academic scholars , over a 10-year period across 2002–2012, from a national database built by the Italian National Agency for the Evaluation of Universities and Research Institutes. We demonstrate that all Italian scholars of SSH and LHS are equals, as far as their publishing habits. They share the same general law, which is a lognormal. At the same time, however, they are different, because we measured their scientific production with different indicators required by the Italian law; we eliminated the “silent” scholars and obtained different scaling values—proxy of their productivity rates. Our findings may be useful to further develop indirect quali–quantitative comparative analysis across heterogeneous disciplines and, more broadly, to investigate on the generative mechanisms behind the observed empirical regularities. © 2017, Akadémiai Kiadó, Budapest, Hungary

Peer Review in SSH. Addressing the Interpretation of Quality Criteria

The chapter examines the experience of High keve exoert groups launched in the Italian experience in order to fine tune the instruments for research evaluatio in SSH, namely the peer review questionnaires, formats and scores. It also gives an account of the main contriversies surrounding the evaluation

Beyond coverage : Toward a bibliometrics for the humanities

 In this chapter, the possibility of using bibliometric measures for evaluating research in the humanities is pondered. A review of recent attempts to develop bibliometric methods for studying the humanities shows that organizational, epistemological differences as well as distinct research practices in research fields ought to be considered. The dependence on colleagues, interdisciplinarity and the ‘rural’ nature of research in many humanistic disciplines are identified as factors that influence the possibilities of applying bibliometric methods. A few particularly promising approaches are highlighted, and the possibility of developing a ‘bibliometrics for the humanities’ is examined. Finally, the intellectual characteristics of specific disciplines should be considered when quality indicators are constructed, and the importance of including scholars from the humanities in the process is stressed