Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

Library Catalog Analysis and Library Holdings Counts: origins, methodological issues and application to the field of Informetrics

Unrevised version to be published in "Evaluative informetrics – the art of metrics based research assessment. Festschrift in honour of Henk F. Moed" , edited by Cinzia Daraio and Wolfgang Glänzel.In 2009, Torres-Salinas & Moed proposed the use of library catalogs to analyze the impact and dissemination of academic books in different ways. Library Catalog Analysis (LCA) can be defined as the application of bibliometric techniques to a set of online library catalogs in order to describe quantitatively a scientific-scholarly field on the basis of published book titles. The aim of the present chapter is to conduct an in-depth analysis of major scientific contributions since the birth of LCA in order to determine the state of the art of this research topic. Hence, our specific objectives are: 1) to discuss the original purposes of library holdings 2) to present correlations between library holdings and altmetrics indicators and interpret their feasible meanings 3) to analyze the principal sources of information 4) to use WorldCat Identities to identify the principal authors and works in the field of Informetrics

Performance-based publisher ratings and the visibility/impact of books:Small fish in a big pond, or big fish in a small pond?

This study compares publisher ratings to the visibility and impact of individual books, based on a 2017 data set from three Nordic performance-based research funding systems (PRFS) (Denmark, Norway, and Finland). Although there are Journal Impact Factors ( JIFs) for journals, there is no similar indicator for book publishers. National publisher lists are used instead to account for the general “quality” of books, leading to institutional rewards. But, just as the JIF is not recommended as a proxy for the “citedness” of a paper, a publisher rating is also not recommended as a proxy for the impact of an individual book. We introduce a small fish in a big pond versus big fish in a small pond metaphor, where a “fish” is a book and “the pond” represents its publishing house. We investigate how books fit on this metaphorical fish and pond continuum, using WorldCat holdings (visibility) and Google Scholar citations (impact), and test other variables to determine their predictive value with respect to these two indicators. Our statistics show that publisher levels do not have predictive value when other variables are held constant. This has implications for PRFS and book evaluations in general, as well as ongoing developments related to a newly proposed international publisher registry.Peer reviewe

Exploring WorldCat identities as an altmetric information source: A library catalog analysis experiment in the field of Scientometrics

This is an accepted manuscript of an article published by Springer in Scientometrics on 03/01/2021, available online at: https://doi.org/10.1007/s11192-020-03814-w The accepted version of the publication may differ from the final published version.Assessing the impact of scholarly books is a difficult research evaluation problem. Library Catalog Analysis facilitates the quantitative study, at different levels, of the impact and diffusion of academic books based on data about their availability in libraries. The WorldCat global catalog collates data on library holdings, offering a range of tools including the novel WorldCat Identities. This is based on author profiles and provides indicators relating to the availability of their books in library catalogs. Here, we investigate this new tool to identify its strengths and weaknesses based on a sample of Bibliometrics and Scientometrics authors. We review the problems that this entails and compare Library Catalog Analysis indicators with Google Scholar and Web of Science citations. The results show that WorldCat Identities can be a useful tool for book impact assessment but the value of its data is undermined by the provision of massive collections of ebooks to academic libraries

The insoluble problems of books: What does Altmetric.com have to offer?

Purpose: Analyze the capabilities, functionalities and appropriateness of Altmetric.com as a data source for the bibliometric analysis of books in comparison to PlumX. Methodology: We perform an exploratory analysis on the metrics the Altmetric Explorer for Institutions platform offers for books. We use two distinct datasets of books. On the one hand, we analyze the Book Collection included in Altmetric.com. On the other, we use Clarivate’s Master Book List, to analyze Altmetric.com’s capabilities to download and merge data with external databases. Finally, we compare our findings with those obtained in a previous study performed in PlumX. Findings: Altmetric.com combines and orderly tracks a set of data sources combined by DOI identifiers to retrieve metadata from books, being Google Books its main provider. It also retrieves information from commercial publishers and from some Open Access initiatives, including those led by university libraries such as Harvard Library. We find issues with linkages between records and mentions or ISBN discrepancies. Furthermore, we find that automatic bots affect greatly Wikipedia mentions to books. Our comparison with PlumX suggests that none of these tools provide a complete picture of the social attention generated by books and are rather complementary than comparable tools. Practical implications: This study targets different audiences which can benefit from our findings. First, bibliometricians and researchers who seek for alternative sources to develop bibliometric analyses of books, with a special focus on the Social Sciences and Humanities fields. Second, librarians and research managers who are the main clients to which these tools are directed. Third, Altmetric.com itself as well as other altmetric providers who might get a better understanding of the limitations users encounter and improve this promising tool. Originality/value: This is the first study to analyze Altmetric.com’s functionalities and capabilities for providing metric data for books and to compare results from this platform, with those obtained via Plum

Bibliometria e scienze del libro: internazionalizzazione e vitalità degli studi italiani

The vitality of a scientific field is usually attested by the fallout that research and publications have on the community of scholars who practice it and, therefore, on the development of the discipline itself and of its methodologies. The degree of “health” of a field of study, as well as its ability to get out of its niche of scholars and to see its validity recognised, is assessed by analysing whether and to what extent the works are read, commented on, cited by scholars from other geographical contexts and / or other scientific branches. The volume presents a research focused on the analysis of the level of internationalisation and vitality of Italian studies in the disciplines related to the study of books and documents, conducted through a series of parallel bibliometric investigations (query of citation databases, searches in Google scholar, application alternative metrics, library catalog analysis)