Reviewing, indicating, and counting books for modern research evaluation systems

In this chapter, we focus on the specialists who have helped to improve the conditions for book assessments in research evaluation exercises, with empirically based data and insights supporting their greater integration. Our review highlights the research carried out by four types of expert communities, referred to as the monitors, the subject classifiers, the indexers and the indicator constructionists. Many challenges lie ahead for scholars affiliated with these communities, particularly the latter three. By acknowledging their unique, yet interrelated roles, we show where the greatest potential is for both quantitative and qualitative indicator advancements in book-inclusive evaluation systems.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springer Some corrections made in subsection 'Publisher prestige or quality

A Relative Impact Ranking of Political Studies In Ireland

Against a background of the Irish government’s concerns with Key Performance Indicators (KPIs) and the British government’s wishes for a more quantitative Research Assessment Exercise (RAE), our study conducts a relative impact assessment of the study of politics, government, political science, and international relations in Ireland. Impact is measured as citations from the publications of permanent staff in eight Irish politics departments, based on data compiled in April 2008 from two leading academic indexes – ISI’s Web of Science and Scopus – as well as the now popular Google Scholar. We discuss some of the criticisms that naturally arise in a study of this nature. Then, following similar exercises in other disciplines (e.g. economics), we use the impact measures to compare and rank individual scholars as well as departments. We also explore the extent to which the choice of different indexes, and different measures, influences the results that we obtain. While there are differences, in particular between indexes based purely on articles and those that access books and other material, the results from the different indexes are strongly correlated.

Theory and Practice of Data Citation

Citations are the cornerstone of knowledge propagation and the primary means of assessing the quality of research, as well as directing investments in science. Science is increasingly becoming "data-intensive", where large volumes of data are collected and analyzed to discover complex patterns through simulations and experiments, and most scientific reference works have been replaced by online curated datasets. Yet, given a dataset, there is no quantitative, consistent and established way of knowing how it has been used over time, who contributed to its curation, what results have been yielded or what value it has. The development of a theory and practice of data citation is fundamental for considering data as first-class research objects with the same relevance and centrality of traditional scientific products. Many works in recent years have discussed data citation from different viewpoints: illustrating why data citation is needed, defining the principles and outlining recommendations for data citation systems, and providing computational methods for addressing specific issues of data citation. The current panorama is many-faceted and an overall view that brings together diverse aspects of this topic is still missing. Therefore, this paper aims to describe the lay of the land for data citation, both from the theoretical (the why and what) and the practical (the how) angle.Comment: 24 pages, 2 tables, pre-print accepted in Journal of the Association for Information Science and Technology (JASIST), 201

Creating a Comprehensive Review of the Literature

Research normally begins by doing a “review of the literature” to see what has already been written and to determine “gaps” in the literature for further research. This handout reviews ways to search for books, scholarly articles, dissertations, and grey literature on any topic using our library subscription resources

Scholarly Metrics Baseline: A Survey of Faculty Knowledge, Use, and Opinion About Scholarly Metrics

This article presents the results of a faculty survey conducted at the University of Vermont during academic year 2014-2015. The survey asked faculty about: familiarity with scholarly metrics, metric seeking habits, help seeking habits, and the role of metrics in their department’s tenure and promotion process. The survey also gathered faculty opinions on how well scholarly metrics reflect the importance of scholarly work and how faculty feel about administrators gathering institutional scholarly metric information. Results point to the necessity of understanding the campus landscape of faculty knowledge, opinion, importance, and use of scholarly metrics before engaging faculty in further discussions about quantifying the impact of their scholarly work