A review of the characteristics of 108 author-level bibliometric indicators

An increasing demand for bibliometric assessment of individuals has led to a growth of new bibliometric indicators as well as new variants or combinations of established ones. The aim of this review is to contribute with objective facts about the usefulness of bibliometric indicators of the effects of publication activity at the individual level. This paper reviews 108 indicators that can potentially be used to measure performance on the individual author level, and examines the complexity of their calculations in relation to what they are supposed to reflect and ease of end-user application.Comment: to be published in Scientometrics, 201

The pros and cons of the use of altmetrics in research assessment

© 2020 The Authors. Published by Levi Library Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: http://doi.org/10.29024/sar.10Many indicators derived from the web have been proposed to supplement citation-based indicators in support of research assessments. These indicators, often called altmetrics, are available commercially from Altmetric.com and Elsevier’s Plum Analytics or can be collected directly. These organisations can also deliver altmetrics to support institutional selfevaluations. The potential advantages of altmetrics for research evaluation are that they may reflect important non-academic impacts and may appear before citations when an article is published, thus providing earlier impact evidence. Their disadvantages often include susceptibility to gaming, data sparsity, and difficulties translating the evidence into specific types of impact. Despite these limitations, altmetrics have been widely adopted by publishers, apparently to give authors, editors and readers insights into the level of interest in recently published articles. This article summarises evidence for and against extending the adoption of altmetrics to research evaluations. It argues that whilst systematicallygathered altmetrics are inappropriate for important formal research evaluations, they can play a role in some other contexts. They can be informative when evaluating research units that rarely produce journal articles, when seeking to identify evidence of novel types of impact during institutional or other self-evaluations, and when selected by individuals or groups to support narrative-based non-academic claims. In addition, Mendeley reader counts are uniquely valuable as early (mainly) scholarly impact indicators to replace citations when gaming is not possible and early impact evidence is needed. Organisations using alternative indicators need recruit or develop in-house expertise to ensure that they are not misused, however

On the Predictability of Talk Attendance at Academic Conferences

This paper focuses on the prediction of real-world talk attendances at academic conferences with respect to different influence factors. We study the predictability of talk attendances using real-world tracked face-to-face contacts. Furthermore, we investigate and discuss the predictive power of user interests extracted from the users' previous publications. We apply Hybrid Rooted PageRank, a state-of-the-art unsupervised machine learning method that combines information from different sources. Using this method, we analyze and discuss the predictive power of contact and interest networks separately and in combination. We find that contact and similarity networks achieve comparable results, and that combinations of different networks can only to a limited extend help to improve the prediction quality. For our experiments, we analyze the predictability of talk attendance at the ACM Conference on Hypertext and Hypermedia 2011 collected using the conference management system Conferator

Determinants of the international influence of a R&D organisation: a bibliometric approach

Traditionally, studies on the influence and impact of knowledge-producing organisations have been addressed by means of strict economic analysis, stressing their economic impact to a local, regional or national extent. In the present study, an alternative methodology is put forward in order to evaluate the international scientific impact and influence of a knowledge-producing and -diffusing institution. We introduce a new methodology, based on scientometric and bibliometric tools, which complement traditional assessments by considering the influence of a R&D institution when looking at the scientific production undertaken and the recognition of its relevance by its international peer community. Focusing on the most prolific scientific areas of INESC Porto, and resorting to published scientific work recorded in the Science Citation Index (SCI), we show that INESC Porto has enlarged its international scientific network. The logit estimations demonstrate that the wide geographical influence of INESC Porto scientific research is a result not of its international positioning in terms of co-authorships, but rather a result of the quality of its scientific output.Impact and influence assessment methods; R&D Institutions; Bibliometrics, Scientometrics; knowledge network; INESC Porto

Determinants of the international influence of a R&D organisation: a bibliometric approach

Traditionally, studies on the influence and impact of knowledge-producing organisations have been addressed by means of strict economic analysis, stressing their economic impact to a local, regional or national extent. In the present study, an alternative methodology is put forward in order to evaluate the international scientific impact and influence of a knowledge-producing and -diffusing institution. We introduce a new methodology, based on scientometric and bibliometric tools, which complement traditional assessments by considering the influence of a R&D institution when looking at the scientific production undertaken and the recognition of its relevance by its international peer community. Focusing on the most prolific scientific areas of INESC Porto, and resorting to published scientific work recorded in the Science Citation Index (SCI), we show that INESC Porto has enlarged its international scientific network. The logit estimations demonstrate that the wide geographical influence of INESC Porto scientific research is a result not of its international positioning in terms of co-authorships, but rather a result of the quality of its scientific output.Impact and influence assessment methods; R&D Institutions; Bibliometrics, Scientometrics; knowledge network; INESC Porto