The distorted mirror of Wikipedia: a quantitative analysis of Wikipedia coverage of academics

Activity of modern scholarship creates online footprints galore. Along with traditional metrics of research quality, such as citation counts, online images of researchers and institutions increasingly matter in evaluating academic impact, decisions about grant allocation, and promotion. We examined 400 biographical Wikipedia articles on academics from four scientific fields to test if being featured in the world's largest online encyclopedia is correlated with higher academic notability (assessed through citation counts). We found no statistically significant correlation between Wikipedia articles metrics (length, number of edits, number of incoming links from other articles, etc.) and academic notability of the mentioned researchers. We also did not find any evidence that the scientists with better WP representation are necessarily more prominent in their fields. In addition, we inspected the Wikipedia coverage of notable scientists sampled from Thomson Reuters list of "highly cited researchers". In each of the examined fields, Wikipedia failed in covering notable scholars properly. Both findings imply that Wikipedia might be producing an inaccurate image of academics on the front end of science. By shedding light on how public perception of academic progress is formed, this study alerts that a subjective element might have been introduced into the hitherto structured system of academic evaluation.Comment: To appear in EPJ Data Science. To have the Additional Files and Datasets e-mail the corresponding autho

Applied Evaluative Informetrics: Part 1

This manuscript is a preprint version of Part 1 (General Introduction and Synopsis) of the book Applied Evaluative Informetrics, to be published by Springer in the summer of 2017. This book presents an introduction to the field of applied evaluative informetrics, and is written for interested scholars and students from all domains of science and scholarship. It sketches the field's history, recent achievements, and its potential and limits. It explains the notion of multi-dimensional research performance, and discusses the pros and cons of 28 citation-, patent-, reputation- and altmetrics-based indicators. In addition, it presents quantitative research assessment as an evaluation science, and focuses on the role of extra-informetric factors in the development of indicators, and on the policy context of their application. It also discusses the way forward, both for users and for developers of informetric tools.Comment: The posted version is a preprint (author copy) of Part 1 (General Introduction and Synopsis) of a book entitled Applied Evaluative Bibliometrics, to be published by Springer in the summer of 201

A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

Do Mendeley reader counts reflect the scholarly impact of conference papers? An investigation of computer science and engineering

This is an accepted manuscript of an article published by Springer in Scientometrics on 13/04/2017, available online: https://doi.org/10.1007/s11192-017-2367-1 The accepted version of the publication may differ from the final published version.Counts of Mendeley readers may give useful evidence about the impact of published re-search. Although previous studies have found significant positive correlations between counts of Mendeley readers and citation counts for journal articles, it is not known if this is equally true for conference papers. To fill this gap, Mendeley readership data and Scopus citation counts were extracted for both journal articles and conference papers published in 2011 in four fields for which conferences are important: Computer Science Applications; Computer Software; Building & Construction Engineering; and Industrial & Manufacturing Engineer-ing. Mendeley readership counts correlated moderately with citation counts for both journal articles and conference papers in Computer Science Applications and Computer Software. The correlations were much lower between Mendeley readers and citation counts for confer-ence papers than for journal articles in Building & Construction Engineering and Industrial & Manufacturing Engineering. Hence, there seem to be disciplinary differences in the useful-ness of Mendeley readership counts as impact indicators for conference papers, even between fields for which conferences are important

Information and Scientific Impact of Advanced Therapies in the Age of Mass Media: Altmetrics-Based Analysis of Tissue Engineering

This study was supported by CTS-115 (Tissue Engineering Research Group, University of Granada) from Junta de Andalucia, Spain, the Spanish State Research Agency through the project PID2019-105381GA-I00/AEI/10.13039/501100011033 (iScience) , a postdoctoral grant (RH-0145-2020) from the Andalusia Health System, and the European Union Fondo Europeo de Desarrollo Regional para la Inversion Territorial Integrada Grant for Cadiz Province (PI-0032-2017) . The authors thank Altmetric LLP (London, UK) for granting access to Altmetric Explorer for research purposes.Background: Tissue engineering (TE) constitutes a multidisciplinary field aiming to construct artificial tissues to regenerate end-stage organs. Its development has taken place since the last decade of the 20th century, entailing a clinical revolution. TE research groups have worked and shared relevant information in the mass media era. Thus, it would be interesting to study the online dimension of TE research and to compare it with traditional measures of scientific impact. Objective: The objective of this study was to evaluate the online dimension of TE documents from 2012 to 2018 using metadata obtained from the Web of Science (WoS) and Altmetric and to develop a prediction equation for the impact of TE documents from altmetric scores. Methods: We analyzed 10,112 TE documents through descriptive and statistical methods. First, the TE temporal evolution was exposed for WoS and 15 online platforms (news, blogs, policy, Twitter, patents, peer review, Weibo, Facebook, Wikipedia, Google, Reddit, F1000, Q&A, video, and Mendeley Readers). The 10 most cited TE original articles were ranked according to the normalized WoS citations and the normalized Altmetric Attention Score. Second, to better comprehend the TE online framework, correlation and factor analyses were performed based on the suitable results previously obtained for the Bartlett sphericity and Kaiser–Meyer–Olkin tests. Finally, the linear regression model was applied to elucidate the relation between academics and online media and to construct a prediction equation for TE from altmetrics data. Results: TE dynamic shows an upward trend in WoS citations, Twitter, Mendeley Readers, and Altmetric Scores. However, WoS and Altmetric rankings for the most cited documents clearly differ. When compared, the best correlation results were obtained for Mendeley Readers and WoS (ρ=0.71). In addition, the factor analysis identified 6 factors that could explain the previously observed differences between academic institutions and the online platforms evaluated. At this point, the mathematical model constructed is able to predict and explain more than 40% of TE WoS citations from Altmetric scores. Conclusions: Scientific information related to the construction of bioartificial tissues increasingly reaches society through different online media. Because the focus of TE research importantly differs when the academic institutions and online platforms are compared, basic and clinical research groups, academic institutions, and health politicians should make a coordinated effort toward the design and implementation of adequate strategies for information diffusion and population health education.CTS-115 (Tissue Engineering Research Group, University of Granada) from Junta de Andalucia, SpainSpanish Government PID2019-105381GA-I00/AEI/10.13039/501100011033Andalusia Health System RH-0145-2020European Union Fondo Europeo de Desarrollo Regional para la Inversion Territorial Integrada Grant for Cadiz Province PI-0032-201

Online Visibility, Social Networks and Glamorous Scientific Publications

In a context of transformation of the higher education institutions’ mission, there is a growing need for the academy to respond to the needs placed both at the political and social levels, which has implications for the scholar’s expected activity. The purpose of this paper is to analyse the growing importance, besides publishing and being cited, of having visibility in the digital world. We conclude that this new dimension, which is being added to the success and legitimacy of the scholar and his/her institution, will have probable direct consequences both on the form and on the contents of future publications. The willingness of scholars to produce publications worthy of social visibility may foster a growing number of publications that are attractive, perhaps less complex and more accessible to the “uninitiated”, what we call glamorous publications

Altmetrics as an Answer to the Need for Democratization of Research and Its Evaluation

In the evaluation of research, the same unequal structure present in the production of research is reproduced. Despite a few very productive researchers (in terms of papers and citations received), there are also few researchers who are involved in the research evaluation process (in terms of being editorial board members of journals or reviewers). To produce a high number of papers and receive many citations and to be involved in the evaluation of research papers, you need to be in the minority of giants who have high productivity and more scientific success. As editorial board members and reviewers, we often find the same minority of giants. In this paper, we apply an economic approach to interpret recent trends in research evaluation and derive a new interpretation of Altmetrics as a response to the need for democratization of research and its evaluation. In this context, the majority of pygmies can participate in evaluation with Altmetrics, whose use is more democratic, that is, much wider and open to all