A critical cluster analysis of 44 indicators of author-level performance

This paper explores the relationship between author-level bibliometric indicators and the researchers the "measure", exemplified across five academic seniorities and four disciplines. Using cluster methodology, the disciplinary and seniority appropriateness of author-level indicators is examined. Publication and citation data for 741 researchers across Astronomy, Environmental Science, Philosophy and Public Health was collected in Web of Science (WoS). Forty-four indicators of individual performance were computed using the data. A two-step cluster analysis using IBM SPSS version 22 was performed, followed by a risk analysis and ordinal logistic regression to explore cluster membership. Indicator scores were contextualized using the individual researcher's curriculum vitae. Four different clusters based on indicator scores ranked researchers as low, middle, high and extremely high performers. The results show that different indicators were appropriate in demarcating ranked performance in different disciplines. In Astronomy the h2 indicator, sum pp top prop in Environmental Science, Q2 in Philosophy and e-index in Public Health. The regression and odds analysis showed individual level indicator scores were primarily dependent on the number of years since the researcher's first publication registered in WoS, number of publications and number of citations. Seniority classification was secondary therefore no seniority appropriate indicators were confidently identified. Cluster methodology proved useful in identifying disciplinary appropriate indicators providing the preliminary data preparation was thorough but needed to be supplemented by other analyses to validate the results. A general disconnection between the performance of the researcher on their curriculum vitae and the performance of the researcher based on bibliometric indicators was observed.Comment: 28 pages, 7 tables, 2 figures, 2 appendice

Predicting the long-term citation impact of recent publications

A fundamental problem in citation analysis is the prediction of the long-term citation impact of recent publications. We propose a model to predict a probability distribution for the future number of citations of a publication. Two predictors are used: The impact factor of the journal in which a publication has appeared and the number of citations a publication has received one year after its appearance. The proposed model is based on quantile regression. We employ the model to predict the future number of citations of a large set of publications in the field of physics. Our analysis shows that both predictors (i.e., impact factor and early citations) contribute to the accurate prediction of long-term citation impact. We also analytically study the behavior of the quantile regression coefficients for high quantiles of the distribution of citations. This is done by linking the quantile regression approach to a quantile estimation technique from extreme value theory. Our work provides insight into the influence of the impact factor and early citations on the long-term citation impact of a publication, and it takes a step toward a methodology that can be used to assess research institutions based on their most recently published work.Comment: 17 pages, 17 figure

Factors predicting the scientific wealth of nations

It has been repeatedly demonstrated that economic affluence is one of the main predictors of the scientific wealth of nations. Yet, the link is not as straightforward as is often presented. First, only a limited set of relatively affluent countries is usually studied. Second, there are differences between equally rich countries in their scientific success. The main aim of the present study is to find out which factors can enhance or suppress the effect of the economic wealth of countries on their scientific success, as measured by the High Quality Science Index (HQSI). The HQSI is a composite indicator of scientific wealth, which in equal parts considers the mean citation rate per paper and the percentage of papers that have reached the top 1% of citations in the Essential Science Indicators (ESI; Clarivate Analytics) database during the 11-year period from 2008 to 2018. Our results show that a high position in the ranking of countries on the HQSI can be achieved not only by increasing the number of high-quality papers but also by reducing the number of papers that are able to pass ESI thresholds but are of lower quality. The HQSI was positively and significantly correlated with the countries’ economic indicators (as measured by gross national income and Research and Development expenditure as a percentage from GDP), but these correlations became insignificant when other societal factors were controlled for. Overall, our findings indicate that it is small and well-governed countries with a long-standing democratic past that seem to be more efficient in translating economic wealth into high-quality science

A review of the characteristics of 108 author-level bibliometric indicators

An increasing demand for bibliometric assessment of individuals has led to a growth of new bibliometric indicators as well as new variants or combinations of established ones. The aim of this review is to contribute with objective facts about the usefulness of bibliometric indicators of the effects of publication activity at the individual level. This paper reviews 108 indicators that can potentially be used to measure performance on the individual author level, and examines the complexity of their calculations in relation to what they are supposed to reflect and ease of end-user application.Comment: to be published in Scientometrics, 201

Climate Change Research in View of Bibliometrics

This bibliometric study of a large publication set dealing with research on climate change aims at mapping the relevant literature from a bibliometric perspective and presents a multitude of quantitative data: (1) The growth of the overall publication output as well as (2) of some major subfields, (3) the contributing journals and countries as well as their citation impact, and (4) a title word analysis aiming to illustrate the time evolution and relative importance of specific research topics. The study is based on 222,060 papers published between 1980 and 2014. The total number of papers shows a strong increase with a doubling every 5-6 years. Continental biomass related research is the major subfield, closely followed by climate modeling. Research dealing with adaptation, mitigation, risks, and vulnerability of global warming is comparatively small, but their share of papers increased exponentially since 2005. Research on vulnerability and on adaptation published the largest proportion of very important papers. Research on climate change is quantitatively dominated by the USA, followed by the UK, Germany, and Canada. The citation-based indicators exhibit consistently that the UK has produced the largest proportion of high impact papers compared to the other countries (having published more than 10,000 papers). The title word analysis shows that the term climate change comes forward with time. Furthermore, the term impact arises and points to research dealing with the various effects of climate change. Finally, the term model and related terms prominently appear independent of time, indicating the high relevance of climate modeling.Comment: 40 pages, 6 figures, and 4 table

A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments