A Review of Theory and Practice in Scientometrics

Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments

An Assessment of Impact Metrics’ Potential as Research Indicators Based on Their Perception, Usage, and Dependencies from External Science Communication

The demand for practicable methods for quantitative assessments of scientific products’ relevance has risen considerably over the past decades. As a consequence, research and commercial providers of scholarly data developed a wide variety of impact indicators, ranging from citation-based to so-called altmetrics. This highly heterogeneous family of indicators is based on the principle of measuring interactions with scientific publications that are observable online, and covers for instance mentions of publications in social and journalistic media, in literature management software, or in policy documents. The various metrics' theoretical validity as impact indicators is debated constantly, as questions regarding what it is that different metrics measure or express in many facets remain unanswered. This thesis makes two central contributions towards answering these questions. Its first part systematically assesses the status quo of various metrics’ perception and usage by researchers. This assessment serves to determine the significance of metrics in academic daily routines, as well as to identify relevant perceived problems concerning their usage. The challenges identified this way are in later sections of the thesis opposed with concrete measures to be taken during the development of future research metrics and their infrastructure to effectively solve common criticisms regarding current metrics and their use. Proceeding from the first part’s user studies, this thesis’ second part examines the relationship between research metrics and external science communication. It this way addresses a wide research gap with considerable potential implications for metrics’ validity as indicators for quality - the question to which degree these metrics are merely the result of promotion, which respective research publications receive

Study on Scientific outputs of Scholars in the Field of Digital Libraries Using Altmetrics Indicators

The current study aims to calculate the relationship between Altmetric scores obtained from the observation and dissemination of digital library resources in the Dimensions database and the number of citations received in the Scopus database. Also, in another part of the research, the predictive power of the number of Scopus citations by Altmetric scores is examined. The present research is applied in terms of purpose and survey-descriptive in terms of type, which is done by the scientometric method and with an Altmetric approach. The statistical population of the study includes all articles in the field of digital libraries (24183 records) that are indexed in the Scopus citation database during 1960-2020. Dimensions database has been used to evaluate the Altmetric scores obtained from these articles on social networks. Due to the limited access to the required data in the Scopus database, 2000 highly cited articles in the field of digital libraries in this Scopus database were studied through the Dimensions database. The data collection tools are Scopus Citation Database and Dimensions Database. The required data is collected through the Scopus database. In this study, the studied indicators from the Dimensions database appear as the independent variable of the research. The dependent variables in this study are the number of citations to articles in the Scopus database. Correlation tests and multiple regression between the studied indices are used to examine the relationships between variables and perform statistical tests. The software used is Excel and SPSS version 23. The present study results show that the social networks Patent, Facebook, Wikipedia, and Twitter have the highest correlation with the number of citations in the Dimensions database. The social networks Blog, Google User, and Q&A do not significantly relate to the number of citations received in Dimensions. Patent social networks, Wikipedia, and Twitter have the highest correlation with the number of Scopus citations. In this case, the social networks of Blog, Google User, Pulse Source and Q&A do not significantly correlate with the number of citations received. Among the citation databases studied, Mendeley has the highest correlation between the numbers of citations. Other results indicate that the publication and viewing of documents on social networks cannot predict the number of citations in the Dimensions and Scopus databases.https://dorl.net/dor/20.1001.1.20088302.2022.20.4.10.

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

Theories of Informetrics and Scholarly Communication

Scientometrics have become an essential element in the practice and evaluation of science and research, including both the evaluation of individuals and national assessment exercises. Yet, researchers and practitioners in this field have lacked clear theories to guide their work. As early as 1981, then doctoral student Blaise Cronin published "The need for a theory of citing" —a call to arms for the fledgling scientometric community to produce foundational theories upon which the work of the field could be based. More than three decades later, the time has come to reach out the field again and ask how they have responded to this call. This book compiles the foundational theories that guide informetrics and scholarly communication research. It is a much needed compilation by leading scholars in the field that gathers together the theories that guide our understanding of authorship, citing, and impact

Theories of Informetrics and Scholarly Communication

Scientometrics have become an essential element in the practice and evaluation of science and research, including both the evaluation of individuals and national assessment exercises. Yet, researchers and practitioners in this field have lacked clear theories to guide their work. As early as 1981, then doctoral student Blaise Cronin published The need for a theory of citing - a call to arms for the fledgling scientometric community to produce foundational theories upon which the work of the field could be based. More than three decades later, the time has come to reach out the field again and ask how they have responded to this call. This book compiles the foundational theories that guide informetrics and scholarly communication research. It is a much needed compilation by leading scholars in the field that gathers together the theories that guide our understanding of authorship, citing, and impact

Quantity versus impact of software engineering papers: a quantitative study

According to the data from the Scopus publication database, as analyzed in several recent studies, more than 70,000 papers have been published in the area of Software Engineering (SE) since late 1960’s. According to our recent work, 43% of those papers have received no citations at all. Since citations are the most commonly used metric for measuring research (academic) impact, these figures raise questions (doubts) about the (non-existing) impact of such a large set of papers. It is a reality that typical academic reward systems encourage researchers to publish more papers and do not place a major emphasis on research impact. To shed light on the issue of volume (quantity) versus citation-based impact of SE research papers, we conduct and report in this paper a quantitative bibliometrics assessment in four aspects: (1) quantity versus impact of different paper types (e.g., conference versus journal papers), (2) ratios of uncited (non-impactful) papers, (3) quantity versus impact of papers originating from different countries, and (4) quantity versus impact of papers by each of the top-10 authors (in terms of number of papers). To achieve the above objective, we conducted a quantitative exploratory bibliometrics assessment, comprised of four research questions, to assess quantity versus impact of SE papers with respect to the aspects discussed above. We extracted the data through a systematic, automated and repeatable process from the Scopus paper database, which we also used in two previous papers. Our results show that the distribution of SE publications has a major inequality in terms of impact overall, and also when categorized in terms of the above four aspects. The situation in the SE literature is similar to the other areas of science as studied by previous bibliometrics studies. Also, among our results is the fact that journal articles and conference papers have been cited 12.6 and 3.6 times on average, confirming the expectation that journal articles have more impact, in general, than conference papers. Also, papers originated from English-speaking countries have in general more visibility and impact (and consequently citations) when compared to papers originated from non-English-speaking countries. Our results have implications for improvement of academic reward systems, which nowadays mainly encourage researchers to publish more papers and usually neglect research impact. Also, our results can help researchers in non-English-speaking countries to consider improvements to increase their research impact of their upcoming papers.Vahid Garousi was partially supported by several internal grants provided by the Hacettepe University and the Scientific and Technological Research Council of Turkey (TUBITAK). Joao M. Fernandes was supported by FCT - Fundacao para a Ciencia e Tecnologia within the Project Scope UID/CEC/00319/2013

Theories of Informetrics and Scholarly Communication

Scientometrics have become an essential element in the practice and evaluation of science and research, including both the evaluation of individuals and national assessment exercises. Yet, researchers and practitioners in this field have lacked clear theories to guide their work. As early as 1981, then doctoral student Blaise Cronin published "The need for a theory of citing" —a call to arms for the fledgling scientometric community to produce foundational theories upon which the work of the field could be based. More than three decades later, the time has come to reach out the field again and ask how they have responded to this call. This book compiles the foundational theories that guide informetrics and scholarly communication research. It is a much needed compilation by leading scholars in the field that gathers together the theories that guide our understanding of authorship, citing, and impact

Study on open science: The general state of the play in Open Science principles and practices at European life sciences institutes

Nowadays, open science is a hot topic on all levels and also is one of the priorities of the European Research Area. Components that are commonly associated with open science are open access, open data, open methodology, open source, open peer review, open science policies and citizen science. Open science may a great potential to connect and influence the practices of researchers, funding institutions and the public. In this paper, we evaluate the level of openness based on public surveys at four European life sciences institute