Impact Factor: outdated artefact or stepping-stone to journal certification?

A review of Garfield's journal impact factor and its specific implementation as the Thomson Reuters Impact Factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.Comment: 25 pages, 12 figures, 6 table

Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus

Journal classification systems play an important role in bibliometric analyses. The two most important bibliographic databases, Web of Science and Scopus, each provide a journal classification system. However, no study has systematically investigated the accuracy of these classification systems. To examine and compare the accuracy of journal classification systems, we define two criteria on the basis of direct citation relations between journals and categories. We use Criterion I to select journals that have weak connections with their assigned categories, and we use Criterion II to identify journals that are not assigned to categories with which they have strong connections. If a journal satisfies either of the two criteria, we conclude that its assignment to categories may be questionable. Accordingly, we identify all journals with questionable classifications in Web of Science and Scopus. Furthermore, we perform a more in-depth analysis for the field of Library and Information Science to assess whether our proposed criteria are appropriate and whether they yield meaningful results. It turns out that according to our citation-based criteria Web of Science performs significantly better than Scopus in terms of the accuracy of its journal classification system

Between fetishism and survival : is the scientific article an academic commodity?

This article discusses the possible meanings of the intense prevailing concern in academic circles over the notion of research productivity, as reflected in an excess number of articles published in various scientific journals. The numerical accounting of articles published by researchers in scientific journals with renowned academic status serves to legitimize academics in their fields of work, in various ways. In this sense, we suggest that scientific articles take on aspects of merchandise-as-fetish, according to Marx's theory of use-value and exchange-value and Benjamin's exposure value. Meanwhile, the biological notions of selection and evolution are used as metaphorical elements in "bibliographic Darwinism". There are references as to the possibility many of the prevailing bibliometric concerns serve as instruments for econometric analysis, especially to orient and enhance cost-effectiveness analysis in research investments of various orders and types, from the point of view of their economic return

A bibliometric analysis of research output, citation analysis, author productivity, collaborations, and institutions of repute in smartphone Health Applications (spHealth Apps).

Due to rapid smartphone health application growth and usage, we analyzed literature published in the field of spHealth Apps. SciVerse Scopus was used as the database of choice for this study. Research productivity, collaborations, citation analysis, authors and institutions were presented using well established bibliometric indicators. During the study period (2000-20), 4546 documents were published in total. The average count of documents per year was 227. English was the language predominantly used in the retrieved documents (97%). The h- index of the retrieved documents was 137. Author submission of keywords used in documents pertaining to sp- Health Apps included human, randomized controlled trials, telemedicine, health care delivery, health promotion, physical activity among others. During the study period, Relative Growth Rate (RGR) and Doubling Time (DT) of retrieved literature fluctuated. An analysis of authorship and collaboration based on published data revealed 4244 multi-authored documents. The mean Collaboration Index (CI) was 5.8 authors per article. The country with the highest productivity was the United States of America with Harvard Medical School as the most prolific academic institution. Jmir Mhealth And Uhealth was the most productive journal in the field of spHealth Apps. Top cited articles in the field of spHealth Apps included the use of smartphone applications in phone sensing, point-of-care testing, health behavior promotion & modeling, mental health, contact tracing etc. spHealth Apps is a growing field with increasing impact in people\u27s day-to-day lifestyles. Our bibliometric indicators of research output in spHealth Apps mirror this increasing impact

A Bibliometric Analysis of an International Research Ethics Trainee Program

We used bibliometric analysis to evaluate the citations associated with publications by trainees in the Fogarty International Center’s International Research Ethics Education and Curriculum Development program. Papers published between 2004 and 2008 were identified for analysis. The outcome measures were total citations, h-index, and i-10. A total of 328 manuscripts were identified, with a yearly average of 66 publications and 363 citations. The median number of citations per paper is 3 (IQR Q1–Q3:6). 12.6% (n = 53) of papers were cited over 10 times and the h-index is 22, indicating that 22 papers had been cited at least 22 times. The data indicate that trainees have been productive and contributed to the scholarly literature. Future studies to benchmark this performance with other bioethics education programs are required to make interpretation of citation analysis more meaningful

The effect of the rapid growth of covid-19 publications on citation indicators

A concern has been raised that "covidization" of research would cause an overemphasizes on covid-19 and pandemics at the expense of other research. The rapid growth of publications related to the Covid-19 pandemic renders a vast amount of citations from this literature. This growth may affect bibliometric indicators. In this paper I explored how the growth of covid-19 publications influences bibliometric indicators commonly used in university rankings, research evaluation and research allocation, namely the field normalized citation score and the journal impact factor. I found that the burst of publications in the early stage of the covid-19 pandemic affects field-normalized citation scores and will affect the journal impact factor. Publications unrelated to covid-19 are also heavily affected. I conclude that there is a considerable risk to draw misleading conclusions from citation indicators spanning over the beginning of the covid-19 pandemic, in particular when time series are used and when the biomedical literature is assessed

The skewness of science in 219 sub-fields and a number of aggregates

This paper studies evidence from Thomson Scientific about the citation process of 3.7 million articles published in the period 1998-2002 in 219 Web of Science categories, or sub-fields. Reference and citation distributions have very different characteristics across sub-fields. However, when analyzed with the Characteristic Scores and Scales technique, which is size and scale independent, the shape of these distributions appear extraordinarily similar. Reference distributions are mildly skewed, but citation distributions with a five-year citation window are highly skewed: the mean is twenty points above the median, while 9-10% of all articles in the upper tail account for about 44% of all citations. The aggregation of sub-fields into disciplines and fields according to several aggregation schemes preserve this feature of citation distributions. On the other hand, for 140 of the 219 sub-fields the existence of a power law cannot be rejected. However, contrary to what is generally believed, at the sub-field level the scaling parameter is above 3.5 most of the time, and power laws are relatively small: on average, they represent 2% of all articles and account for 13.5% of all citations. The results of the aggregation into disciplines and fields reveal that power law algebra is a subtle phenomenon.

The skewness of science in 219 sub-fields and a number of aggregates

This paper studies evidence from Thomson Scientific about the citation process of 3.7 million articles published in the period 1998-2002 in 219 Web of Science categories, or sub-fields. Reference and citation distributions have very different characteristics across sub-fields. However, when analyzed with the Characteristic Scores and Scales technique, which is replication and scale invariant, the shape of these distributions over three broad categories of articles appears strikingly similar. Reference distributions are mildly skewed, but citation distributions with a five-year citation window are highly skewed: the mean is twenty points above the median, while 9-10% of all articles in the upper tail account for about 44% of all citations. The aggregation of sub-fields into disciplines and fields according to several aggregation schemes preserve this feature of citation distributions. It should be noted that when we look into subsets of articles within the lower and upper tails of citation distributions the universality partially breaks down. On the other hand, for 140 of the 219 sub-fields the existence of a power law cannot be rejected. However, contrary to what is generally believed, at the sub-field level the scaling parameter is above 3.5 most of the time, and power laws are relatively small: on average, they represent 2% of all articles and account for 13.5% of all citations. The results of the aggregation into disciplines and fields reveal that power law algebra is a subtle phenomenon.