A Rejoinder on Energy versus Impact Indicators

Citation distributions are so skewed that using the mean or any other central tendency measure is ill-advised. Unlike G. Prathap's scalar measures (Energy, Exergy, and Entropy or EEE), the Integrated Impact Indicator (I3) is based on non-parametric statistics using the (100) percentiles of the distribution. Observed values can be tested against expected ones; impact can be qualified at the article level and then aggregated.Comment: Scientometrics, in pres

Scopus's Source Normalized Impact per Paper (SNIP) versus a Journal Impact Factor based on Fractional Counting of Citations

Impact factors (and similar measures such as the Scimago Journal Rankings) suffer from two problems: (i) citation behavior varies among fields of science and therefore leads to systematic differences, and (ii) there are no statistics to inform us whether differences are significant. The recently introduced SNIP indicator of Scopus tries to remedy the first of these two problems, but a number of normalization decisions are involved which makes it impossible to test for significance. Using fractional counting of citations-based on the assumption that impact is proportionate to the number of references in the citing documents-citations can be contextualized at the paper level and aggregated impacts of sets can be tested for their significance. It can be shown that the weighted impact of Annals of Mathematics (0.247) is not so much lower than that of Molecular Cell (0.386) despite a five-fold difference between their impact factors (2.793 and 13.156, respectively)

Can Scientific Journals be Classified in terms of Aggregated Journal-Journal Citation Relations using the Journal Citation Reports?

This is a preprint of an article published in Journal of the American Society for Information Science and Technology 57 (5):601-613, 2006. The aggregated citation relations among journals included in the Science Citation Index provide us with a huge matrix which can be analyzed in various ways. Using principal component analysis or factor analysis, the factor scores can be used as indicators of the position of the cited journals in the citing dimensions of the database. Unrotated factor scores are exact, and the extraction of principal components can be made stepwise since the principal components are independent. Rotation may be needed for the designation, but in the rotated solution a model is assumed. This assumption can be legitimated on pragmatic or theoretical grounds. Since the resulting outcomes remain sensitive to the assumptions in the model, an unambiguous classification is no longer possible in this case. However, the factor-analytic solutions allow us to test classifications against the structures contained in the database. This will be demonstrated for the delineation of a set of biochemistry journals

An Integrated Impact Indicator (I3): A New Definition of "Impact" with Policy Relevance

Allocation of research funding, as well as promotion and tenure decisions, are increasingly made using indicators and impact factors drawn from citations to published work. A debate among scientometricians about proper normalization of citation counts has resolved with the creation of an Integrated Impact Indicator (I3) that solves a number of problems found among previously used indicators. The I3 applies non-parametric statistics using percentiles, allowing highly-cited papers to be weighted more than less-cited ones. It further allows unbundling of venues (i.e., journals or databases) at the article level. Measures at the article level can be re-aggregated in terms of units of evaluation. At the venue level, the I3 creates a properly weighted alternative to the journal impact factor. I3 has the added advantage of enabling and quantifying classifications such as the six percentile rank classes used by the National Science Board's Science & Engineering Indicators.Comment: Research Evaluation (in press

Remaining problems with the "New Crown Indicator" (MNCS) of the CWTS

In their article, entitled "Towards a new crown indicator: some theoretical considerations," Waltman et al. (2010; at arXiv:1003.2167) show that the "old crown indicator" of CWTS in Leiden was mathematically inconsistent and that one should move to the normalization as applied in the "new crown indicator." Although we now agree about the statistical normalization, the "new crown indicator" inherits the scientometric problems of the "old" one in treating subject categories of journals as a standard for normalizing differences in citation behavior among fields of science. We further note that the "mean" is not a proper statistics for measuring differences among skewed distributions. Without changing the acronym of "MNCS," one could define the "Median Normalized Citation Score." This would relate the new crown indicator directly to the percentile approach that is, for example, used in the Science and Engineering Indicators of US National Science Board (2010). The median is by definition equal to the 50th percentile. The indicator can thus easily be extended with the 1% (= 99th percentile) most highly-cited papers (Bornmann et al., in press). The seeming disadvantage of having to use non-parametric statistics is more than compensated by possible gains in the precision

Dynamic Animations of Journal Maps: Indicators of Structural Changes and Interdisciplinary Developments

The dynamic analysis of structural change in the organization of the sciences requires methodologically the integration of multivariate and time-series analysis. Structural change--e.g., interdisciplinary development--is often an objective of government interventions. Recent developments in multi-dimensional scaling (MDS) enable us to distinguish the stress originating in each time-slice from the stress originating from the sequencing of time-slices, and thus to locally optimize the trade-offs between these two sources of variance in the animation. Furthermore, visualization programs like Pajek and Visone allow us to show not only the positions of the nodes, but also their relational attributes like betweenness centrality. Betweenness centrality in the vector space can be considered as an indicator of interdisciplinarity. Using this indicator, the dynamics of the citation impact environments of the journals Cognitive Science, Social Networks, and Nanotechnology are animated and assessed in terms of interdisciplinarity among the disciplines involved