1,542 research outputs found
An Integrated Impact Indicator (I3): A New Definition of "Impact" with Policy Relevance
Allocation of research funding, as well as promotion and tenure decisions,
are increasingly made using indicators and impact factors drawn from citations
to published work. A debate among scientometricians about proper normalization
of citation counts has resolved with the creation of an Integrated Impact
Indicator (I3) that solves a number of problems found among previously used
indicators. The I3 applies non-parametric statistics using percentiles,
allowing highly-cited papers to be weighted more than less-cited ones. It
further allows unbundling of venues (i.e., journals or databases) at the
article level. Measures at the article level can be re-aggregated in terms of
units of evaluation. At the venue level, the I3 creates a properly weighted
alternative to the journal impact factor. I3 has the added advantage of
enabling and quantifying classifications such as the six percentile rank
classes used by the National Science Board's Science & Engineering Indicators.Comment: Research Evaluation (in press
Accounting for the Uncertainty in the Evaluation of Percentile Ranks
In a recent paper entitled "Inconsistencies of Recently Proposed Citation
Impact Indicators and how to Avoid Them," Schreiber (2012, at arXiv:1202.3861)
proposed (i) a method to assess tied ranks consistently and (ii) fractional
attribution to percentile ranks in the case of relatively small samples (e.g.,
for n < 100). Schreiber's solution to the problem of how to handle tied ranks
is convincing, in my opinion (cf. Pudovkin & Garfield, 2009). The fractional
attribution, however, is computationally intensive and cannot be done manually
for even moderately large batches of documents. Schreiber attributed scores
fractionally to the six percentile rank classes used in the Science and
Engineering Indicators of the U.S. National Science Board, and thus missed, in
my opinion, the point that fractional attribution at the level of hundred
percentiles-or equivalently quantiles as the continuous random variable-is only
a linear, and therefore much less complex problem. Given the quantile-values,
the non-linear attribution to the six classes or any other evaluation scheme is
then a question of aggregation. A new routine based on these principles
(including Schreiber's solution for tied ranks) is made available as software
for the assessment of documents retrieved from the Web of Science (at
http://www.leydesdorff.net/software/i3).Comment: Journal of the American Society for Information Science and
Technology (in press
On the calculation of percentile-based bibliometric indicators
A percentile-based bibliometric indicator is an indicator that values
publications based on their position within the citation distribution of their
field. The most straightforward percentile-based indicator is the proportion of
frequently cited publications, for instance the proportion of publications that
belong to the top 10% most frequently cited of their field. Recently, more
complex percentile-based indicators were proposed. A difficulty in the
calculation of percentile-based indicators is caused by the discrete nature of
citation distributions combined with the presence of many publications with the
same number of citations. We introduce an approach to calculating
percentile-based indicators that deals with this difficulty in a more
satisfactory way than earlier approaches suggested in the literature. We show
in a formal mathematical framework that our approach leads to indicators that
do not suffer from biases in favor of or against particular fields of science
- …