Detecting h-index manipulation through self-citation analysis

The h-index has received an enormous attention for being an indicator that measures the quality of researchers and organizations. We investigate to what degree authors can inflate their h-index through strategic self-citations with the help of a simulation. We extended Burrell’s publication model with a procedure for placing self-citations, following three different strategies: random self-citation, recent self-citations and h-manipulating self-citations. The results show that authors can considerably inflate their h-index through self-citations. We propose the q-index as an indicator for how strategically an author has placed self-citations, and which serves as a tool to detect possible manipulation of the h-index. The results also show that the best strategy for an high h-index is publishing papers that are highly cited by others. The productivity has also a positive effect on the h-index

Self-citations at the meso and individual levels: effects of different calculation methods

This paper focuses on the study of self-citations at the meso and micro (individual) levels, on the basis of an analysis of the production (1994–2004) of individual researchers working at the Spanish CSIC in the areas of Biology and Biomedicine and Material Sciences. Two different types of self-citations are described: author self-citations (citations received from the author him/herself) and co-author self-citations (citations received from the researchers’ co-authors but without his/her participation). Self-citations do not play a decisive role in the high citation scores of documents either at the individual or at the meso level, which are mainly due to external citations. At micro-level, the percentage of self-citations does not change by professional rank or age, but differences in the relative weight of author and co-author self-citations have been found. The percentage of co-author self-citations tends to decrease with age and professional rank while the percentage of author self-citations shows the opposite trend. Suppressing author self-citations from citation counts to prevent overblown self-citation practices may result in a higher reduction of citation numbers of old scientists and, particularly, of those in the highest categories. Author and co-author self-citations provide valuable information on the scientific communication process, but external citations are the most relevant for evaluative purposes. As a final recommendation, studies considering self-citations at the individual level should make clear whether author or total self-citations are used as these can affect researchers differently

A New Approach to Analyzing Patterns of Collaboration in Co-authorship Networks - Mesoscopic Analysis and Interpretation

This paper focuses on methods to study patterns of collaboration in co-authorship networks at the mesoscopic level. We combine qualitative methods (participant interviews) with quantitative methods (network analysis) and demonstrate the application and value of our approach in a case study comparing three research fields in chemistry. A mesoscopic level of analysis means that in addition to the basic analytic unit of the individual researcher as node in a co-author network, we base our analysis on the observed modular structure of co-author networks. We interpret the clustering of authors into groups as bibliometric footprints of the basic collective units of knowledge production in a research specialty. We find two types of coauthor-linking patterns between author clusters that we interpret as representing two different forms of cooperative behavior, transfer-type connections due to career migrations or one-off services rendered, and stronger, dedicated inter-group collaboration. Hence the generic coauthor network of a research specialty can be understood as the overlay of two distinct types of cooperative networks between groups of authors publishing in a research specialty. We show how our analytic approach exposes field specific differences in the social organization of research.Comment: An earlier version of the paper was presented at ISSI 2009, 14-17 July, Rio de Janeiro, Brazil. Revised version accepted on 2 April 2010 for publication in Scientometrics. Removed part on node-role connectivity profile analysis after finding error in calculation and deciding to postpone analysis

The influence of self-citation corrections on Egghe's g index

The g index was introduced by Leo Egghe as an improvement of Hirsch's index h for measuring the overall citation record of a set of articles. It better takes into account the highly skewed frequency distribution of citations than the h index. I propose to sharpen this g index by excluding the self-citations. I have worked out nine practical cases in physics and compare the h and g values with and without self-citations. As expected, the g index characterizes the data set better than the h index. The influence of the self-citations appears to be more significant for the g index than for the h index.Comment: 9 pages, 2 figures, submitted to Scientometric

The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals

In our chapter we address the statistical analysis of percentiles: How should the citation impact of institutions be compared? In educational and psychological testing, percentiles are already used widely as a standard to evaluate an individual's test scores - intelligence tests for example - by comparing them with the percentiles of a calibrated sample. Percentiles, or percentile rank classes, are also a very suitable method for bibliometrics to normalize citations of publications in terms of the subject category and the publication year and, unlike the mean-based indicators (the relative citation rates), percentiles are scarcely affected by skewed distributions of citations. The percentile of a certain publication provides information about the citation impact this publication has achieved in comparison to other similar publications in the same subject category and publication year. Analyses of percentiles, however, have not always been presented in the most effective and meaningful way. New APA guidelines (American Psychological Association, 2010) suggest a lesser emphasis on significance tests and a greater emphasis on the substantive and practical significance of findings. Drawing on work by Cumming (2012) we show how examinations of effect sizes (e.g. Cohen's d statistic) and confidence intervals can lead to a clear understanding of citation impact differences

Evolutionary Events in a Mathematical Sciences Research Collaboration Network

This study examines long-term trends and shifting behavior in the collaboration network of mathematics literature, using a subset of data from Mathematical Reviews spanning 1985-2009. Rather than modeling the network cumulatively, this study traces the evolution of the "here and now" using fixed-duration sliding windows. The analysis uses a suite of common network diagnostics, including the distributions of degrees, distances, and clustering, to track network structure. Several random models that call these diagnostics as parameters help tease them apart as factors from the values of others. Some behaviors are consistent over the entire interval, but most diagnostics indicate that the network's structural evolution is dominated by occasional dramatic shifts in otherwise steady trends. These behaviors are not distributed evenly across the network; stark differences in evolution can be observed between two major subnetworks, loosely thought of as "pure" and "applied", which approximately partition the aggregate. The paper characterizes two major events along the mathematics network trajectory and discusses possible explanatory factors.Comment: 30 pages, 14 figures, 1 table; supporting information: 5 pages, 5 figures; published in Scientometric

The fruits of collaboration in a multidisciplinary field

Collaboration between researchers and between research organizations is generally considered a desirable course of action, in particular by some funding bodies. However, collaboration within a multidisciplinary community, such as the Computer–Human Interaction (CHI) community, can be challenging. We performed a bibliometric analysis of the CHI conference proceedings to determine if papers that have authors from different organization or countries receive more citations than papers that are authored by members of the same organization. There was no significant difference between these three groups, indicating that there is no advantage for collaboration in terms of citation frequency. Furthermore, we tested if papers written by authors from different organizations or countries receive more best paper awards or at least award nominations. Papers from only one organization received significantly fewer nominations than collaborative papers

Proposals for evaluating the regularity of a scientist'sresearch output

Evaluating the career of individual scientists according to their scientific output is a common bibliometric problem. Two aspects are classically taken into account: overall productivity and overall diffusion/impact, which can be measured by a plethora of indicators that consider publications and/or citations separately or synthesise these two quantities into a single number (e.g. h-index). A secondary aspect, which is sometimes mentioned in the rules of competitive examinations for research position/promotion, is time regularity of one researcher's scientific output. Despite the fact that it is sometimes invoked, a clear definition of regularity is still lacking. We define it as the ability of generating an active and stable research output over time, in terms of both publications/ quantity and citations/diffusion. The goal of this paper is introducing three analysis tools to perform qualitative/quantitative evaluations on the regularity of one scientist's output in a simple and organic way. These tools are respectively (1) the PY/CY diagram, (2) the publication/citation Ferrers diagram and (3) a simplified procedure for comparing the research output of several scientists according to their publication and citation temporal distributions (Borda's ranking). Description of these tools is supported by several examples

Generalized h-index for Disclosing Latent Facts in Citation Networks

What is the value of a scientist and its impact upon the scientific thinking? How can we measure the prestige of a journal or of a conference? The evaluation of the scientific work of a scientist and the estimation of the quality of a journal or conference has long attracted significant interest, due to the benefits from obtaining an unbiased and fair criterion. Although it appears to be simple, defining a quality metric is not an easy task. To overcome the disadvantages of the present metrics used for ranking scientists and journals, J.E. Hirsch proposed a pioneering metric, the now famous h-index. In this article, we demonstrate several inefficiencies of this index and develop a pair of generalizations and effective variants of it to deal with scientist ranking and with publication forum ranking. The new citation indices are able to disclose trendsetters in scientific research, as well as researchers that constantly shape their field with their influential work, no matter how old they are. We exhibit the effectiveness and the benefits of the new indices to unfold the full potential of the h-index, with extensive experimental results obtained from DBLP, a widely known on-line digital library.Comment: 19 pages, 17 tables, 27 figure