The citation merit of scientific publications

We propose a new method to assess the merit of any set of scientific papers in a given field based on the citations they receive. Given a citation indicator, such as the mean citation or the h-index, we identify the merit of a given set of n articles with the probability that a randomly drawn sample of n articles from a reference set of articles in that field presents a lower citation index. The method allows for comparisons between research units of different sizes and fields. Using a dataset acquired from Thomson Scientific that contains the articles published in the periodical literature in the period 1998-2007, we show that the novel approach yields rankings of research units different from those obtained by a direct application of the mean citation or the h-index.Citation analysis, Citation merit, Mean citation, h-index

The skewness of science in 219 sub-fields and a number of aggregates

This paper studies evidence from Thomson Scientific about the citation process of 3.7 million articles published in the period 1998-2002 in 219 Web of Science categories, or sub-fields. Reference and citation distributions have very different characteristics across sub-fields. However, when analyzed with the Characteristic Scores and Scales technique, which is replication and scale invariant, the shape of these distributions over three broad categories of articles appears strikingly similar. Reference distributions are mildly skewed, but citation distributions with a five-year citation window are highly skewed: the mean is twenty points above the median, while 9-10% of all articles in the upper tail account for about 44% of all citations. The aggregation of sub-fields into disciplines and fields according to several aggregation schemes preserve this feature of citation distributions. It should be noted that when we look into subsets of articles within the lower and upper tails of citation distributions the universality partially breaks down. On the other hand, for 140 of the 219 sub-fields the existence of a power law cannot be rejected. However, contrary to what is generally believed, at the sub-field level the scaling parameter is above 3.5 most of the time, and power laws are relatively small: on average, they represent 2% of all articles and account for 13.5% of all citations. The results of the aggregation into disciplines and fields reveal that power law algebra is a subtle phenomenon.

A comparison of the scientific performance of the U.S. and the European Union at the turn of the 21st century.

In this paper, scientific performance is identified with the impact that journal articles have through the citations they receive. In 15 disciplines, as well as in all sciences as a whole, the EU share of total publications is greater than that of the U.S. However, as soon as the citations received by these publications are taken into account the picture is completely reversed. Firstly, the EU share of total citations is still greater than the U.S. in only seven fields. Secondly, the mean citation rate in the U.S. is greater than in the EU in every one of the 22 fields studied. Thirdly, since standard indicators—such as normalized mean citation ratios—are silent about what takes place in different parts of the citation distribution, this paper compares the publication shares of the U.S. and the EU at every percentile of the world citation distribution in each field. It is found that in seven fields the initial gap between the U.S. and the EU widens as we advance towards the more cited articles, while in the remaining 15 fields—except for Agricultural Sciences—the U.S. always surpasses the EU when it counts, namely, at the upper tail of citation distributions. Finally, for all sciences as a whole the U.S. publication share becomes greater than that of the EU for the top 50% of the most highly cited articles. The data used refers to 3.6 million articles published in 1998–2002, and the more than 47 million citations they received in 1998–2007Research performance; Citation analysis; Scientific ranking; European paradox;

A comparison of the scientific performance of the U. S. and the European Union at the turn of the XXI century.

In this paper, scientific performance is identified with the impact journal articles achieve through the citations they receive. The empirical exercise refers to 3.6 million articles published in 1998-2002 in 22 scientific fields, and the more than 47 million citations they receive in 1998-2007. The first finding is that a failure to exclude co-authorship among member countries within the EU (European Union) may lead to a serious upward bias in the assignment of articles to this geographical area. In the second place, standard indicators, such as normalized mean citation ratios, are silent about what takes place in different parts of the citation distribution. Consequently, this paper compares the publication shares of the U.S. and the EU at every percentile of the world citation distribution in each field. In 15 disciplines, as well as in all sciences as a whole, the EU share of total publications is greater than that of the U.S. one. But as soon as the citations received by these publications are taken into account the picture is completely reversed. The mean citation rate in the U.S. is greater than in the EU in every one of the 22 fields. In seven fields, the initial gap between the U.S. and the EU widens up as we advance towards the more cited articles, while in the remaining 15 fields –except for Agricultural Sciences– the U.S. always surpasses the EU when it counts, namely, at the upper tail of citation distributions. For all sciences as a whole, the U.S publication share becomes greater than that of the EU one for the top 50% of the most highly cited articles.

The skewness of science in 219 sub-fields and a number of aggregates

This paper studies evidence from Thomson Scientific about the citation process of 3.7 million articles published in the period 1998-2002 in 219 Web of Science categories, or sub-fields. Reference and citation distributions have very different characteristics across sub-fields. However, when analyzed with the Characteristic Scores and Scales technique, which is size and scale independent, the shape of these distributions appear extraordinarily similar. Reference distributions are mildly skewed, but citation distributions with a five-year citation window are highly skewed: the mean is twenty points above the median, while 9-10% of all articles in the upper tail account for about 44% of all citations. The aggregation of sub-fields into disciplines and fields according to several aggregation schemes preserve this feature of citation distributions. On the other hand, for 140 of the 219 sub-fields the existence of a power law cannot be rejected. However, contrary to what is generally believed, at the sub-field level the scaling parameter is above 3.5 most of the time, and power laws are relatively small: on average, they represent 2% of all articles and account for 13.5% of all citations. The results of the aggregation into disciplines and fields reveal that power law algebra is a subtle phenomenon.

The citation merit of scientific publications

We propose a new method to assess the merit of any set of scientific papers in a given field based on the citations they receive. Given a citation indicator, such as the mean citation or the h-index, we identify the merit of a given set of n articles with the probability that a randomly drawn sample of n articles from a reference set of articles in that field presents a lower citation index. The method allows for comparisons between research units of different sizes and fields. Using a dataset acquired from Thomson Scientific that contains the articles published in the periodical literature in the period 1998-2007, we show that the novel approach yields rankings of research units different from those obtained by a direct application of the mean citation or the h-index.Ruiz-Castillo acknowledges financial help from the Spanish MEC through grant SEJ2007-67436. Crespo and Ortuño-Ortín also acknowledge financial help from the Spanish MEC through grant ECO2010-19596. This paper is produced as part of the project Science, Innovation, Firms and markets in a Globalised World (SCIFI-GLOW), a Collaborative Project funded by the European Commission's Seventh Research Framework Programme, Contract number SSH7-CT-2008-217436

Constructing household specific consumer price indexes : an analysis of different techniques and methods

An earlier version of this paper was presented at the 1997 National Bureau of Economic Research Summer Institute on Price Index MeasurementThe primary purpose of this study is to produce household specific price indexes for consumer units or households living in the United States in the early 1990s. This paper is a report on how these household specific indexes were created. With household specific indexes, households are assumed to have nonhomothetic preferences, so changes in prices involve relative price changes between different sets of commodities and the resulting indexes will differ systematically between different households. We examine several different approaches to construct these indexes. Our indexes are based on internal U.S. Consumer Expenditure Survey (CEX) data for 1990-91 and Bureau of Labor Statistics Consumer Price Index (CPI) data from winter 1981, 1987, and 1991. Our base period is 1990-91. Using these data we produce Paasche type household specific indexes. In addition we propose an alternative definition of total expenditures, based on the CPI market basket commodity space, to be used for welfare analysis. Our underlying motivation for conducting this study was to compare real welfare inequality in Spain and the U.S. in the 1980s for another study (Garner et al. forthcoming 1997). Because of this comparison, we were somewhat restricted in our approach. CEX data are used to calculate CPI market basket item budget shares for each interviewed household. Price indexes are merged with the household budget data at various levels of geographic and market basket item aggregation, and the variability in these indexes are compared in order to measure the value of using detailed consumption space over aggregated consumption space. In this study we introduce two novel approaches to producing household specific price indexes using BLS data. First, expenditure data from the Consumer Expenditure Diary survey, which is more detailed than the Interview, are used to impute missing consumption items for the Interview households. And second, a method to impute household indexes for the rural population is presented. Two different types of samples, horizontal and vertical (based on assumptions about the Interview households selected to define the base period), are used to provide the weights for the price indexes. Indexes are presented based on Interview only items and all items commodity spaces for the horizontal and vertical samples with additional indexes produced for consumer units living in urban and rural areas. From our study we conclude that indexes based on expenditures for the horizontal and vertical samples do not differ significantly for the time periods of our study. However, differences in the indexes do result for the urban versus rural samples, with consumer units living in urban areas facing greater changes in relative prices than are faced by consumer units living in rural areas. The all-item indexes produced slightly higher index values than did the Interview-only item indexes. Relative prices appear to be pro-poor during the 1980s

Differences in citation impact across scientific fields

This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in the 22 broad fields distinguished by Thomson Scientific. The paper is based on a model in which the number of citations received by any article is a function of the article’s scientific influence, and the field to which it belongs. The model includes a key assumption according to which articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998-2003 with a five-year citation window, we find that differences in citation practices between the 22 fields account for about 14% of overall citation inequality. Our empirical strategy for making comparisons of citation counts across fields is based on the strong similarities found in the behavior of citation distributions over a large quantile interval. We obtain three main results. Firstly, we provide a set of exchange rates to express citations in any field into citations in the all-fields case. (This can be done for articles in the interval between, approximately, the 71st and the 99th percentiles of their citation distributions). The answer is very satisfactory for 20 out of 22 fields. Secondly, when the raw citation data is normalized with our exchange rates, the effect of differences in citation practices is reduced to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields’ mean citation rates is found to be equally successful.The authors acknowledge financial support by Santander Universities Global Division of Banco Santander. Crespo and Ruiz-Castillo also acknowledge financial help from the Spanish MEC through grants SEJ2007-67436 and ECO2010-1959