This paper examines the practical usefulness of two new journal performance metrics, namely the Eigenfactor score, which is said to measure “importance”, and Article Influence score, which is said to measure “prestige”, using the most recent ISI data for 2009 for the 200 most highly cited journals in each of the Sciences and Social Sciences, and compares them with two existing ISI metrics, namely Total Citations and the 5-year Impact Factor (5YIF) of a journal. It is shown that the Sciences and Social Sciences are different in terms of the strength of the relationship of journal performance metrics, although the actual relationships are very similar. Moreover, the importance and prestige journal performance metrics are shown to be closely related to the two existing ISI metrics, and hence add little in practical usefulness to what is already known. These empirical results are compared with existing results in the literature.Journal performance metrics; Research assessment measures; Total citations; 5-year impact factor (5YIF); Eigenfactor; Article influence; Importance; Prestige

Chia-Lin Chang

Les Oxley

Michael McAleer

English

Research Papers in Economics

DEPARTMENT OF ECONOMICS AND FINANCE 
 
COLLEGE OF BUSINESS AND ECONOMICS 
 
UNIVERSITY OF CANTERBURY 
 
CHRISTCHURCH, NEW ZEALAND 
 
 
 
 
 
Journal Impact Factor Versus 
Eigenfactor and Article Influence 
 
  
 Chia-Lin Chang, Michael McAleer, and Les Oxley 
 
 
 
 
 
 
 
WORKING PAPER 
 
 
No. 67/2010 
 
 
 
Department of Economics and Finance 
College of Business and Economics 
University of Canterbury 
Private Bag 4800, Christchurch 
  New Zealand Journal Impact Factor Versus  
Eigenfactor and Article Influence*  
 
 
Chia-Lin Chang 
Department of Applied Economics 
National Chung Hsing University 
Taichung, Taiwan 
 
 
 
Michael McAleer 
Econometric Institute 
Erasmus School of Economics 
Erasmus University Rotterdam 
and 
Tinbergen Institute 
The Netherlands 
and 
Institute of Economic Research 
Kyoto University 
Japan 
 
 
Les Oxley 
Department of Economics and Finance 
University of Canterbury 
New Zealand 
 
 
 
Revised: November 2010  
  
 
 
 
* For financial support, the first author acknowledges the National Science Council, Taiwan; 
the second author acknowledges the Australian Research Council, National Science Council, 
Taiwan, and the Japan Society for the Promotion of Science; and the third author 
acknowledges the Royal Society of New Zealand, Marsden Fund. An earlier abridged version 
was circulated under the title “Article Influence = 5YIF/2”. 
1 
  
Abstract 
 
This paper examines the practical usefulness of two new journal performance metrics, 
namely the Eigenfactor score, which is said to measure “importance”, and Article Influence 
score, which is said to measure “prestige”, using the most recent ISI data for 2009 for the 200 
most highly cited journals in each of the Sciences and Social Sciences, and compares them 
with two existing ISI metrics, namely Total Citations and the 5-year Impact Factor (5YIF) of 
a journal. It is shown that the Sciences and Social Sciences are different in terms of the 
strength of the relationship of journal performance metrics, although the actual relationships 
are very similar. Moreover, the importance and prestige journal performance metrics are 
shown to be closely related to the two existing ISI metrics, and hence add little in practical 
usefulness to what is already known. These empirical results are compared with existing 
results in the literature. 
 
Keywords: Journal performance metrics, Research assessment measures, Total citations, 5-
year impact factor (5YIF), Eigenfactor, Article influence, Importance, Prestige. 
 
JEL Classification: A12. 
 
 
 
2 
  
“They’re digging in the wrong place!” 
Indiana Jones, Raiders of the Lost Ark 
 
 
 
1.  Introduction  
 
Evaluating research quality is fundamental to the Sciences and Social Sciences. Research 
assessment rankings are essential to evaluate the research performance of individuals and the 
quality of academic journals. The perceived research performance of individual researchers is 
crucial for hiring, firing, tenure and promotion decisions. In the absence of clear signals 
regarding the inherent, and frequently latent, quality of research, the perceived quality of a 
journal may frequently be used as a proxy, albeit inappropriately, for the quality of a research 
paper.  
 
Most journal performance metrics are based on citations. The Thomson Reuters ISI Web of 
Science database (hereafter ISI) is a leading high quality database for generating research 
assessment measures, especially citations, to evaluate the research performance of individual 
researchers and the quality of academic journals.  
 
This paper examines the novelty and usefulness of two new journal performance metrics, 
namely the Eigenfactor score, which is said to measure “importance”, and Article Influence 
score, which is said to measure “prestige”, using ISI data for 2009 for the 200 most highly 
cited journals in each of the Sciences and Social Sciences, and compares them with two 
existing ISI metrics, namely Total Citations and the 5-year Impact Factor (5YIF) of a journal.  
 
It is shown that the Sciences and Social Sciences are different in terms of the strength of the 
relationship of journal performance metrics, although the actual relationships are nevertheless 
very similar. Moreover, the importance and prestige metrics are shown to be closely related 
to the two existing ISI metrics, and hence add little to what is already known. These empirical 
results are compared with existing results in the literature.  
 
The plan of the remainder of the paper is as follows. Section 2 presents four key research 
assessment measures (RAM), namely the 2-year impact factor (2YIF) of a journal, 5-year 
3 
 impact factor (5YIF) of a journal, Eigenfactor score and Article Influence score. Section 3 
reports some empirical analyses of the key RAM, as well as Total Citations, and compares 
the results with those that are available in the literature. Section 4 gives some concluding 
remarks. 
 
2.  Key Research Assessment Measures (RAM) 
 
Leading journal performance measures for an ISI Journal Citations Reports (JCR) calendar 
year, which is the year before the annual Research Assessment Measures (RAM) are 
released, are as follows: 
 
(1) 2-year impact factor (2YIF): The classic 2-year impact factor (2YIF) of a journal is 
typically referred to as “the impact factor”. For a given year, the 2YIF of a journal is defined 
as “Total citations in a year to papers published in a journal in the previous 2 years / Total 
papers published in a journal in the previous 2 years”.  The choice of 2 years by ISI is 
arbitrary. 
 
(3) 5-year impact factor (5YIF): For a given year, the 5YIF of a journal is defined as “Total 
citations in a year to papers published in a journal in the previous 5 years / Total papers 
published in a journal in the previous 5 years.” As in the case of 2YIF, the choice of 5 years 
by ISI is arbitrary.  
 
(5) Eigenfactor score: The Eigenfactor score (see Bergstrom (2007), Bergstrom, West and 
Wiseman (2008), and Bergstrom and West (2008)) is a modified 5YIF. For a given year, the 
Eigenfactor algorithm (see www.eigenfactor.org/methods.htm) effectively ranks journals 
according to citations and the length of time that researchers are logged on to a journal’s 
website. It is, in effect, a journal website citation search, and is said to measure “importance”. 
The amount of time spent checking hard copies of journals is not included in the Eigenfactor 
score. 
 
(6) Article Influence: The Article Influence score measures the relative importance of an ISI 
journal on a per-article basis, and is a standardized Eigenfactor score. For a given year, 
Article Influence of an ISI journal is defined as “Eigenfactor score divided by the fraction of 
4 
 all ISI articles published by the ISI journal.” Article Influence is defined in terms of the 
relative time that researchers are logged on to a journal’s website, and is said to measure 
“prestige”. 
 
3. Empirical Analysis 
 
3.1 Existing empirical results 
 
Davis (2008) used two simple linear regressions to relate the logarithm of Eigenfactor score 
to the logarithm of Total Citations, and the logarithm of 2YIF to the logarithm of Eigenfactor 
score, giving a high 
2 R = 0.950 and a reasonably high 
2 R = 0.860, respectively. These are 
interesting empirical findings, even though it might be argued that the results would have 
been more informative if the Eigenfactor score had been related to 5YIF as both bibliometric 
measures are calculated over a 5-year citation period. 
 
Using a simple linear regression in levels rather than logarithms, Fersht (2009) showed that 
there was a high 
2 R  = 0.968 between the Eigenfactor score and Total Citations for the top 
200 most highly cited ISI journals (based on 2YIF) in the Sciences, based on ISI Total 
Citations data for 2007. This is very similar to the results obtained for the Eigenfactor score 
and Total Citations in Davis (2008), even though the simple linear regressions used in the two 
papers differed in terms of the data transformations. 
 
Rousseau et al. (2009) calculated the Spearman correlation coefficients between the pairs 
2YIF and Eigenfactor score, 2YIF and Article Influence score, and Eigenfactor and Article 
Influence scores to be 0.827, 0.918 and 0.827, respectively. It might be argued that these 
interesting empirical results might have been more relevant if 5YIF had been related to the 
Eigenfactor and Article Influence scores as each of these three bibliometric measures is 
calculated over a 5-year citation period.  
 
Franceschet (2009) considered three pairs of variables for calculating correlation coefficients, 
namely 2YIF and Eigenfactor score, 5YIF and Eigenfactor score, and the Eigenfactor and 
Article Influence scores, giving correlations of 0.770, 0.770 and 0.760, respectively. Two 
simple linear regressions in the levels of three RAM were considered, with 
2 R  = 0.810 for a 
5 
 simple linear regression of Article Influence score on 2YIF, and 
2 R  = 0.880 for a simple 
linear regression of Article Influence score on 5YIF. The marginal effects of 2YIF and 5YIF 
on the Article Influence score were 0.446 and 0.452, respectively. 
 
Elkins et al. (2010) and Arendt (2010) both considered the relationship between 2YIF and 
Article Influence score, with the former calculating a correlation coefficient of 0.790 and the 
latter a relatively low 
2 R  = 0.596 from a simple linear regression based on median values 
across scientific fields. It might be repeated that these interesting empirical results would 
have been more meaningful if the Article Influence score had been related to 5YIF rather 
than 2YIF so that the bibliometric measures would have been calculated over the same 
citation period. 
 
3.2 Additional empirical results 
 
In order to contribute to the existing literature on empirical findings regarding alternative 
RAM, in Figures 1-4 we evaluate the 200 most highly cited journals, according to 2YIF, in 
both the sciences and social sciences for 2009. These figures relate the Eigenfactor score to 
Total Citations and the Article Influence score to 5YIF. The Total Citations data for 2009 for 
the Sciences and Social Sciences were downloaded from ISI on 19 June 2010 and 20 June 
2010, respectively.  
 
A simple linear regression, with the Eigenfactor score as a function of Total Citations, is 
given in Figures 1 and 3 for the Sciences and Social Sciences, respectively. The estimated 
model shows that the Eigenfactor score increases, on average, by 0.000004 and 0.000003 for 
each unit increase in Total Citations for 2009 for the Sciences and Social Sciences, 
respectively. The goodness-of-fit measures, namely 
2 R   = 0.931 and 
2 R   = 0.659 for the 
Sciences and Social Sciences, respectively, show that the Eigenfactor score can be estimated 
accurately, especially for the Sciences, on the basis of a simple linear regression against Total 
Citations.  
 
The approximate relationships between the Eigenfactor score and Total Citations for the 
Sciences and Social Sciences, respectively, can be expressed as: 
 
6 
 Eigenfactor score = k (Total Citations) 
 
where k = 0.0000033 and k = 0.000002 for Sciences and Social Sciences, respectively. The 
estimated value of k = 0.00000396 in Ferscht (2009) for the Sciences, based on ISI Total 
Citations data for 2007, is in accordance with the result obtained in the present paper, as is 
the value of 
2 R . 
 
Another simple linear regression, with the Article Influence score as a function of 5YIF, is 
given in Figures 2 and 4 for 2009 for the Sciences and Social Sciences, respectively. The 
estimated models show that the Article Influence score increases, on average, by 0.489 and 
0.479 for each unit increase in 5YIF for 2009 for the Sciences and Social Sciences, 
respectively.  
 
The goodness-of-fit measures, as given by 
2 R   = 0.923 and 
2 R   = 0.572 for 2009 for the 
Sciences and Social Sciences, respectively, show that the Article Influence score can be 
approximated very accurately for the Sciences, and reasonably accurately for the Social 
Sciences, on the basis of a simple linear regression relationship of Article Influence score 
against 5YIF, namely: 
 
Article Influence score = 5YIF/2. 
 
Although the goodness-of-fit value of 
2 R  obtained in the present paper is slightly higher than 
in Franceschet (2009), namely 
2 R  = 0.880, in relating the Article Influence score to 5YIF, 
the latter paper had an effect of 5YIF on Article Influence score of  0.452, which is very 
similar to that proposed above. 
 
4. Conclusion  
  
Although the Sciences and Social Sciences are dramatically different in terms of the strength 
of the underlying relationship of the journal performance metrics considered in this paper, the 
actual empirical relationships are very similar. As Article Influence is a modification of 
5YIF, it is perhaps not surprising that the two scores are highly and positively correlated.  
 
7 
 Given the very high correlations between the Eigenfactor score and Total Citations, and 
between the Article Influence score and 5YIF, and the corresponding high 
2 R  values for the 
simple linear regressions, the Eigenfactor score and Article Influence score would not seem 
to be entirely necessary for the Social Sciences, and not at all necessary for the Sciences, 
relative to the leading journal performance measures that are already available, namely Total 
Citations and 5YIF, respectively. 
 
As the journal performance measures captured in the Eigenfactor and Article Influence scores, 
which are said to measure “importance” and “prestige”, respectively, add little to what is 
already available in the ISI Total Citations and 5-year impact factor (5YIF) of a journal, we 
have no hesitation in concurring with Indiana Jones, who made the following remark 
regarding his competitors who were searching for the Lost Ark of the Covenant: “They’re 
digging in the wrong place!” 
8 
  
References 
 
Arendt, J. (2010), Are article influence scores comparable across scientific fields?, Issues in 
Science and Technology Librarianship, 60. Retrieved from http://www.istl.org/10-
winter/refereed2.html.  
 
Bergstrom C. (2007), Eigenfactor: Measuring the value and prestige of scholarly journals, 
C&RL News, 68, 314-316.  
 
Bergstrom, C.T. and. West (2008), Assessing citations with the Eigenfactor™ metrics, 
Neurology, 71, 1850–1851. 
 
Bergstrom, C.T., J.D. West and M.A. Wiseman (2008), The Eigenfactor™ metrics, Journal 
of Neuroscience, 28(45), 11433–11434. 
 
Davis, P.M. (2008), Eigenfactor: Does the principle of repeated improvement result in better 
estimates than raw citation counts?, Journal of the American Society for Information Science 
and Technology, 59(13), 2186-2188. 
 
Elkins, M.R., C.G. Maher, R.D. Herbert, A.M. Moseley and C. Sherrington (2010), 
Correlation between the journal impact factor and three other journal citation indices, 
Scientometrics, 85, 81-93. 
 
Fersht, A. (2009), The most influential journals: Impact factor and Eigenfactor, Proceedings 
of the National Academy of Sciences of the United States of America, 106(17), 6883-6884.  
 
Franceschet, M. (2010), Journal influence factors, Journal of Informetrics, 4, 239-248. 
 
ISI Web of Science (2010), Journal Citation Reports, Essential Science Indicators, Thomson 
Reuters ISI. 
 
Rousseau, R. et al. (2009), On the relation between the WoS impact factor, the Eigenfactor, 
the SCImago journal rank, the article influence score and the journal h-index, Conference 
Proceedings, Nanjing University, April 2009. 
9 
  
Table 1  
 
Correlations and 
2 R  for Various Research Assessment Measures (RAM) 
 
Authors Correlated  Variables  Correlation  2 R  
(log Eigenfactor, log TC)  -  0.950 
Davis (2008) 
(log 2YIF, log Eigenfactor)  -  0.860 
Ferscht (2009)  (Eigenfactor, TC)  -  0.968 
(2YIF, Eigenfactor)  0.827  - 
(2YIF, AI)  0.918  -  Rousseau et al. (2009) 
(Eigenfactor, AI)  0.827  - 
(2YIF, AI)  -  0.810 
(5YIF, AI)  -  0.880 
(2YIF, Eigenfactor)  0.770  - 
(5YIF, Eigenfactor)  0.770  - 
Franceschet (2009) 
(Eigenfactor, AI)  0.760  - 
Elkins et al. (2010)  (2YIF, AI)  0.790  - 
Arendt (2010)  (2YIF, AI)  -  0.596 
(Eigenfactor, TC)  -  0.931 
This paper (2010) 
(5YIF, AI)  -  0.923 
 
Note: AI denotes Article Influence and TC denotes Total Citations. The 
correlations are Spearman’s correlation coefficients, and the 
2 R  values  are 
calculated from simple linear regression models.  
10 
 Figure 1 
 
Eigenfactor Score and Total Citations for 200 Most Highly Cited Journals  
in Sciences for 2009 
 
 
 
Note: Citations data were downloaded from ISI on 19 June 2010. The OLS regression 
results are as follows (t-ratios in parentheses): 
 
(51.59)     (-3.42)                                     
931 . 0          error,     Citations   Total 06 - 3.32E 0.022 -       Score r  Eigenfacto
2 = + × + = R
 
 
 
 
 
11 
 Figure 2 
 
Article Influence Score and 5YIF for 200 Most Highly Cited Journals 
in Sciences for 2009 
 
 
 
Note: Citations data were downloaded from ISI on 19 June 2010. The OLS regression 
results are as follows (t-ratios in parentheses): 
 
(48.54)     (-6.65.)                                     
923 . 0          error,     YIF 5 0.489 0.719 -       Influence   Article
2 = + × + = R  
12 
 Figure 3 
 
Eigenfactor Score and Total Citations for 200 Most Highly Cited Journals  
in Social Sciences for 2009 
 
 
 
Note: Citations data were downloaded from ISI on 20 June 2010. The OLS regression 
results are as follows (t-ratios in parentheses): 
 
(19.55)     (2.85)                                     
659 . 0          error,     Citations   Total 06 - 1.99E 0.029       Score r  Eigenfacto
2 = + × + = R    
13 
 Figure 4 
 
Article Influence Score and 5YIF for 200 Most Highly Cited Journals  
in Social Sciences for 2009 
 
 
 
 
Note: Citations data were downloaded from ISI on 20 June 2010. The OLS regression 
results are as follows (t-ratios in parentheses): 
 
(16.25)     (0.10)                                     
572 . 0          error,     YIF 5 0.479 0.160       Influence   Article
2 = + × + = R  
14 
 

Are article influence scores comparable across scientific fields?, Issues

Correlation between the journal impact factor and three other journal citation indices,

Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts?,

Eigenfactor: Measuring the value and prestige of scholarly journals,

Journal influence factors,

On the relation between the WoS impact factor, the Eigenfactor, the SCImago journal rank, the article influence score and the journal h-index, Conference Proceedings,

The most influential journals: Impact factor and Eigenfactor,

Journal Impact Factor Versus Eigenfactor and Article Influence

http://www.econ.canterbury.ac.nz/RePEc/cbt/econwp/1067.pdf

Journal Impact Factor Versus Eigenfactor and Article Influence

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics