This editorial discusses the rationale for using article-specific rather than journal-specific metrics for evaluating highly published authors

AM Lever

JE Hirsch

K Boris-Lawrie

KT Jeang

Kuan-Teh Jeang

English

PubMed

Abstract This editorial discusses the rationale for using article-specific rather than journal-specific metrics for evaluating highly published authors.</p

Jeang Kuan-Teh

Directory of Open Access Journals

Retrovirology

The importance of individualized article-specific metrics for evaluating research productivity

Crossref

Springer - Publisher Connector

BioMed CentralRetrovirology
ssOpen AcceEditorial
The importance of individualized article-specific metrics for 
evaluating research productivity
Kuan-Teh Jeang
Address: National Institutes of Health, Bethesda, MD, USA
Email: Kuan-Teh Jeang - kj7e@nih.gov 
Abstract
This editorial discusses the rationale for using article-specific rather than journal-specific metrics
for evaluating highly published authors.
Editorial
Mark Patterson of PLoS (Public Library of Science)
recently wrote an online piece http://www.plos.org/cms/
node/478 on how to measure impact where it matters.
Patterson makes an important point that one should
focus on article specific metrics when evaluating a pub-
lished paper rather than relying "on the name and the
impact factor (IF) of the journal in which the work is pub-
lished". In the past, it was not always easy to assess quickly
and accurately the citations to individually published arti-
cles. Today, many electronic tools (e.g. ISI Web of Science,
Scopus, Google Scholar) exist that can accomplish this
task facilely and reliably. Because there are inherent short-
comings to how a journal's IF is calculated and because of
the rather poor representativeness of the IF for the cita-
tions to individual articles [1,2], institutions and peer-
review bodies should be encouraged strongly to employ
article-specific measures in preference to journal IFs in
evaluations.
Article-specific citations are often not used properly in
evaluating published authors. For example, in some cir-
cles, it has become fashionable to create lists of "highly
cited" scientists in various fields (e.g. http://isihighly
cited.com/; highly cited in immunology, highly cited in
microbiology, highly cited in molecular biology and
genetics etc...). In some respects, these lists could be useful
conveniences, provided that the users understand clearly
how they are generated and what they mean (and do not
mean). One could assume that "highly cited in microbiol-
ogy" is based on article specific-citations. In fact, this
would be a mistaken assumption because the listing is
actually based on journal-specific data. What does this
mean? By way of explanation, let's consider a hypothetical
illustration. If John Smith were an author of 10 papers on
HIV-1 published in Cell or the Journal of Biological Chemis-
try (which are not counted by ISIHighlyCited as microbi-
ology journals) and if these 10 papers were cited
cumulatively 1,000 times over a specified duration, then
Smith's citation counts based on these papers for pur-
poses of "highly cited in microbiology" would be 0. On
the other hand, if the exactly same 10 Smith papers on
HIV were unsuccessful in initial submissions to Cell or the
Journal of Biological Chemistry, but were subsequently suc-
cessfully published in the Journal of Virology, Retrovirology,
or Virology (all counted as microbiology journals), then
the 1,000 citations to these papers would add 1,000
counts to Smith's ranking for purposes of "highly cited in
microbiology". So, here is an example where journal-spe-
cific metrics trump article-specific measures. In order to be
"highly cited in microbiology", what one publishes (i.e.
article-specific content on HIV) counts not unless it is
published in a journal deemed as "microbiology" (i.e. a
journal-specific metric). Thus, this illustration shows that
Published: 16 September 2009
Retrovirology 2009, 6:82 doi:10.1186/1742-4690-6-82
Received: 15 September 2009
Accepted: 16 September 2009
This article is available from: http://www.retrovirology.com/content/6/1/82
© 2009 Jeang; licensee BioMed Central Ltd. 
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), 
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Page 1 of 4
(page number not for citation purposes)
Retrovirology 2009, 6:82 http://www.retrovirology.com/content/6/1/82Page 2 of 4
Table 1: H-index and citation frequencies of selected Retrovirology editorial board members.
Title Name Role within Retro-
virology
Institution City Country H index Total times cited 
since 1996
Dr. Kuan-Teh Jeang Editor-in-Chief NIH Bethesda USA 46 9799
Dr. Monsef Benkirane Editor CNRS Montpellier France 23 2210
Dr. Ben Berkhout Editor Academic Med. Ctr Amsterdam the Netherlands 40 6925
Dr. Andrew Lever Editor Cambridge 
University
Cambridge UK 19 2065
Dr. Mark Wainberg Editor McGill University Montreal Canada 40 10058
Dr. Masahiro Fujii Editor Niigata University Niigata Japan 21 2186
Dr. Michael Lairmore Editor Ohio State 
University
Columbus USA 21 2226
Dr. Michael Bukrinsky Ed Board George Washington 
Univ
Washington DC USA 26 5218
Dr. Dong-yan Jin Ed Board Hong Kong U Hong Kong China 25 2675
Dr. Klaus Strebel Ed Board NIH Bethesda USA 27 4395
Dr. Tom J. Hope Ed Board U. Illinois Chicago USA 27 4730
Dr. Stephane Emiliani Ed Board Cochin Institute Paris France 19 2061
Dr. Patrick Green Ed Board Ohio State 
University
Columbus USA 19 1050
Dr. Mauro Giacca Ed Board Int. Ctr. Genetics Trieste Italy 38 5795
Dr. Olivier Schwartz Ed Board Institut Pasteur Paris France 31 5209
Dr. Leonid Margolis Ed Board National Inst Child 
Health
Bethesda USA 23 2028
Dr. Fatah Kashanchi Ed Board George Washington 
U.
Washington DC USA 27 2725
Dr. Masao Matsuoka Ed Board Kyoto University Kyoto Japan 29 3834
Dr. Naoki Mori Ed Board University of the 
Ryukyus
Okinawa Japan 28 3375
Dr. Chou-Zen Giam Ed Board Uniform Services 
Med School
Bethesda USA 16 1698
Dr. David Derse Ed Board NCI Frederick USA 15 1828
Dr. Tatsuo Shioda Ed Board Osaka Univ Osaka Japan 24 2110
Dr. John Semmes Ed Board Eastern Virginia Med 
College
Norfolk USA 29 3416
Dr. Anne Gatignol Ed Board McGill Univ. Montreal Canada 17 1542(page number not for citation purposes)
Retrovirology 2009, 6:82 http://www.retrovirology.com/content/6/1/82Dr. Rogier Sanders Ed Board Academic Med. Ctr Amsterdam the Netherlands 13 955
Dr. Chen Liang Ed Board McGill Univ. Montreal Canada 19 976
Dr. Finn Skou Pedersen Ed Board University of Aarhus Aarhus Denmark 19 1498
Dr. Renaud Mahieux Ed Board Pasteur Int. Paris France 24 1489
Dr. Neil Almond Ed Board NIBSC Potters Bar UK 15 1370
Dr. Stephen P. Goff Ed Board Columbia University New York USA 44 14851
Dr. Johnson Mak Ed Board Burnet Inst. Med. 
Research
Victoria Australia 17 1679
Dr. Christine Kozak Ed Board NIH Bethesda USA 29 7814
Dr. Greg Towers Ed Board Univ. College London UK 17 1558
Dr. Eric Cohen Ed Board Univ. Montreal Montreal Canada 37 7047
Dr. Warner Greene Ed Board UCSF San Francisco USA 42 11011
Dr. Jean-luc Darlix Ed Board U. Lyon Lyon France 33 6070
Dr. Eric Freed Ed Board NCI Frederick USA 31 4906
Dr. Toshiki Watanabe Ed Board Univ. of Tokyo Tokyo Japan 24 2576
Dr. Mari Kannagi Ed Board Tokyo Med and 
Dental U
Tokyo Japan 17 1474
Dr. Frank Kirchhoff Ed Board University of Ulm Ulm Germany 34 5478
Dr. Jennifer Nyborg Ed Board Colorado State U Fort Collins USA 18 1671
Dr. Akifumi Takaori-
Kondo
Ed Board Kyoto University Kyoto Japan 14 718
Dr. Marc Sitbon Ed Board CNRS Montpellier France 13 814
Dr. Paul Gorry Ed Board MacFarlane Burnet 
Institute
Melbourne Australia 16 835
Dr. David Harrich Ed Board Queensland Inst 
Medical Res.
Brisbane Australia 12 1063
Dr. Susan Marriott Ed Board Baylor Houston USA 15 1102
Dr. Alan Cochrane Ed Board U Toronto Toronto Canada 11 1191
Dr. Yiming Shao Ed Board China CDC Beijing China 14 1123
Dr. Vinayaka Prasad Ed Board Albert Einstein 
College Medicine
New York USA 19 1239
Dr. Roger Pomerantz Ed Board Tibotec Yardley USA 34 6912
Table 1: H-index and citation frequencies of selected Retrovirology editorial board members. (Continued)Page 3 of 4
(page number not for citation purposes)
Retrovirology 2009, 6:82 http://www.retrovirology.com/content/6/1/82Dr. Li Wu Ed Board Medical College 
Wisconsin
Milwaukee USA 30 5617
Dr. Anne-Mieke 
Vandamme
Ed Board Rega Inst. and Univ 
Hospitals
Leuven Belgium 35 4994
Dr. Alan Engelman Ed Board Harvard Univ. Boston USA 25 4070
Dr. Paul Clapham Ed Board Univ. Massachusetts Worcester USA 30 6495
Dr. Vinay Pathak Ed Board NCI Frederick USA 25 1951
Dr. Jeremy Luban Ed Board Univ. Geneva Geneva Switzerland 29 4469
Table 1: H-index and citation frequencies of selected Retrovirology editorial board members. (Continued)
ratings based on journal-specific data that do not properly ReferencesPublish with BioMed Central   and  every 
scientist can read your work free of charge
"BioMed Central will be the most significant development for 
disseminating the results of biomedical research in our lifetime."
Sir Paul Nurse, Cancer Research UK
Your research papers will be:
available free of charge to the entire biomedical community
peer reviewed and published immediately upon acceptance
cited in PubMed and archived on PubMed Central 
yours — you keep the copyright
Submit your manuscript here:
http://www.biomedcentral.com/info/publishing_adv.asp
BioMedcentral
integrate article-specific measures can be misleading when
used to rate scientists. For retrovirologists, Retrovirology
has emphasized consistently the use of person-specific
measures of H-index [3] and total citations. Indeed, annu-
ally for the past three years, these data have been pre-
sented, using the Scopus data base http://
www.scopus.com, for selected Retrovirology editorial
board members (see Table 1) [1,2].
Finally, one should not overlook the merits of awards and
prizes in evaluating highly accomplished colleagues.
Awards/prizes can come in two flavors; one as "leading"
and the other as "lagging" indicators of scientific poten-
tial/productivity. For example, "life-time achievement"
awards would be a "lagging" measure of one's achieve-
ments, while a "young" investigator prize might be a
"leading" indicator of future potential. Retrovirology annu-
ally awards a "Retrovirology Prize" to a mid-career scientist
[4,5]. The Prize aims to recognize "lagging" and "leading"
benchmarks. It rewards the past achievements of a scien-
tist who is in his/her mid-career and who still has substan-
tial lead-time to accomplish future breakthrough research
in retrovirology [5-8]. With this editorial, this year's nom-
ination period for the 2009 Retrovirology Prize to recognize
a retrovirologist for non-HIV-retrovirology research is
open. The nomination period will close on October 31,
2009. The rules for nomination and the selection proce-
dures remain the same as in past years [9,10]. Interested
individuals can direct email inquiries to editorial@retro-
virology.com.
Authors' contributions
KTJ wrote this editorial.
Acknowledgements
I thank Andrew Dayton and Mark Wainberg for critically reading this edi-
torial and Daniel Schmidt for assistance in preparing Table 1.
1. Jeang KT: Impact factor, H index, peer comparisons, and Ret-
rovirology: is it time to individualize citation metrics?  Retro-
virology 2007, 4:42.
2. Jeang KT: H-index, mentoring-index, highly-cited and highly-
accessed: how to evaluate scientists?  Retrovirology 2008, 5:106.
3. Hirsch JE: Does the H index have predictive power?  Proc Natl
Acad Sci USA 2007, 104:19193-19198.
4. Jeang KT: Life after 45 and before 60: the Retrovirology Prize.
Retrovirology 2005, 2:26.
5. Jeang KT: The 2008 Retrovirology Prize: Ben Berkhout and his
RNA world.  Retrovirology 2008, 5:113.
6. Jeang KT: Small philanthropy and big science: the RETROVI-
ROLOGY prize and Stephen P. Goff.  Retrovirology 2005, 2:43.
7. Lever AM: Science--a life fully lived: Joe Sodroski wins the
2006 Retrovirology Prize.  Retrovirology 2006, 3:45.
8. Boris-Lawrie K: Bridging fundamental RNA biology, retroviral
replication, and oncogenesis: Karen Beemon wins the 2007
Retrovirology Prize.  Retrovirology 2007, 4:88.
9. Jeang KT: The young, not-so-young, and the 2007 Retrovirol-
ogy Prize: call for nominations.  Retrovirology 2007, 4:64.
10. Jeang KT: Recognizing mid-career productivity: the 2008 Ret-
rovirology Prize, call for nomination.  Retrovirology 2008, 5:80.Page 4 of 4
(page number not for citation purposes)


Bridging fundamental RNA biology, retroviral replication, and oncogenesis: Karen Beemon wins the 2007 Retrovirology Prize. Retrovirology

Does the H index have predictive power?

H-index, mentoring-index, highly-cited and highlyaccessed: how to evaluate scientists? Retrovirology

Impact factor, H index, peer comparisons, and Retrovirology: is it time to individualize citation metrics? Retrovirology

Life after 45 and before 60: the Retrovirology Prize. Retrovirology

Recognizing mid-career productivity: the 2008 Retrovirology Prize, call for nomination. Retrovirology

Science--a life fully lived: Joe Sodroski wins the 2006 Retrovirology Prize. Retrovirology

Small philanthropy and big science: the RETROVIROLOGY prize and

The young, not-so-young, and the 2007 Retrovirology Prize: call for nominations. Retrovirology

http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2757017

The importance of individualized article-specific metrics for evaluating research productivity

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals

Crossref

Springer - Publisher Connector