Article thumbnail
Location of Repository

The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor

By Andy R Weale, Mick Bailey and Paul A Lear


BACKGROUND: Current methods of measuring the quality of journals assume that citations of articles within journals are normally distributed. Furthermore using journal impact factors to measure the quality of individual articles is flawed if citations are not uniformly spread between articles. The aim of this study was to assess the distribution of citations to articles and use the level of non-citation of articles within a journal as a measure of quality. This ranking method is compared with the impact factor, as calculated by ISI(®). METHODS: Total citations gained by October 2003, for every original article and review published in current immunology (13125 articles; 105 journals) and surgical (17083 articles; 120 journals) fields during 2001 were collected using ISI(® )Web of Science. RESULTS: The distribution of citation of articles within an individual journal is mainly non-parametric throughout the literature. One sixth (16.7%; IQR 13.6–19.2) of articles in a journal accrue half the total number of citations to that journal. There was a broader distribution of citation to articles in higher impact journals and in the field of immunology compared to surgery. 23.7% (IQR 14.6–42.4) of articles had not yet been cited. Levels of non-citation varied between journals and subject fields. There was a significant negative correlation between the proportion of articles never cited and a journal's impact factor for both immunology (rho = -0.854) and surgery journals (rho = -0.924). CONCLUSION: Ranking journals by impact factor and non-citation produces similar results. Using a non-citation rate is advantageous as it creates a clear distinction between how citation analysis is used to determine the quality of a journal (low level of non-citation) and an individual article (citation counting). Non-citation levels should therefore be made available for all journals

Topics: Research Article
Publisher: BioMed Central
Year: 2004
DOI identifier: 10.1186/1471-2288-4-14
OAI identifier:
Provided by: PubMed Central

Suggested articles


  1. (1998). Buffalo hump" in men with HIV-1 infection. Lancet
  2. (2003). Changing times for BJS.
  3. (1972). Citation analysis as a tool in journal evaluation. Science
  4. (1999). Divakar P: Sprucing up one's impact factor. Nature
  5. (2000). End of impact factors? Nature
  6. (1999). Journal impact factor: a brief review. CMAJ
  7. (1998). Mortality results for randomised controlled trial of early elective surgery or ultrasonographic surveillance for small abdominal aortic aneurysms. The UK Small Aneurysm Trial Participants. Lancet
  8. (2003). Nat Immunol
  9. (2003). Randomised controlled trials and quality of journals. Lancet
  10. (2003). Reflections on the impact factor. Arch Bronconeumol
  11. (2004). RL: The simultaneous evolution of author and paper networks.
  12. (1998). Spence JD: Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators.
  13. (2003). The "omnipotent" Science Citation Index impact factor.
  14. (2002). The counting house. Nature
  15. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.