Further clarifications about the success-index

The aim of this brief communication is to reply to a letter by Kosmulski (Journal of Informetrics 6(3):368-369, 2012), which criticizes a recent indicator called "success-index". The most interesting features of this indicator, presented in Franceschini et al. (Scientometrics, in press), are: (i) allowing the selection of an "elite" subset from a set of publications and (ii) implementing the field-normalization at the level of an individual publication. We show that the Kosmulski's criticism is unfair and inappropriate, as it is the result of a misinterpretation of the indicato

Coercive Journal Self Citations, Impact Factor, Journal Influence and Article Influence

This paper examines the issue of coercive journal self citations and the practical usefulness of two recent journal performance metrics, namely the Eigenfactor score, which may be interpreted as measuring “Journal Influence”, and the Article Influence score, using the Thomson Reuters ISI Web of Science (hereafter ISI) data for 2009 for the 200 most highly cited journals in each of the Sciences and Social Sciences. The paper also compares the two new bibliometric measures with two existing ISI metrics, namely Total Citations and the 5-year Impact Factor (5YIF) (including journal self citations) of a journal. It is shown that the Sciences and Social Sciences are different in terms of the strength of the relationship of journal performance metrics, although the actual relationships are very similar. Moreover, the journal influence and article influence journal performance metrics are shown to be closely related empirically to the two existing ISI metrics, and hence add little in practical usefulness to what is already known, except for eliminating the pressure arising from coercive journal self citations. These empirical results are compared with existing results in the bibliometrics literature

Participation bias assessment in three high-impact journals

Studies into participation bias have examined participation trends, where it occurs, the factors affecting it, and methods to try to reduce it. However, some authors only discuss participation bias at the end of the study, some acknowledge it and apply a method to try to reduce it, while others ignore it or dismiss it as negligible. Issues of three high-impact epidemiology journals were examined; 81 articles were read and reviewed for potential participation bias. Categories were used to classify the approach taken to participation bias and the results recorded. Of the 81 articles considered, 42 (51%) were eligible and could have suffered from participation bias. It was found that 57% of these articles ignored the effects of participation bias, while 17% only considered it briefly in the discussion. Few articles (22%) attempted to reduce the participation bias, with over half of these using unsuitable methods (55%). This review highlights how participation bias is often not considered and hence the conclusions drawn from these studies may not be correct

Searching biomedical databases on complementary medicine: the use of controlled vocabulary among authors, indexers and investigators

BACKGROUND: The optimal retrieval of a literature search in biomedicine depends on the appropriate use of Medical Subject Headings (MeSH), descriptors and keywords among authors and indexers. We hypothesized that authors, investigators and indexers in four biomedical databases are not consistent in their use of terminology in Complementary and Alternative Medicine (CAM). METHODS: Based on a research question addressing the validity of spinal palpation for the diagnosis of neuromuscular dysfunction, we developed four search concepts with their respective controlled vocabulary and key terms. We calculated the frequency of MeSH, descriptors, and keywords used by authors in titles and abstracts in comparison to standard practices in semantic and analytic indexing in MEDLINE, MANTIS, CINAHL, and Web of Science. RESULTS: Multiple searches resulted in the final selection of 38 relevant studies that were indexed at least in one of the four selected databases. Of the four search concepts, validity showed the greatest inconsistency in terminology among authors, indexers and investigators. The use of spinal terms showed the greatest consistency. Of the 22 neuromuscular dysfunction terms provided by the investigators, 11 were not contained in the controlled vocabulary and six were never used by authors or indexers. Most authors did not seem familiar with the controlled vocabulary for validity in the area of neuromuscular dysfunction. Recently, standard glossaries have been developed to assist in the research development of manual medicine. CONCLUSIONS: Searching biomedical databases for CAM is challenging due to inconsistent use of controlled vocabulary and indexing procedures in different databases. A standard terminology should be used by investigators in conducting their search strategies and authors when writing titles, abstracts and submitting keywords for publications

What makes a great journal great in economics?: The singer not the song

This paper analyses what makes a great journal great in economics. Alternative research assessment measures (RAM) are discussed, with an emphasis on the Thomson Reuters Institute for Scientific Information (ISI) Web of Science database. ISI RAM that are calculated annually or updated daily are defined, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), immediacy (zero-year impact factor (0YIF)), eigenfactor score, article influence, citation performance per paper online, h-index, Zinfluence, PI-BETA (papers ignored - by even the authors) and two new RAM measures, self-citation threshold approval rating and impact factor inflation. The data are analysed for the most highly cited journals in economics, management, business and business-finance on the basis of 2YIF. In addition to evaluating research in the most highly cited journals in economics, management, business and business-finance, the paper evaluates alternative RAM, highlights similarities and differences in RAM criteria, finds that several RAM capture similar performance characteristics, and finds that immediacy and PI-BETA are not highly correlated with other RAM. Harmonic mean rankings of the 12 RAM criteria are also presented. Emphasizing 2YIF to the exclusion of other useful RAM criteria can lead to a distorted evaluation of journal performance and influence. © 2010 Blackwell Publishing Ltd