The structure of various indices that are used to evaluate the research output of a scientist is discussed on a moral base.Indices should be used with caution. Albeit scientists who work in the field of infometrics are aware on this, other scientistsof other disciplines may not be fully aware. Measuring quality in science is only the half story. Some verbal description isrequired too

A. Ch. Mitropoulos

Directory of Open Access Journals

68Letter to the EditorIs it more difficult to write or to cite a paper?A. Ch. Mitropoulos*Department of Petroleum Technology, Cavala Institute of Technology, St. Lucas 65 404 Cavala.Received 4 May 2009; Accepted 15 June 2009AbstractThe structure of various indices that are used to evaluate the research output of a scientist is discussed on a moral base. Indices should be used with caution. Albeit scientists who work in the field of infometrics are aware on this, other scientists of other disciplines may not be fully aware. Measuring quality in science is only the half story. Some verbal description is required too. Keywords: mean number of citations, h-index, bibliometrics, benign addition principle, repugnance conclusion, vague representation.Journal of Engineering Science and Technology Review 2 (1) (2009) 68-70JOURNAL OFEngineering Science and Technology Reviewwww.jestr.orgDear Editor,It is common knowledge within the research community that an active scientist has to produce papers worth to be published in proper journals. When an author of this kind writes a paper usu-ally cites in his/her article the work of other authors. It seems to be easier to gain a citation than to write a paper. But again before a scientist earns a citation, it is probably necessary to write a paper. It is reasonable therefore to assume that there is a correlation or at least a naive correlation between the number of papers Np and the number of citations Nc that a scientist has. Scholar data bases [1-6] keep such records that may be used for measuring quality in science. Citation analysis usually concludes to an index that describes the performance of the scientist under consideration in terms of a mean number of citations per paper; nc=Nc/Np. A few years ago Hirsch [7] proposed an alternative index de-fined as follows. A scientist has index h if h of his or her Np papers have at least h citations each and the other (Np-h) papers have ≤ h citations each. The h-index has been tested in various case studies [8-12] and its proponents believe that it reflects reasonably well the potentiality of a scientist whereas any limitations that arise [13-21] can be smoothed out by auxiliary indices; the most known of which is the g-index [22].Concisely there are 4 major single-number criteria common-ly used to evaluate scientific output of a researcher, namely: Np, Nc, nc, and h. While the superiority of each one of these criteria against the others is still the subject of intense debate [23,24], I would like in this article to focus on the legitimacy of overusing such indices in the frame of the following ethical issue. Evaluat-ing the work of a scientist is not an easy exercise and in many instances it is offensive to reduce the effort of many years in a number. When necessary, a scientist is better to be judged by a proper committee. Figure 1. Benign addition principle. It starts with a scientist who has A papers that have earned (a) total citations; average utility a/A. (blank column). Then B papers are added with b total citations (b<a); the scientist has now (A+B) papers that have earned (a+b) total citations (+ column). Next is the average utility of the previous case (- column); i.e. C=(A+B) and c=(a+b)/(A+B). It follows the repugnant conclusion as a possible state; Z>>C but z<<c (… column). ComparisonLet us assume that a scientist has Np papers which have earned Nc citations. We start with the benign addition principle [25] i.e. it is a good thing if this particular scientist in-creases the number of his/her papers even if the new papers receive few only citations. As a matter of fact even if the new papers do not gain any citations it is worth to be published, since they are already published. Therefore it is a good thing that this scientist adds more papers to his/her collec-tion in any case. We can then repeat the argument and ask for more papers and so on as before. It is possible then to arrive at a situation with a large number of papers but a small mean number of citations per paper (Fig.1). However citations are only the utility attributed to the papers [26]; it is wrong a scientist to cease writing worth published papers for they diminish the average utility of his/her col-lection of papers. Therefore nc either contradicts the mere addition ethical axiom or may end to a repugnance conclusion [27].       *  E-mail address: amitrop@teikav.edu.grISSN: 1791-2377 © 2009 Kavala Institute of Technology. All rights reserved. 69A. Ch. Mitropoulos /Journal of Engineering Science and Technology Review 2 (1) (2009) 68-70 Figure 2. The h-index. It starts with a scientist who has 1 paper that has earned (a) citations (column A). New papers are added to the scientist’s collection that has now 5 papers in total. By numbering the papers in a decreasing order of cita-tions that each one has received we reach to column A+. The h-index is defined at the point where the number of the numbered paper is equal to the number of citations that it has received; in the present example the 3rd paper has c=3 cita-tions and thus h=3 (column B-). The index underestimates the citations of the 1st and 2nd paper and ignores the citations of the 4th and 5th paper (column Z). Let  us  examine  now  the  h-index.  According  to  Hirsch, Nc=αh2; where α is a constant that ranges between 3 and 5. From the definition of h there must be a number of significant papers Np* such that Np*=h. Then Hirsch claims that the average util-ity of the total Np papers is the number of significant papers Np*. Although the measure avoids the repugnance conclusion, because any paper with citations less than h is ignored, it underestimates the utility of any paper with citations more than h (Fig.2). Appar-ently, h does not reflect the optimum point in terms of total utility principle; the optimum is the point where total utility is greatest. This would be the case of nc*=Nc*/Np*; where Nc* is the total number of citations of the Np* papers. But this is not a different index, at least structurally, from nc. It is always possible to avoid repugnance  conclusion  by  killing  insignificant  although  worth published papers. Consequently h can be distorted, although less easily than nc, more easily than Nc just by finding a subtle way to cite one’s own papers that are bubbling under [28]. DiscussionI argue that the problem of scientific excellence is not a problem of measure but a problem of categories. Nevertheless measures may help to categorise. For instance a scientist’s level may be defined as:    (1)where  ν  is  a  superscript,  not  a  power,  ν=0,1,2,3,4,5  and  L  is R=[rounddown] to the first digit of the quotient; N*={1,2,3,…}. Currently ν is highly unlikely to be >5; i.e. Nc>106. Now I will follow the example of Ref.[24] where a given scientist A has 88 papers with 2,288 citations or; Lν=23, nc=26, and h=29. None of these measures however say anything to us unless there is a pre-defined verbal description. Based on h, A must be a successful scientist if he or she serves science for about 30 years. Based on nc, A’s work must be of great impact although his/her papers have less than 187 citations each. Based on Lν, A is a low high class scientist (ν=3 but 2,000<Nc<3,000). In all 3 cases there is a vague representation of scientist A and judging him/her involves some kind of semi-principled decision-making [29]. The problem is fur-ther complicated due to the fact that a global measure refers to the inverse rather than to the direct procedure. For example the direct procedure would be to ask, what is the e.g. h-index of a success-ful scientist? The inverse is to linguistically describe what kind of scientist is someone who has e.g. h=15. Figure 3. The Lv classification. Evaluation of a scientist requires some kind of decision-making; only a number is not enough. The year’s line follows a Fibonacci sequence. In the present example it is assumed that the given scien-tist has reached to the mid (rank 6) of level 2 after about 16 years from his/her first publication. Presumably he or she moves from one level to the next a little faster than the following sequence: [8 years for level 0], [13 years for level 1], and [21 years for level 2]. A faster sequence would be (5, 8, 13) and a less fast would be (13, 21, 34). The decision-making line ends to three categories of verbal description: very, just, less. To my opinion the scientist under con-sideration is (just) influential and promising. Apparently all this is an arbitrary classification (just for the sake of discussion); but it is a moral one. Scientists are judged rather than numbered. In this light I suppose that Eq.1 allows more tolerance. To show this I will discuss the case of a scientist with an Lν=62. This scientist has at least a sum of 600 papers and citations; i.e. glo-bally, he or she is at the upper middle level. Again, if his/her first paper has been published 16 years ago, by sliding the years-bar (see Fig.3) it is possible to guess how fast this particular scien-tist develops the Fibonacci sequence. Next we have to define the linguistic content of our judgment. Now, this is a very subtle situ-ation because it may be influenced by secondary considerations. To help on this I suggest a three fold expression. For instance a scientist may be “very” influential, (“just”) influential, or “less” influential. Similarly, according to how fast he/she moves from one level to the next he or she may be very promising, promising, or less promising. On this ground the scientist of the given exam-ple is both promising and influential. ConclusionsFrom a scientist’s point of view is by far more difficult to write a paper than to cite another’s scientist paper. Moreover science 70A. Ch. Mitropoulos /Journal of Engineering Science and Technology Review 2 (1) (2009) 68-70relies on papers not on citations. It is therefore important in the evaluation of a scientist research output to assess the number of papers that he or she has contributed. On the other hand citations are little prizes to an author’s work from his/her peer colleagues that encourage him/her to continue writing papers. Let us now evaluate a scientist who has written 10 articles that have gained 0 citations. Indices nc and h are both 0; while Lν=11. Is this zeroing morally right? If it is, then ~200 citations that presumably were granted to other authors by this annihilated work should be with-drawn from their records. If it is not, then Lv is a better although an arbitrary measure that reflects a less influential scientist.1.  E. Garfield, Citation analysis as a tool in journal evaluation. Science 178,   471–479 (1972).2.  E. Garfield, Citation Indexes to Science: A new dimension in documenta-tion through association of ideas, Science 122, 108-111 (1955).3.  E. Garfield, The history and meaning of the journal impact factor, Journal of the American Medical Association 295, 90-93 (2006).4.  P. Jacsó, Testing the calculation of a realistic h-index in Google Scholar, Scopus, and Web of Science, Library Trends, 56, 784-815 (2008).5.  P. Jacsó, The pros and cons of computing the h-index using Web of Sci-ence, Online Information Review 32, 673-688 (2008).6.  L.I. Meho, and K. Yang, Impact of data sources on citation counts and rankings of LIS Faculty: Web of Science vs. Scopus and Google Scholar, Journal of the American Society for Information Science and Technology 58, 2105-2125 (2007).7.  J.E. Hirsch, An index to quantify an individual’s scientific research out-put, Proceedings of the National Academy of Sciences (USA) 102, 16569-16572 (2005).8.  B. Cronin and L. Meho, Using the h-index to rank influential information scientists, Journal of the American Society for Information Science and Technology 59, 1275-1278 (2006).9.  C. Oppenheim, Using the h-Index to rank influential British researchers in information science and librarianship, Journal of the American Society for Information Science and Technology 58, 297–301 (2007).10.  M. Schreiber, A case study of the Hirsch index for 26 non-prominent physicists, http://arxiv.org/abs/physics/0708.0120 (2007).11.  J. Imperial and A. Rodriguez-Navarro, Usefulness of Hirsch’s h-index to evaluate scientific research in Spain, Scientometrics 7, 271-282 (2007).12.  C.D. Kelly and M.D. Jennions, The h index and career assessment by numbers, Trends in Ecology and Evolution 21, 167–170 (2006).13.  J.K. Vanclay, On the robustness of the h-index. Journal of the American Society for Information Science and Technology 58, 1547-1550 (2007).14.  L. Bornmann, R. Mutz, H.D. Daniel, Are there better indices for evalua-tion purposes than the h-index? A comparison of nine different variants of the h index using data from biomedicine, Journal of the American Society for Information Science and Technology 59(5), 1-8. (2008).15.  R. Costas and M. Bordons, The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level, Journal of Informetrics 1, 193-203 (2007).16.  A. Sidiropoulos, D. Katsaros, and Y. Manolopoulos, Generalized h-index for disclosing latent facts in citation networks, Scientometrics 72, 253-280 (2006).17.  J. Panaretos and C. Malesios, Assessing scientific research performance and impact with single indices, MPRA Paper No. 12842, http://mpra.ub.uni-muenchen.de/12842 (2009).18.  R. Rousseau, New developments related to the Hirsch index, Science Fo-cus 1, 23-25 (2006).19.  W. Glänzel, On the opportunities and limitations of the h-index, Science Focus 1, 10-11 (2006).20.  Q.L. Burrell, On the h-index, the size of the Hirsch core and Jin’s A-index, Journal of Informetrics 1, 170-177 (2007).21.  R. Rousseau, The influence of missing publications on the hirsch index, Journal of Informetrics, 1, 2-7 (2007).22.  L. Egghe, Theory and practice of the g-index. Scientometrics 69, 131-152 (2006).23.  J.E. Hirsch, Does the h-index have a predictive power? http://arxiv.org/abs/physics/0708.0646 (2007).24.  S.  Lehmann, A.D.  Jackson  and  B.E.  Lautrup, A  quantitative  analysis of measures of quality in science, http://arxiv.org/abs/physics/0701311 (2007).25.  D. Parfit, Reasons and persons, Oxford, Clarendon (1984).26.  S.E. Cozzens, What do citations count? The rhetoric-first model, Sciento-metrics 15, 437-447 (1989).27.  M. Huemer, In defence of repugnance, Mind 117, 899-933 (2008). 28.  A. Purvis, The h index: playing the numbers game, Trends in Ecology and Evolution 21, 422 (2006).29.  A. Rayo, Vague representation, Mind 117, 329-373 (2008).References 

Is it more difficult to write or to cite a paper?

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals