Citation Statistics

This is a report about the use and misuse of citation data in the assessment of scientific research. The idea that research assessment must be done using ``simple and objective'' methods is increasingly prevalent today. The ``simple and objective'' methods are broadly interpreted as bibliometrics, that is, citation data and the statistics derived from them. There is a belief that citation statistics are inherently more accurate because they substitute simple numbers for complex judgments, and hence overcome the possible subjectivity of peer review. But this belief is unfounded.Comment: This paper commented in: [arXiv:0910.3532], [arXiv:0910.3537], [arXiv:0910.3543], [arXiv:0910.3546]. Rejoinder in [arXiv:0910.3548]. Published in at http://dx.doi.org/10.1214/09-STS285 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Factors Influencing Cities' Publishing Efficiency

Recently, a vast number of scientific publications have been produced in cities in emerging countries. It has long been observed that the publication output of Beijing has exceeded that of any other city in the world, including such leading centres of science as Boston, New York, London, Paris, and Tokyo. Researchers have suggested that, instead of focusing on cities' total publication output, the quality of the output in terms of the number of highly cited papers should be examined. However, in the period from 2014 to 2016, Beijing produced as many highly cited papers as Boston, London, or New York. In this paper, I propose another method to measure cities' publishing performance; I focus on cities' publishing efficiency (i.e., the ratio of highly cited articles to all articles produced in that city). First, I rank 554 cities based on their publishing efficiency, then I reveal some general factors influencing cities' publishing efficiency. The general factors examined in this paper are as follows: the linguistic environment, cities' economic development level, the location of excellent organisations, cities' international collaboration patterns, and the productivity of scientific disciplines

Searching for seeds to rest in libraries : European collecting habits towards Malay books and manuscripts in the nineteenth century

European scholars, colonial administrators, missionaries, bibliophiles and others were the main collectors of Malay books in the nineteenth century, both in manuscript or printed form. Among these persons were many well-known names in the field of Malay literature and culture like Raffles, Marsden, Crawfurd, Klinkert, van der Tuuk, von Dewall, Roorda, Favre, Maxwell, Overbeck, Wilkinson and Skeat, to name only a few. Their collections were often handed over to public libraries where they form an important part of the relevant Oriental or Southeast Asian manuscript collections. Therefore the knowledge of the intellectual culture of the Malay Peninsula and the Malay World in general depended very much on these manuscripts and printed books collected often by chance or in a rather unsystematic way. The collections reflect in a strong sense the interests of its administrative or philologist collectors: court histories, genealogies of aristocratic lineages, law collections (adat-istiadat as well as undangundang) or prose belles-lettres build a vast bulk of these collections, while Islamic religious texts and poetry forms popular in the 19th century (especially syair) are fairly underrepresented. Malay manuscripts and books located in religious institutions like mosques or pondok/pesantren schools have not been searched for; until today there are more or less no systematic studies of these collections. As in some statistics religious texts build about 20% of all existing Malay manuscripts, their neglect by Europeans scholars leads to a distorted view of the literary culture in the Malay language

Software tools for conducting bibliometric analysis in science: An up-to-date review

Bibliometrics has become an essential tool for assessing and analyzing the output of scientists, cooperation between universities, the effect of state-owned science funding on national research and development performance and educational efficiency, among other applications. Therefore, professionals and scientists need a range of theoretical and practical tools to measure experimental data. This review aims to provide an up-to-date review of the various tools available for conducting bibliometric and scientometric analyses, including the sources of data acquisition, performance analysis and visualization tools. The included tools were divided into three categories: general bibliometric and performance analysis, science mapping analysis, and libraries; a description of all of them is provided. A comparative analysis of the database sources support, pre-processing capabilities, analysis and visualization options were also provided in order to facilitate its understanding. Although there are numerous bibliometric databases to obtain data for bibliometric and scientometric analysis, they have been developed for a different purpose. The number of exportable records is between 500 and 50,000 and the coverage of the different science fields is unequal in each database. Concerning the analyzed tools, Bibliometrix contains the more extensive set of techniques and suitable for practitioners through Biblioshiny. VOSviewer has a fantastic visualization and is capable of loading and exporting information from many sources. SciMAT is the tool with a powerful pre-processing and export capability. In views of the variability of features, the users need to decide the desired analysis output and chose the option that better fits into their aims

A Relative Impact Ranking of Political Studies In Ireland

Against a background of the Irish government’s concerns with Key Performance Indicators (KPIs) and the British government’s wishes for a more quantitative Research Assessment Exercise (RAE), our study conducts a relative impact assessment of the study of politics, government, political science, and international relations in Ireland. Impact is measured as citations from the publications of permanent staff in eight Irish politics departments, based on data compiled in April 2008 from two leading academic indexes – ISI’s Web of Science and Scopus – as well as the now popular Google Scholar. We discuss some of the criticisms that naturally arise in a study of this nature. Then, following similar exercises in other disciplines (e.g. economics), we use the impact measures to compare and rank individual scholars as well as departments. We also explore the extent to which the choice of different indexes, and different measures, influences the results that we obtain. While there are differences, in particular between indexes based purely on articles and those that access books and other material, the results from the different indexes are strongly correlated.

Benchmarking some Portuguese S&T system research units: 2nd Edition

The increasing use of productivity and impact metrics for evaluation and comparison, not only of individual researchers but also of institutions, universities and even countries, has prompted the development of bibliometrics. Currently, metrics are becoming widely accepted as an easy and balanced way to assist the peer review and evaluation of scientists and/or research units, provided they have adequate precision and recall. This paper presents a benchmarking study of a selected list of representative Portuguese research units, based on a fairly complete set of parameters: bibliometric parameters, number of competitive projects and number of PhDs produced. The study aimed at collecting productivity and impact data from the selected research units in comparable conditions i.e., using objective metrics based on public information, retrievable on-line and/or from official sources and thus verifiable and repeatable. The study has thus focused on the activity of the 2003-06 period, where such data was available from the latest official evaluation. The main advantage of our study was the application of automatic tools, achieving relevant results at a reduced cost. Moreover, the results over the selected units suggest that this kind of analyses will be very useful to benchmark scientific productivity and impact, and assist peer review.Comment: 26 pages, 20 figures F. Couto, D. Faria, B. Tavares, P. Gon\c{c}alves, and P. Verissimo, Benchmarking some portuguese S\&T system research units: 2nd edition, DI/FCUL TR 13-03, Department of Informatics, University of Lisbon, February 201