Research assessment by percentile-based double rank analysis

In the double rank analysis of research publications, the local rank position of a country or institution publication is expressed as a function of the world rank position. Excluding some highly or lowly cited publications, the double rank plot fits well with a power law, which can be explained because citations for local and world publications follow lognormal distributions. We report here that the distribution of the number of country or institution publications in world percentiles is a double rank distribution that can be fitted to a power law. Only the data points in high percentiles deviate from it when the local and world $\mu$ parameters of the lognormal distributions are very different. The likelihood of publishing very highly cited papers can be calculated from the power law that can be fitted either to the upper tail of the citation distribution or to the percentile-based double rank distribution. The great advantage of the latter method is that it has universal application, because it is based on all publications and not just on highly cited publications. Furthermore, this method extends the application of the well-established percentile approach to very low percentiles where breakthroughs are reported but paper counts cannot be performed.Comment: A pdf file containing text, 9 figures and 4 tables. Accepted in Journal of Informetric

Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all?

The phenomenon of rankings is intimately related with the government interest in fiscalizing the research outputs of universities. New forms of managerialism have been introduced into the higher education system, leading to an increasing interest from funding bodies in developing external evaluation tools to allocate funds. Rankings rely heavily on bibliometric indicators. But bibliometricians have been very critical with their use. Among other, they have pointed out the over-simplistic view rankings represent when analyzing the research output of universities, as they consider them as homogeneous ignoring disciplinary differences. Although many university rankings now include league tables by fields, reducing the complex framework of universities' research activity to a single dimension leads to poor judgment and decision making. This is partly because of the influence disciplinary specialization has on research evaluation. This chapter analyzes from a methodological perspective how rankings suppress disciplinary differences which are key factors to interpret correctly these rankings.Comment: Robinson-Garcia, N., Jim\'enez-Contreras, E. (2017). Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all? In: Downing, K., F.A. Ganotice (eds). World University Rankings and the Future of Higher Education. IGI Global, pp. 161-185. doi:10.4018/978-1-5225-0819-9.ch00

Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores

In over five years, Bornmann, Stefaner, de Moya Anegon, and Mutz (2014b) and Bornmann, Stefaner, de Moya Anegón, and Mutz (2014c, 2015) have published several releases of the www.excellencemapping.net tool revealing (clusters of) excellent institutions worldwide based on citation data. With the new release, a completely revised tool has been published. It is not only based on citation data (bibliometrics), but also Mendeley data (altmetrics). Thus, the institutional impact measurement of the tool has been expanded by focusing on additional status groups besides researchers such as students and librarians. Furthermore, the visualization of the data has been completely updated by improving the operability for the user and including new features such as institutional profile pages. In this paper, we describe the datasets for the current excellencemapping.net tool and the indicators applied. Furthermore, the underlying statistics for the tool and the use of the web application are explained

A tudománymetria elmélete a gyakorlatban: a tudományklasszifikáció problematikája

A tudománymetria népszerűsége mögött az informatikai és módszertani háttér bővülése, valamint a kutatásértékelési és -finanszírozási gyakorlat által támasztott bővülő igények állnak. A gyors fejlődés azonban gyakran vezet ahhoz, hogy a kutatásértékelési indikátorokat – a számszerűsíthetőség „bűvöletében” – nem kellő körültekintéssel használják, és az általuk kapott eredményeket abszolutizált módon értelmezik. A tudománymetriával professzionálisan foglalkozó szakmai kör erre vonatkozóan már többször (például a 2015-ös Leideni Kiáltványban) megfogalmazta aggályait. Jelen dolgozat két esettanulmányon keresztül igazolja egy, a témához kapcsolódó, méltánytalanul elhanyagolt módszertani probléma, a tudományterületi besorolás alapvető szerepét a kutatói teljesítmény mérésében. Példaként a hazai gyakorlatban központi jelentőségűvé vált Magyar Tudományos Művek Tára kutatásértékelési célú alkalmazását és az ún. Frascati-rendszer következetes használatának egy módját mutatja be