Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all?

The phenomenon of rankings is intimately related with the government interest in fiscalizing the research outputs of universities. New forms of managerialism have been introduced into the higher education system, leading to an increasing interest from funding bodies in developing external evaluation tools to allocate funds. Rankings rely heavily on bibliometric indicators. But bibliometricians have been very critical with their use. Among other, they have pointed out the over-simplistic view rankings represent when analyzing the research output of universities, as they consider them as homogeneous ignoring disciplinary differences. Although many university rankings now include league tables by fields, reducing the complex framework of universities' research activity to a single dimension leads to poor judgment and decision making. This is partly because of the influence disciplinary specialization has on research evaluation. This chapter analyzes from a methodological perspective how rankings suppress disciplinary differences which are key factors to interpret correctly these rankings.Comment: Robinson-Garcia, N., Jim\'enez-Contreras, E. (2017). Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all? In: Downing, K., F.A. Ganotice (eds). World University Rankings and the Future of Higher Education. IGI Global, pp. 161-185. doi:10.4018/978-1-5225-0819-9.ch00

Six indicators in search of a meaning

This is a kind of conceptual document, and at the same time a reflection, but also something more, an act of provocation in the form of a piece of theater that shows a reality, something dramatic and comical at the same time, as is the panorama ofour research. It also includes a good portion of humor, but the kind of humor the great writer and thinker Luigi Pirandello defined as „the one that does not provoke the immediate laughter, but only pulls a smile behind which hides the bitter taste of the absurd or the unbearable lightness of our existence.“ This piece of theater, short and concise so as not to bore anyone, narrates the efforts that are required daily, as well as the situations created, to carry out the task –which is perhaps a utopia- of trying to measure something that is incommensurable, as is science on the part of those who are dedicated to its evaluation or to the study of its development. In this comedy, all actors involved, both protagonists and the indicators themselves, have the opportunity to present their hopes and fears, their merits and their complaints, their recommendations and warnings. They narrate the search for a path in a jungle in which the world of information has been converted, that is practicable, augurs a promising future for science, and allows us a more just interpretation

Tracking the performance of an R&D programme in the biomedical sciences

Pre-refereed version of manuscript accepted for publication in Research EvaluatioN: arXiv:1602.04049This article aims at offering an evaluation framework of an R&D programme in the biomedical sciences. It showcases the Spanish Biomedical Research Networking Centres (CIBER) initiative as an example of the effect of research policy management on performance. For this, it focuses on three specific aspects: its role on the national research output in the biomedical sciences, its effect on promoting translational research through internal collaboration between research groups, and the perception of researchers on the programme as defined by their inclusion of their CIBER centres in the address field. Research output derived from this programme represents around 25% of the country’s publications in the biomedical fields. After analysing a 7-year period, the programme has enhanced collaborations between its members, but they do not seem to be sufficiently strong. 54.5% of the publications mentioned this programme in their address; however, an increase in the share of papers mention is observed 2 years after it was launched. We suggest that by finding the point at which the share of mentions stabilizes may be a good strategy to identify the complete fulfilment of these types of R&D policies.This work was supported by the Spanish Fondo de Investigacion Sanitaria (FIS) [PI10/01122].Peer reviewe

Productividad e impacto de los investigadores españoles: umbrales de referencia por áreas científicas

Reference thresholds for the scientifi c production and impact of internationally visible Spanish research within the areas defi ned by the Spanish National Agency for Evaluation and Prospective (ANEP) are presented. These percentile reference tables are constructed from the population of researchers who applied for a project within Spain’s National R & D Plan 2007 (n = 3.356) and are to serve as benchmarks, permitting comparisons between researchers’ bibliometric behavior and mean performance in their respective scientifi c disciplines. Data relating to mean production, impact and visibility for each ANEP area are also presented. The internationalization of these areas between 2000 and 2006 is discussed, with special emphasis on the Social Sciences. Finally, we suggest funding agencies and research institutions use these reference thresholds as assessment tools in their selection processes.Se presentan umbrales de referencia de producción e impacto científi co de la investigación española con visibilidad internacional para las áreas defi nidas por la Agencia Nacional de Evaluación y Prospectiva (ANEP) en sus convocatorias. Tomando como población los solicitantes de proyectos del Plan Nacional de I + D 2007 (n = 3.356) se construyen tablas de referencia por percentiles que funcionan a modo de benchmarks, permitiendo efectuar comparaciones entre el comportamiento bibliométrico de un investigador y los registros de referencia en su área científi ca. Igualmente se ofrecen los datos de producción, impacto y visibilidad promedios para las áreas ANEP, y se discute el proceso de internacionalización de dichas áreas en el período 2000-2006

Trends in Science Mapping: The co-use of scientific literature as evidence of the interests of researchers

This is a preprint version of a paper accepted to be published in El Profesional de la Información – ISSN 1386-6710 Special number on information visualization - May 2014This paper explores de possibility of constructing science maps based on the co-use of the staff of an academic institution make of scientific literature. For this, we define co-use as the co-occurrence of scientific information requests by a given user in platform of scientific journals. We use request data from the University of Navarre to the ScienceDirect platform in 2012 in order to analyze the potential of such methodological approach. We conclude by emphasizing the viability of such methodology when exploring the research interests of an academic institution along with the relations between different disciplines.Nicolás Robinson-García is currently supported by a FPU grant from the Spanish Ministerio de Economía y Competitividad

How many citations are there in the Data Citation Index?

19th International Conference on Science and Technology Indicators (STI), Leiden (The Netherlands) 3-5 september 2014.Descriptive analysis on the citation distribution of the Thomson Reuters' Data Citation Index by publication type and four broad areas: Science, Engineering & Technology, Humanities & Arts and Social Sciences.Nicolas Robinson-García is currently supported by a FPU Grant of the Spanish Ministerio de Economía y Competitividad