How reliable are unsupervised author disambiguation algorithms in the assessment of research organization performance?

The paper examines extent of bias in the performance rankings of research organisations when the assessments are based on unsupervised author-name disambiguation algorithms. It compares the outcomes of a research performance evaluation exercise of Italian universities using the unsupervised approach by Caron and van Eck (2014) for derivation of the universities' research staff, with those of a benchmark using the supervised algorithm of D'Angelo, Giuffrida, and Abramo (2011), which avails of input data. The methodology developed could be replicated for comparative analyses in other frameworks of national or international interest, meaning that practitioners would have a precise measure of the extent of distortions inherent in any evaluation exercises using unsupervised algorithms. This could in turn be useful in informing policy-makers' decisions on whether to invest in building national research staff databases, instead of settling for the unsupervised approaches with their measurement biases

Indicadores bibliométricos para el análisis de la actividad de una institución multidisciplinar: el CSIC

An overview is provided of CSIC’s research performance in the context of Spain, through a study of its scholarly production in the Web of Science database, complemented with ICYT and ISOC, during the period 2004-2009. The eight scientific and technical areas in which CSIC’s centers are organised differ as to their national or international research orientation, their basic or applied nature, the degree of their collaboration and the size of their research teams; all of which influences each area’s publication and citation practices as well as its WoS-based productivity. The specific features of the different areas must be thoroughly understood in order to expound on and interpret properly the results of studies dealing with research evaluation.Este artículo ofrece una visión general de la actividad investigadora del CSIC en el contexto de España a través del estudio de su producción científica en la base de datos Web of Science, complementada con ICYT e ISOC, durante el período 2004-2009. Las ocho áreas científico-técnicas en las que se organizan los centros del CSIC difieren en la orientación nacional o internacional de su investigación, su carácter básico o aplicado, la incidencia de la colaboración, y el tamaño de los grupos de investigación; todo lo cual influye sobre las prácticas de publicación y citación imperantes en cada área, y sobre su productividad derivada de WoS. Se señala la importancia de conocer las especificidades de las distintas áreas para plantear e interpretar adecuadamente los resultados de los estudios de evaluación de la actividad científica

White Paper: Measuring Research Outputs Through Bibliometrics

The suggested citation for this white paper is: University of Waterloo Working Group on Bibliometrics, Winter 2016. White Paper: Measuring Research Outputs through Bibliometrics, Waterloo, Ontario: University of Waterloo.This White Paper provides a high-level review of issues relevant to understanding bibliometrics, and practical recommendations for how to appropriately use these measures. This is not a policy paper; instead, it defines and summarizes evidence that addresses appropriate use of bibliometric analysis at the University of Waterloo. Issues identified and recommendations will generally apply to other academic institutions. Understanding the types of bibliometric measures and their limitations makes it possible to identify both appropriate uses and crucial limitations of bibliometric analysis. Recommendations offered at the end of this paper provide a range of opportunities for how researchers and administrators at Waterloo and beyond can integrate bibliometric analysis into their practice

A national-scale cross-time analysis of university research performance

Research policies in the more developed nations are ever more oriented towards the introduction of productivity incentives and competition mechanisms intended to increase efficiency in research institutions. Assessments of the effects of these policy interventions on public research activity often neglect the normal, inherent variation in the performance of research institutions over time. In this work, we propose a cross-time bibliometric analysis of research performance by all Italian universities in two consecutive periods (2001-2003 and 2004-2008) not affected by national policy interventions. Findings show that productivity and impact increased at the level of individual scientists. At the level of university, significant variation in the rank was observed