Regularity in the research output of individual scientists: An empirical analysis by recent bibliometric tools

This paper proposes an empirical analysis of several scientists based on their time regularity, defined as the ability of generating an active and stable research output over time, in terms of both quantity/publications and impact/citations. In particular, we empirically analyse three recent bibliometric tools to perform qualitative/quantitative evaluations under the new perspective of regularity. These tools are respectively (1) the PY/CY diagram, (2) the publication/citation Ferrers diagram and triad indicators, and (3) a year-by-year comparison of the scientists' output (Borda's ranking). Results of the regularity analysis are then compared with those obtained under the classical perspective of overall production. The proposed evaluation tools can be applied to competitive examinations for research position/promotion, as complementary instruments to the commonly adopted bibliometric technique

Cognitive distances between evaluators and evaluees in research evaluation : a comparison between three informetric methods at the journal and subject category aggregation level

This article compares six informetric approaches to determine cognitive distances between the publications of panel members (PMs) and those of research groups in discipline-specific research evaluation. We used data collected in the framework of six completed research evaluations from the period 2009–2014 at the University of Antwerp as a test case. We distinguish between two levels of aggregation—Web of Science Subject Categories and journals—and three methods: while the barycenter method (2-dimensional) is based on global maps of science, the similarity-adapted publication vector (SAPV) method and weighted cosine similarity (WCS) method (both in higher dimensions) use a full similarity matrix. In total, this leads to six different approaches, all of which are based on the publication profile of research groups and PMs. We use Euclidean distances between barycenters and SAPVs, as well as values of WCS between PMs and research groups as indicators of cognitive distance. We systematically compare how these six approaches are related. The results show that the level of aggregation has minor influence on determining cognitive distances, but dimensionality (two versus a high number of dimensions) has a greater influence. The SAPV and WCS methods agree in most cases at both levels of aggregation on which PM has the closest cognitive distance to the group to be evaluated, whereas the barycenter approaches often differ. Comparing the results of the methods to the main assessor that was assigned to each research group, we find that the barycenter method usually scores better. However,the barycenter method is less discriminatory and suggests more potential evaluators, whereas SAPV and WCS are more precise

The long-term causal effects of winning an ERC grant

El objetivo de este documento es investigar los efectos causales a largo plazo de la concesión de una beca del Consejo Europeo de Investigación (ERC, por sus siglas en inglés) en la productividad de los investigadores, en la excelencia de su trabajo y en su capacidad para obtener nuevas becas hasta nueve años después de la concesión de la beca. Nuestro análisis se basa en datos sobre el universo de solicitantes de becas del ERC entre 2007 y 2023, así como en información acerca de su historial completo de publicaciones disponible en la base de datos Scopus. Para identificar el efecto causal, primero utilizamos la regla de asignación de becas basada en la clasificación de los solicitantes, y comparamos las variables de resultados de los solicitantes que obtienen una beca con los de los solicitantes que no la obtienen utilizando un diseño de regresión en discontinuidad (RDD, por sus siglas en inglés). En el marco de esta metodología, el análisis encuentra efectos estadísticamente significativos en la productividad de los investigadores y en la calidad de la investigación, lo que sugiere que la obtención de una beca del ERC no constituye una diferencia en términos de impacto científico para los investigadores en una posición cercana al umbral en la clasificación. Dado que los diseños RDD contribuyen a identificar un efecto local, también realizamos un análisis de diferencias en diferencias (DID, por sus siglas en inglés) utilizando la serie temporal de los indicadores bibliométricos disponibles, lo que nos permite estimar el efecto en una población más amplia de solicitantes que obtienen la beca y de solicitantes a los que se les deniega. En contraposición a los resultados obtenidos del RDD, las estimaciones del DID muestran que la obtención de una beca del ERC genera efectos positivos a largo plazo en la productividad científica, en el impacto y en la capacidad de atraer otras fuentes de financiación de la Unión Europea en las áreas de química, ciencias del universo y de la tierra, instituciones y comportamientos, estudios de la mente humana y medicina. Un análisis adicional de los efectos heterogéneos nos lleva a concluir que los resultados positivos del DID parecen estar determinados por los solicitantes que ocupan las primeras posiciones en la clasificación en estas áreas.This paper investigates the long-term causal effects of receiving an ERC grant on researcher productivity, excellence and the ability to obtain additional research funding up to nine years after grant assignment. We use data on the universe of ERC applicants between 2007 and 2013 and information on their complete publication histories from the Scopus database. For identification, we first exploit the assignment rule based on rankings, comparing the outcomes of the winning and non-winning applicants in a regression discontinuity design (RDD). We fail to find any statistically significant effect on research productivity and quality, which suggests that receiving an ERC grant does not make a difference in terms of scientific impact for researchers with a ranking position close to the threshold. Since RDDs help identify a local effect, we also conduct a difference-in-differences (DID) analysis using the time series of bibliometric indicators available, which allows us to estimate the effect on a wider population of winning and non-winning applicants. By contrast with the RDD results, DID estimates show that obtaining an ERC grant leads to positive long-term effects on scientific productivity, impact and the capacity to attract other EU funds in the fields of Chemistry, Universe and Earth Sciences, Institutions and Behaviours, Human Mind Studies and Medicine. Further analysis of heterogeneous effects leads us conclude that the positive results obtained with DID seem to be driven by the top-ranked applicants in these fields

Scholarship in Social Work: Imperfect Methods, Approximate Truths, and Emerging Challenges

The University Archives has determined that this item is of continuing value to OSU's history.Fraser, Mark W., Ph.D., Professor and Director of the Ph.D. Program, Graduate School of Social Work, University of Utah, Salt Lake City, UT - "Scholarship in Social Work: Imperfect Methods, Approximate Truths, and Emerging Challenges".The Ohio State University College of Social Wor

Co-authorship networks in Swiss political research

Co-authorship is an important indicator of scientific collaboration. Co-authorship networks are composed of sub-communities, and researchers can gain visibility by connecting these insulated subgroups. This article presents a comprehensive co-authorship network analysis of Swiss political science. Three levels are addressed: disciplinary cohesion and structure at large, communities, and the integrative capacity of individual researchers. The results suggest that collaboration exists across geographical and language borders even though different regions focus on complementary publication strategies. The subfield of public policy and administration has the highest integrative capacity. Co-authorship is a function of several factors, most importantly being in the same subfield. At the individual level, the analysis identifies researchers who belong to the “inner circle” of Swiss political science and who link different communities. In contrast to previous research, the analysis is based on the full set of publications of all political researchers employed in Switzerland in 2013, including past publications

Cognitive distances between evaluators and evaluees in research evaluation: A comparison between three Informetric methods at the journal and subject category aggregation level

This article compares six informetric approaches to determine cognitive distances between the publications of panel members (PMs) and those of research groups in discipline-specific research evaluation. We used data collected in the framework of six completed research evaluations from the period 2009–2014 at the University of Antwerp as a test case. We distinguish between two levels of aggregation—Web of Science Subject Categories and journals—and three methods: while the barycenter method (2-dimensional) is based on global maps of science, the similarity-adapted publication vector (SAPV) method and weighted cosine similarity (WCS) method (both in higher dimensions) use a full similarity matrix. In total, this leads to six different approaches, all of which are based on the publication profile of research groups and PMs. We use Euclidean distances between barycenters and SAPVs, as well as values of WCS between PMs and research groups as indicators of cognitive distance. We systematically compare how these six approaches are related. The results show that the level of aggregation has minor influence on determining cognitive distances, but dimensionality (two versus a high number of dimensions) has a greater influence. The SAPV and WCS methods agree in most cases at both levels of aggregation on which PM has the closest cognitive distance to the group to be evaluated, whereas the barycenter approaches often differ. Comparing the results of the methods to the main assessor that was assigned to each research group, we find that the barycenter method usually scores better. However, the barycenter method is less discriminatory and suggests more potential evaluators, whereas SAPV and WCS are more precise

Study on open science: The general state of the play in Open Science principles and practices at European life sciences institutes

Nowadays, open science is a hot topic on all levels and also is one of the priorities of the European Research Area. Components that are commonly associated with open science are open access, open data, open methodology, open source, open peer review, open science policies and citizen science. Open science may a great potential to connect and influence the practices of researchers, funding institutions and the public. In this paper, we evaluate the level of openness based on public surveys at four European life sciences institute

Meritocracia y política interna en las organizaciones públicas: el caso de la Academia Húngara de las Ciencias

Tesis inédita de la Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Departamento de Organización de Empresas, leída el 15-12-2015Depto. de Organización de EmpresasFac. de Ciencias Económicas y EmpresarialesTRUEunpu

Towards hierarchical affiliation resolution: framework, baselines, dataset

Author affiliations provide key information when attributing academic performance like publication counts. So far, such measures have been aggregated either manually or only to top-level institutions, such as universities. Supervised affiliation resolution requires a large number of annotated alignments between affiliation strings and known institutions, which are not readily available. We introduce the task of unsupervised hierarchical affiliation resolution, which assigns affiliations to institutions on all hierarchy levels (e.g. departments), discovering the institutions as well as their hierarchical ordering on the fly. From the corresponding requirements, we derive a simple conceptual framework based on the subset partial order that can be extended to account for the discrepancies evident in realistic affiliations from the Web of Science. We implement initial baselines and provide datasets and evaluation metrics for experimentation. Results show that mapping affiliations to known institutions and discovering lower-level institutions works well with simple baselines, whereas unsupervised top-level- and hierarchical resolution is more challenging. Our work provides structured guidance for further in-depth studies and improved methodology by identifying and discussing a number of observed difficulties and important challenges that future work needs to address