The student evaluation of teaching and the competence of students as evaluators

When the college student satisfaction survey is considered in the promotion and recognition of instructors, a usual complaint is related to the impact that biased ratings have on the arithmetic mean (used as a measure of teaching effectiveness). This is especially significant when the number of students responding to the survey is small. In this work a new methodology, considering student to student perceptions, is presented. Two different estimators of student rating credibility, based on centrality properties of the student social network, are proposed. This method is established on the idea that in the case of on-site higher education, students often know which others are competent in rating the teaching and learning process.Comment: 20 pages, 2 table

Do fixed citation windows affect the impact maturation rates of scientific journals?

ABSTRACTScientific fields employ distinct citation practices. As such, bibliometric indicators based on citations need to be standardized to allow comparisons between fields. This paper examines more than six hundred journals in eight JCR categories. Results indicate that impact maturation rates vary considerably from one category to another. The time elapsed until the citation distribution reaches a maximum oscillates between two and five years; hence the opening and closing of the citation window is crucial to the impact factor. Some journals are penalized by the two-year impact factor and benefited by the five-year impact factor, and the reverse situation was also found. Nonetheless, there are impact factors of variable citation windows that produce closer measures of central tendency

Evaluation of the higher education teaching activity considering the perception that students have of their peers

La opinión de los estudiantes en el proceso de evaluación del desempeño docente en la educación superior se recaba a partir de las encuestas de satisfacción. En este proceso, una de las quejas más habituales por parte del profesorado es que, en ocasiones, la valoración sesgada de algún estudiante puede llegar a tener una influencia determinante en el resultado final de la evaluación, especialmente cuando el número de encuestados es reducido. En este trabajo, se propone un indicador de satisfacción docente que pondera el resultado de las encuestas según la percepción que los estudiantes tienen los unos de los otros en su faceta de evaluadores de la actividad docente universitaria.A opinião dos estudantes no processo de avaliação do desempenho docente na educação superior se angaria a partir das enquetes de satisfação. Neste processo, uma das queixas mais habituais por parte do professorado é do que, em ocasiões, a valoração enviesada de algum estudante pode chegar a ter uma influência determinante no resultado final da avaliação, especialmente quando o número de interrogados é reduzido. Neste trabalho, propõe-se um indicador de satisfação docente que pondera o resultado das enquetes segundo a percepção que os estudantes têm os uns dos outros em sua faceta de avaliadores da atividade docente universitária.The opinion of the student in the higher education teaching evaluation process is carried out by the satisfaction surveys. In this process, one of the more usual teacher complaints is that, sometimes, the biased view of some students can take a decisive influence in the evaluation final result, especially when the number of respondents is reduced. In this work, an indicator of teaching satisfaction, which weighs the result of the surveys according to the perception that students have the one another as evaluators, is proposed

Evaluación de la trayectoria investigadora a través de la distribución de citas: una aplicación a los Nobel de economía

The citation distribution of a researcher reflects the impact of his production and determines the success of his scientific career. However, it is difficult to apply due to the bidimensional character. Some bibliometric indexes that try to synthesize with a numerical value the principal characteristics of this citation distribution have been proposed recently. In contrast with other bibliometric measures, the biases, that the distribution tails provoke, are eliminated by the h-index. On the other hand, some limitations to discriminate between researchers with different publication habits are presented. This index penalizes to selective researchers, distinguished by the number of citations instead of the number of publications, as compared to great producers. In this work, two original indexes are applied to the evaluation of Nobel Prize winners in Economics.La distribución de citas de un investigador refleja el impacto de su producción y determina el éxito de su carrera profesional. Sin embargo, resulta difícil su aplicación por el carácter bidimensional. En los últimos años, han aparecido algunos indicadores que tratan de sintetizar, en un valor numérico, las características principales de esta distribución. A diferencia de otros indicadores, el índice h elimina los sesgos provocados por las colas de la distribución. Sin embargo, este indicador presenta limitaciones a la hora de discriminar entre investigadores con diferentes hábitos de publicación, penalizando a aquellos más selectivos, que no destacan por el número de publicaciones pero sí por el alto número de citas recibidas, frente a los grandes productores. En este trabajo se aplican dos nuevos indicadores en la evaluación de los ganadores del Nobel de Economía

Influencing factors of Twitter mentions of scientific papers

Purpose: This paper explores some influencing factors of Twitter mentions of scientific research. The results can help to understand the relationships between various altmetrics. Design/methodology/approach: Data on research mentions in Altmetric and a multiple linear regression analysis are used. Findings: Among the variables analyzed, the number of mainstream news is the factor that most influences the number of mentions on Twitter, followed by the fact of dealing with a highly topical issue such as COVID-19. The influence is weaker in the case of expert recommendations and the consolidation of knowledge in the form of a review. The lowest influence corresponds to both the public policies through references in reports, and to citations in Wikipedia, while mentions in patent applications does not have a significant influence. Research limitations: A specific field was studied in a specific time frame. Studying other fields and/or different time periods might result in different findings. Practical implications: Governments increasingly push researchers toward activities with societal impact and this study can help understand how different factors affect social media attention. Originality/value: Understanding social media attention of research is essential when implementing societal impact indicators.Comment: 13 pages, 5 table

Societal and scientific impact of policy research: A large-scale empirical study of some explanatory factors using Altmetric and Overton

This study investigates how scientific research influences policymaking by analyzing citations of research articles in policy documents (policy impact) for nearly 125,000 articles across 434 public policy journals. We reveal distinct citation patterns between policymakers and other stakeholders like researchers, journalists, and the public. News and blog mentions, social media engagement, and open access publications (excluding fully open access) significantly increase the likelihood of a research article being cited in policy documents. Conversely, articles locked behind paywalls and those published under the full open access model (based on Altmetric data) have a lower chance of being policy-cited. Publication year and policy type show no significant influence. Our findings emphasize the crucial role of science communication channels like news media and social media in bridging the gap between research and policy. Interestingly, academic citations hold a weaker influence on policy citations compared to news mentions, suggesting a potential disconnect between how researchers reference research and how policymakers utilize it. This highlights the need for improved communication strategies to ensure research informs policy decisions more effectively. This study provides valuable insights for researchers, policymakers, and science communicators. Researchers can tailor their dissemination efforts to reach policymakers through media channels. Policymakers can leverage these findings to identify research with higher policy relevance. Science communicators can play a critical role in translating research for policymakers and fostering dialogue between the scientific and policymaking communities.Comment: 36 pages, 5 tables, 5 figure

Does society show differential attention to researchers based on gender and field?

While not all researchers prioritize social impact, it is undeniably a crucial aspect that adds significance to their work. The objective of this paper is to explore potential gender differences in the social attention paid to researchers and to examine their association with specific fields of study. To achieve this goal, the paper analyzes four dimensions of social influence and examines three measures of social attention to researchers. The dimensions are media influence (mentions in mainstream news), political influence (mentions in public policy reports), social media influence (mentions in Twitter), and educational influence (mentions in Wikipedia). The measures of social attention to researchers are: proportion of publications with social mentions (social attention orientation), mentions per publication (level of social attention), and mentions per mentioned publication (intensity of social attention). By analyzing the rankings of authors -- for the four dimensions with the three measures in the 22 research fields of the Web of Science database -- and by using Spearman correlation coefficients, we conclude that: 1) significant differences are observed between fields; 2) the dimensions capture different and independent aspects of the social impact. Finally, we use non-parametric means comparison tests to detect gender bias in social attention. We conclude that for most fields and dimensions with enough non-zero altmetrics data, gender differences in social attention are not predominant, but are still present and vary across fields.Comment: 23 pages, 5 figures, 7 table