Online impact – surveying the websites most commonly cited in impact case studies

Reporting on their recent survey of websites cited in REF 2014 impact case studies, Kayvan Kousha, Mike Thelwall and Mahshid Abdoli, discuss which websites are most commonly used as supporting evidence for impact and how these vary across academic disciplines

Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis

Many journals post accepted articles online before they are formally published in an issue. Early citation impact evidence for these articles could be helpful for timely research evaluation and to identify potentially important articles that quickly attract many citations. This article investigates whether Microsoft Academic can help with this task. For over 65,000 Scopus in-press articles from 2016 and 2017 across 26 fields, Microsoft Academic found 2-5 times as many citations as Scopus, depending on year and field. From manual checks of 1,122 Microsoft Academic citations not found in Scopus, Microsoft Academic���s citation indexing was faster but not much wider than Scopus for journals. It achieved this by associating citations to preprints with their subsequent in-press versions and by extracting citations from in-press articles. In some fields its coverage of scholarly digital libraries, such as arXiv.org, was also an advantage. Thus, Microsoft Academic seems to be a more comprehensive automatic source of citation counts for in-press articles than Scopus

Which types of online evidence show the non-academic benefits of research? Websites cited in UK impact case studies

© 2021 The Authors. Published by MIT Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1162/qss_a_00145Whilst funders increasingly request evidence of the societal benefits of research, all academics in the UK must periodically provide this information to gain part of their block funding within the Research Excellence Framework (REF). The impact case studies produced in the UK are public and can therefore be used to gain insights into the types of sources used to justify societal impact claims. This study focuses on the URLs cited as evidence in the last public REF to help researchers and resource providers to understand what types can be used and the disciplinary differences in their uptake. Based on a new semi-automatic method to classify the URLs cited in impact case studies, the results show that there are a few key online types of source for most broad fields, but these sources differ substantially between subject areas. For example, news websites are more important in some fields than others, and YouTube is sometimes used for multimedia evidence in the arts and humanities. Knowledge of the common sources selected independently by thousands of researchers may help others to identify suitable sources for the complex task of evidencing societal impacts

Gender disparities in UK research publishing: Differences between fields, methods and topics

This is an accepted manuscript of an article published by EPI SCP in El Profesional de la Información on 21/07/2020, available online: https://doi.org/10.3145/epi.2020.jul.15 The accepted version of the publication may differ from the final published version.Gender disparities persist in UK research, with female minorities in most science, technology, engineering and mathematics (STEM) subjects but female majorities in others. The nature of the gender disparity differences between subjects needs to be understood if effective remedial actions are to be targeted at STEM subjects suffering from a lack of women, in contrast to other subjects without shortages. Evidence from the USA suggests that women engage more in people-related subjects, qualitative methods, veterinary science and cell biology and men engage more in thing-related subjects, power/control fields, patient-related research, abstraction and quantitative methods, except surveys. This article investigates gender disparity differences in UK first authorship for journal articles in nearly all of science split into 26 broad and 308 narrow Scopus fields. The results largely replicate the USA but suggest that more life science topics may be female-associated in the UK and patient-related research might not be male-associated. UK STEM gender parity initiatives might therefore emphasise people-oriented, and perhaps socially positive, aspects of currently masculine STEM topics and approaches (e.g., abstraction, mathematical quantitative methods), and promote female-friendly topics, methods and goals within male-dominated fields in addition to tacking implicit and explicit sexism and providing a supportive working environment.Published onlin

Which international co-authorships produce higher quality journal articles?

International collaboration is sometimes encouraged in the belief that it generates higher quality research or is more capable of addressing societal problems. Nevertheless, while there is evidence that the journal articles of international teams tend to be more cited than average, perhaps from increased international audiences, there is no science-wide direct academic evidence of a connection between international collaboration and research quality. This article empirically investigates the connection between international collaboration and research quality for the first time, with 148,977 UK-based journal articles with post publication expert review scores from the 2021 Research Excellence Framework (REF). Using an ordinal regression model controlling for collaboration, international partners increased the odds of higher quality scores in 27 out of 34 Units of Assessment (UoAs) and all Main Panels. The results therefore give the first large scale evidence of the fields in which international co-authorship for articles is usually apparently beneficial. At the country level, the results suggests that UK collaboration with other high research-expenditure economies generates higher quality research, even when the countries produce lower citation impact journal articles than the United Kingdom. Worryingly, collaborations with lower research-expenditure economies tend to be judged lower quality, possibly through misunderstanding Global South research goals.</p

Terms in journal articles associating with high quality: Can qualitative research be world-leading?

Purpose: Scholars often aim to conduct high quality research and their success is judged primarily by peer reviewers. Research quality is difficult for either group to identify, however, and misunderstandings can reduce the efficiency of the scientific enterprise. In response, we use a novel term association strategy to seek quantitative evidence of aspects of research that associate with high or low quality. Design/methodology/approach: We extracted the words and 2-5-word phrases most strongly associating with different quality scores in each of 34 Units of Assessment (UoAs) in the Research Excellence Framework (REF) 2021. We extracted the terms from 122,331 journal articles 2014-2020 with individual REF2021 quality scores. Findings: The terms associating with high- or low-quality scores vary between fields but relate to writing styles, methods, and topics. We show that the first-person writing style strongly associates with higher quality research in many areas because it is the norm for a set of large prestigious journals. We found methods and topics that associate with both high- and low-quality scores. Worryingly, terms associating with educational and qualitative research attract lower quality scores in multiple areas. REF experts may rarely give high scores to qualitative or educational research because the authors tend to be less competent, because it is harder to make world leading research with these themes, or because they do not value them. Originality: This is the first investigation of journal article terms associating with research quality