Social media metrics for new research evaluation

This chapter approaches, both from a theoretical and practical perspective, the most important principles and conceptual frameworks that can be considered in the application of social media metrics for scientific evaluation. We propose conceptually valid uses for social media metrics in research evaluation. The chapter discusses frameworks and uses of these metrics as well as principles and recommendations for the consideration and application of current (and potentially new) metrics in research evaluation.Comment: Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Springe

Analysis of Emerging Reputation and Funding Mechanisms in the Context of Open Science 2.0

This report covers the outcomes of two studies funded by JRC IPTS to explore emerging drivers for Open Science 2.0. In general, Open Science 2.0 is associated with themes such as open access to scientific outputs, open data, citizen science and open peer evaluation systems. This study, however, focused on less explored themes, namely on alternative funding mechanisms for scientific research and on emerging reputation mechanisms for scholars resulting from Web 2.0 platforms and applications. It has been demonstrated that both are providing significant new opportunities for researchers to disseminate, share, explore and collaborate with other researchers, but it remains to be seen whether they will be able to bring about more disruptive change in how science and research systems function in the future. They could well do so, especially if related changes being considered by the European Commission on ‘Science 2.0: Science in Transition’ are taken into account.JRC.J.3-Information Societ

Scholarly use of social media and altmetrics : a review of the literature

Social media has become integrated into the fabric of the scholarly communication system in fundamental ways: principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics—that is, research indicators based on social media activity. This review provides an extensive account of the state-of-the art in both scholarly use of social media and altmetrics. The review consists of two main parts: the first examines the use of social media in academia, examining the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations, and differences according to platform. The review ends with a critical discussion of the implications of this transformation in the scholarly communication system

Seeking Impact and Visibility: Scholarly Communication in Southern Africa

The Scholarly Communication in Africa Programme (SCAP) was a three-year research and implementation initiative that took place between March 2010 and August 2013. Hosted by the University of Cape Town, the programme engaged the Universities of Botswana, Namibia and Mauritius in a process aimed at better understanding the dynamics around scholarly communication in the Southern African higher education environment and advancing the open access agenda for the purpose of increasing the visibility of African research. This work was made possible by a grant from the Canadian International Development Research Center (IDRC). This report synthesizes research and findings from the four institutional case studies conducted at the Universities of Botswana, Cape Town, Mauritius and Namibia. It provides an overview the scholarly communication activity systems at work in these four Southern African universities

Social media metrics for new research evaluation

Merit, Expertise and Measuremen

Tweet coupling: a social media methodology for clustering scientific publications

This is an accepted manuscript of an article published by Springer in Scientometrics on 18/05/2020, available online: https://doi.org/10.1007/s11192-020-03499-1 The accepted version of the publication may differ from the final published version.© 2020, Akadémiai Kiadó, Budapest, Hungary. We argue that classic citation-based scientific document clustering approaches, like co-citation or Bibliographic Coupling, lack to leverage the social-usage of the scientific literature originate through online information dissemination platforms, such as Twitter. In this paper, we present the methodology Tweet Coupling, which measures the similarity between two or more scientific documents if one or more Twitter users mention them in the tweet(s). We evaluate our proposal on an altmetric dataset, which consists of 3081 scientific documents and 8299 unique Twitter users. By employing the clustering approaches of Bibliographic Coupling and Tweet Coupling, we find the relationship between the bibliographic and tweet coupled scientific documents. Further, using VOSviewer, we empirically show that Tweet Coupling appears to be a better clustering methodology to generate cohesive clusters since it groups similar documents from the subfields of the selected field, in contrast to the Bibliographic Coupling approach that groups cross-disciplinary documents in the same cluster.The authors (Saeed-Ul Hassan & Mudassir Shabbir) were funded by the CIPL (National Center in Big Data and Cloud Computing (NCBC) grant, received from the Planning Commission of Pakistan, through Higher Education Commission (HEC) of Pakistan. This work was partially supported by the Spanish Ministry of Science and Technology under the projects TIN2017-89517-P and TIN2017-83445-P. Eugenio Martínez Cámara was supported by the Spanish Government Programme Juan de la Cierva Incorporación (IJC2018-036092-I).Published versio

Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplinary Analysis Using Altmetrics

A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy.The field of ‘altmetrics’ is concerned with alternative metrics for the impact of research publications using social web data. Empirical studies are needed, however, to assess the validity of altmetrics from different perspectives. This thesis partly fills this gap by exploring the suitability and reliability of two altmetrics resources: Mendeley, a social reference manager website, and Faculty of F1000 (F1000), a post- publishing peer review platform. This thesis explores the correlations between the new metrics and citations at the level of articles for several disciplines and investigates the contexts in which the new metrics can be useful for research evaluation across different fields. Low and medium correlations were found between Mendeley readership counts and citations for Social Sciences, Humanities, Medicine, Physics, Chemistry and Engineering articles from the Web of Science (WoS), suggesting that Mendeley data may reflect different aspects of research impact. A comparison between information flows based on Mendeley bookmarking data and cross-disciplinary citation analysis for social sciences and humanities disciplines revealed substantial similarities and some differences. This suggests that Mendeley readership data could be used to help identify knowledge transfer between scientific disciplines, especially for people that read but do not author articles, as well as providing evidence of impact at an earlier stage than is possible with citation counts. The majority of Mendeley readers for Clinical Medicine, Engineering and Technology, Social Science, Physics and Chemistry papers were PhD students and postdocs. The highest correlations between citations and Mendeley readership counts were for types of Mendeley users that often authored academic papers, suggesting that academics bookmark papers in Mendeley for reasons related to scientific publishing. In order to identify the extent to which Mendeley bookmarking counts reflect readership and to establish the motivations for bookmarking scientific papers in Mendeley, a large-scale survey found that 83% of Mendeley users read more than half of the papers in their personal libraries. The main reasons for bookmarking papers were citing in future publications, using in professional activities, citing in a thesis, and using in teaching and assignments. Thus, Mendeley bookmarking counts can potentially indicate the readership impact of research papers that have educational value for non-author users inside academia or the impact of research papers on practice for readers outside academia. This thesis also examines the relationship between article types (i.e., “New Finding”, “Confirmation”, “Clinical Trial”, “Technical Advance”, “Changes to Clinical Practice”, “Review”, “Refutation”, “Novel Drug Target”), citation counts and F1000 article factors (FFa). In seven out of nine cases, there were no significant differences between article types in terms of rankings based on citation counts and the F1000 Article Factor (FFa) scores. Nevertheless, citation counts and FFa scores were significantly different for articles tagged: “New finding” or “Changes to Clinical Practice”. This means that F1000 could be used in research evaluation exercises when the importance of practical findings needs to be recognised. Furthermore, since the majority of the studied articles were reviewed in their year of publication, F1000 could also be useful for quick evaluations

Scholarship in abundance: Influence, engagement, and attention in scholarly networks

In an era of knowledge abundance, scholars have the capacity to distribute and share ideas and artifacts via digital networks, yet networked scholarly engagement often remains unrecognized within institutional spheres of influence. The purpose of this dissertation study is to explore the meanings constructed and enacted within the networked practices of 13 scholars actively engaged in both institutional and networked participatory scholarship. Using ethnographic methods including participant observation, interviews, and document analysis, the study investigates networks as sites of scholarship, with the intent of furthering institutional academia’s understanding of networked practices. The three papers that make up the dissertation each articulate a specific thread of intersection between institutional and networked scholarship: the first focuses on what counts as academic influence within networked circles, the second on networks’ terms of value and reward, and the third on the relationships between attention, care, and vulnerability in scholarly networks. Together, the papers conclude that networked scholarly practices of engagement align broadly with those of academia, yet enable and demand scholars’ individual cultivation of influence, visibility, and audiences. Thus networked scholarship rewards connection, collaboration, and curation between individuals rather than roles or institutions, fostering cross-disciplinary and public engagement and a bridging of the personal/professional divide. The study contributes to knowledge by situating networked scholarly practices within the scholarly tradition, while articulating the terms on which knowledge abundance and networked practices open up new spheres of opportunity and vulnerability for scholars

Which health and biomedical topics generate the most Facebook interest and the strongest citation relationships?

This is an accepted manuscript of an article published by Elsevier in Information Processing and Management on 26/02/2020, available online: https://doi.org/10.1016/j.ipm.2020.102230 The accepted version of the publication may differ from the final published version.Although more than a million academic papers have been posted on Facebook, there is little detailed research about which fields or cross-field issues are involved and whether there are field or public interest relationships between Facebook mentions and future citations. In response, we identified health and biomedical scientific papers mentioned on Facebook and assigned subjects to them using the MeSH and Science Metrix journal classification schema. Multistage adaptive LASSO and unpenalized least-squares regressions were used to model Facebook mentions by fields and MeSH terms. The fields Science and Technology, General and Internal Medicine, Complementary and Alternative Medicine, and Sport Sciences produced higher Facebook mention counts than average. However, no MeSH cross-field issue differences were found in the rate of attracting Facebook mentions. The relationship between Facebook mentions and citations varies between both fields and MeSH cross-field issues. General and Internal Medicine, Cardiovascular System and Hematology and Developmental Biology have strongest correlations between Facebook mentions and citations, probably due to high citation rates and high Facebook visibility in these areas

E-visibility of environmental sciences researchers at the University of South Africa

Abstract : Research e-visibility in theory enables a researcher to establish and maintain a digital research portfolio utilising various research e-profiles on a number of research online communities and platforms. E-visibility embodies the online presence of the researcher and their research, researcher’s discoverability via research e-profiles and the accessibility of research output on online research communities. The rationale for this study has its foundation in the premise that enhancing the e-visibility of a researcher will increase the research and societal impact of the researcher. The development of an e-visibility strategy for the School of Environmental Sciences (SES) at the University of South Africa (Unisa) would be instrumental in enhancing the e-visibility of the researchers. This study aims at establishing guidelines for the development of an e-visibility strategy for SES researchers at Unisa as part of research support via the Library services. Altmetric and bibliometric data of the SES researchers, were collected during the 2-year period (December 2014 and December 2017) and e-visibility surveys were conducted at the beginning of the study (December 2014) and at the end of the study (April 2017) as part of a longitudinal e-visibility study. The data was analysed using statistical methods to ascertain: 1) the SES researchers e-visibility status, 2) the SES researchers’ perceptions about e-visibility, 3) the altmetric-bibliometric correlations (relationships) from the altmetrics sourced from the academic social networking tools and the bibliometrics derived from the citation resources, and 4) identifying e-visibility practices and actions increasing research and societal impact. The results reflected a total increase in online presence, discoverability, and accessibility therefore indicating an overall increase in the actual and perceived e-visibility of the SES researchers. The survey conducted at the end of the study, found that 73% of the SES researchers indicating that their e-visibility increased with online presence being enhanced, 69% were more discoverable and 76% of their research output was more accessible after applying what they learnt during the e-visibility awareness and training...Ph.D. (Information Management