New Opportunities for Repositories in the Age of Altmetrics

For institutional repositories, alternative metrics reflecting online activity present valuable indicators of interest in their holdings that can supplement traditional usage statistics. A variable mix of built-in metrics is available through popular repository platforms: Digital Commons, DSpace and EPrints. These may include download counts at the collection and/or item level, search terms, total and unique visitors, page views and social media and bookmarking metrics; additional data may be available with special plug-ins. Data provide different types of information valuable for repository managers, university administrators and authors. They can reflect both scholarly and popular impact, show readership, reflect an institution's output, justify tenure and promotion and indicate direction for collection management. Practical considerations for implementing altmetrics include service costs, technical support, platform integration and user interest. Altmetrics should not be used for author ranking or comparison, and altmetrics sources should be regularly reevaluated for relevance

Web indicators for research evaluation. Part 2: Social media metrics

This literature review assesses indicators derived from social media sources, including both general and academic sites. Such indicators have been termed altmetrics, influmetrics, social media metrics, or a type of webometric, and have recently been commercialised by a number of companies and employed by some publishers and university administrators. The social media metrics analysed here derive mainly from Twitter, Facebook, Google+, F1000, Mendeley, ResearchGate, and Academia.edu. They have the apparent potential to deliver fast, free indicators of the wider societal impact of research, or of different types of academic impacts, complementing academic impact indicators from traditional citation indexes. Although it is unwise to employ them in formal evaluations with stakeholders, due to their susceptibility to gaming and lack of real evidence that they reflect wider research impacts, they are useful for formative evaluations and to investigate science itself. Mendeley reader counts are particularly promising

How quickly do publications get read? The evolution of Mendeley reader counts for new articles

This is an accepted manuscript of an article published by Wiley-Blackwell in Journal of the Association for Information Science and Technology on 29/08/2017, available online: https://doi.org/10.1002/asi.23909 The accepted version of the publication may differ from the final published version.Within science, citation counts are widely used to estimate research impact but publication delays mean that they are not useful for recent research. This gap can be filled by Mendeley reader counts, which are valuable early impact indicators for academic articles because they appear before citations and correlate strongly with them. Nevertheless, it is not known how Mendeley readership counts accumulate within the year of publication, and so it is unclear how soon they can be used. In response, this paper reports a longitudinal weekly study of the Mendeley readers of articles in six library and information science journals from 2016. The results suggest that Mendeley readers accrue from when articles are first available online and continue to steadily build. For journals with large publication delays, articles can already have substantial numbers of readers by their publication date. Thus, Mendeley reader counts may even be useful as early impact indicators for articles before they have been officially published in a journal issue. If field normalised indicators are needed, then these can be generated when journal issues are published using the online first date

Mendeley reader counts for US computer science conference papers and journal articles

© 2020 The Authors. Published by MIT Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://direct.mit.edu/qss/article/1/1/347/15566/Mendeley-reader-counts-for-US-computer-scienceAlthough bibliometrics are normally applied to journal articles when used to support research evaluations, conference papers are at least as important in fast-moving computingrelated fields. It is therefore important to assess the relative advantages of citations and altmetrics for computing conference papers to make an informed decision about which, if any, to use. This paper compares Scopus citations with Mendeley reader counts for conference papers and journal articles that were published between 1996 and 2018 in 11 computing fields and had at least one US author. The data showed high correlations between Scopus citation counts and Mendeley reader counts in all fields and most years, but with few Mendeley readers for older conference papers and few Scopus citations for new conference papers and journal articles. The results therefore suggest that Mendeley reader counts have a substantial advantage over citation counts for recently-published conference papers due to their greater speed, but are unsuitable for older conference papers

IMPROVING COLLABORATIVE FILTERING RECOMMENDER BY USING MULTI-CRITERIA RATING AND IMPLICIT SOCIAL NETWORKS TO RECOMMEND RESEARCH PAPERS

Research paper recommender systems (RSs) aim to alleviate the information overload of researchers by suggesting relevant and useful papers. The collaborative filtering in the area of recommending research papers can benefit by using richer user feedback data through multi-criteria rating, and by integrating richer social network data into the recommender algorithm. Existing approaches using collaborative filtering or hybrid approaches typically allow only one rating criterion (overall liking) for users to evaluate papers. We conducted a qualitative study using focus group to explore the most important criteria for rating research papers that can be used to control the paper recommendation by enabling users to set the weight for each criterion. We investigated also the effect of using different rating criteria on the user interface design and how the user can control the weight of the criteria. We followed that by a quantitative study using a questionnaire to validate our findings from the focus group and to find if the chosen criteria are domain independent. Combining social network information with collaborative filtering recommendation algorithms has successfully reduced some of the drawbacks of collaborative filtering and increased the accuracy of recommendations. All existing recommendation approaches that combine social network information with collaborative filtering in this domain have used explicit social relations that are initiated by users (e.g. “friendship”, “following”). The results have shown that the recommendations produced using explicit social relations cannot compete with traditional collaborative filtering and suffer from the low user coverage. We argue that the available data in social bookmarking Web sites can be exploited to connect similar users using implicit social connections based on their bookmarking behavior. We explore the implicit social relations between users in social bookmarking Web sites (such as CiteULike and Mendeley), and propose three different implicit social networks to recommend relevant papers to users: readership, co-readership and tag-based implicit social networks. First, for each network, we tested the interest similarities of users who are connected using the proposed implicit social networks and compare them with the interest similarities using two explicit social networks: co-authorship and friendship. We found that the readership implicit social network connects users with more similarities than users who are connected using co-authorship and friendship explicit social networks. Then, we compare the recommendation using three different recommendation approaches and implicit social network alone with the recommendation using implicit and explicit social network. We found that fusing recommendation from implicit and explicit social networks can increase the prediction accuracy, and user coverage. The trade-off between the prediction accuracy and diversity was also studied with different social distances between users. The results showed that the diversity of the recommended list increases with the increase of social distance. To summarize, the main contributions of this dissertation to the area of research paper recommendation are two-fold. It is the first to explore the use of multi-criteria rating for research papers. Secondly, it proposes and evaluates a novel approach to improve collaborative filtering in both prediction accuracy (performance) and user coverage and diversity (nonperformance measures) in social bookmarking systems for sharing research papers, by defining and exploiting several implicit social networks from usage data that is widely available

Social media in scholarly communication : a review of the literature and empirical analysis of Twitter use by SSHRC doctoral award recipients

This report has been commissioned by the Social Sciences and Humanities Research Council (SSHRC) to analyze the role that social media currently plays in scholarly communication as well as to what extent metrics derived from social media activity related to scholarly content can be applied in an evaluation context. Scholarly communication has become more diverse and open with research being discussed, shared and evaluated online. Social media tools are increasingly being used in the research and scholarly communication context, as scholars connect on Facebook, LinkedIn and Twitter or specialized platforms such as ResearchGate, Academia.edu or Mendeley. Research is discussed on blogs or Twitter, while datasets, software code and presentations are shared on Dryad, Github, FigShare and similar websites for reproducibility and reuse. Literature is managed, annotated and shared with online tools such as Mendeley and Zotero, and peer review is starting to be more open and transparent. The changing landscape of scholarly communication has also brought about new possibilities regarding its evaluation. So-called altmetrics are based on scholarly social media activity and have been introduced to reflect scholarly output and impact beyond considering only peer-reviewed journal articles and citations within them to measure scientific success. This includes the measurement of more diverse types of scholarly work and various forms of impact including that on society. This report provides an overview of how various social media tools are used in the research context based on 1) an extensive review of the current literature as well as 2) an empirical analysis of the use of Twitter by the 2010 cohort of SSHRC Doctoral Award recipients was analyzed in depth. Twitter has been chosen as one of the most promising tools regarding interaction with the general public and scholarly communication beyond the scientific community. The report focuses on the opportunities and challenges of social media and derived metrics and attempts to provide SSHRC with information to develop guidelines regarding the use of social media by funded researchers as well support the informed used of social media metrics

Researchers\u27 Scientific performance in ResearchGate: The Case of a Technology University

With the advancement of technology and changes made in the scientific communication model, changes have been made in scientific evaluation methods. New technologies offer indicators that measure all the actions and interactions of scientists in the digital environment and create new aspects of scientific communication. This work has some purposes; first, mapping profile of scientific activities of faculty members of Sharif University of Technology (SUT) in “ResearchGate” (RG), second, intend to test the correlation h-index between the RG and Web of Science (WoS) and Scopus and Google Scholar (GS). Third, investigate SUT faculty members’ top h cited research RG in WoS, Scopus, and GS. Fourth, investigate Altmetric score of SUT faculty members’ top h cited research RG with Altmetric Explorer (AE). For this purpose, the SUT faculty members were searched in RG. Information was noted for those who were members in RG. Then, all their h index and the number of citations of top h cited research were extracted from WoS, Scopus and GS. Altmetric scores of Top h cited research of SUT faculty members was obtained by using AE. The degree of correlation between RG h index and WoS h index was higher than the h index of other citation databases. Also, the results indicate that there are errors in the calculation of the Almetric score by AE. Only 3% of the top h cited research of SUT faculty members had Altmetric score. While 95.72% of them had been read at least once in Mendeley and 78.7% of them had been cited in Mendeley

Traditional Citation Indexes and Alternative Metrics of Readership

The present study aimed to investigate the relationship between traditional citation indexes representing hot papers in the field of “Clinical Medicine” and their bookmarking and readership in “Mendeley software”. The citation counts of hot papers were extracted from Essential Science Indicators (ESI) and Web of Science (WoS). As an applied research adopting a descriptive-exploratory method, the present study used the Essential Science Indicators to retrieve hot articles published between 2014 and 2016, indexed in the category “Medical Sciences”. Each record was then searched in Mendeley to obtain the number of readership of the paper and the academic status of the users. The results showed a significant positive correlation between Mendeley readership and citation indexes in both ESI and WoS. Moreover, the most frequently-cited articles in both databases attracted more readers in Mendeley than lowly-cited publications and both hypotheses were confirmed. Moreover, the findings revealed that Mendeley users had assigned a total number of 3847 tags to the hot papers, with the tags ranging in frequency from zero to 38 for individual articles. Compared with author keywords and Plus, about 10 percent of users’ tags were either meaningless or repetitive. The value of present study shows that “Mendeley Sofware” with the possibility of tagging articles, can be used to create a searchable folksonomy of information and as a source of data in information retrieval studies, help professionals to manage their literatures and make their research life easier