Altmetrics for Digital Libraries: Concepts, Applications, Evaluation, and Recommendations

The volume of scientific literature is rapidly increasing, which has led to researchers becoming overloaded by the number of articles that they have available for reading and difficulties in estimating their quality and relevance (e.g., based on their research interests). Library portals, in these circumstances, are increasingly getting more relevant by using quality indicators that can help researchers during their research discovery process. Several evaluation methods (e.g., citations, Journal Impact Factor, and peer-reviews) have been used and suggested by library portals to help researchers filter out the relevant articles (e.g., articles that have received high citations) for their needs. However, in some cases, these methods have been criticized, and a number of weaknesses have been identified and discussed. For example, citations usually take a long time to appear, and some articles that are important can remain uncited. With the growing presence of social media today, new alternative indicators, known as “altmetrics,” have been encountered and proposed as complementary indicators to traditional measures (i.e., bibliometrics). They can help to identify the online attention received by articles, which might act as a further indicator for research assessment. One often mentioned advantage of these alternative indicators is, for example, that they appear much faster compared to citations. A large number of studies have explored altmetrics for different disciplines, but few studies have reported about altmetrics in the fields of Economics and Business Studies. Furthermore, no studies can be found so far that analyzed altmetrics within these disciplines with respect to libraries and information overload. Thus, this thesis explores opportunities for introducing altmetrics as new method for filtering relevant articles (in library portals) within the discipline of Economic and Business Studies literature. To achieve this objective, we have worked on four main aspects of investigating altmetrics and altmetrics data, respectively, of which the results can be used to fill the gap in this field of research. (1) We first highlight to what extent altmetric information from the two altmetric providers Mendeley and Altmetric.com is present within the journals of Economics and Business Studies. Based on the coverage, we demonstrate that altmetrics data are sparse in these disciplines, and when considering altmetrics data for real-world applications (e.g., in libraries), higher aggregation levels, such as journal level, can overcome their sparsity well. (2) We perform and discuss the correlations of citations on article and journal levels between different types and sources of altmetrics. We could show that Mendeley counts are positive and strongly correlated with citation counts on both article and journal levels, whereas other indicators such as Twitter counts and Altmetric Attention Score are significantly correlated only on journal level. With these correlations, we could suggest Mendeley counts for Economic and Business Studies journals/articles as an alternative indicator to citations. (3) In conjunction with the findings related to altmetrics in Economics and Business Studies journals, we discuss three use cases derived from three ZBW personas in terms of altmetrics. We investigate the use of altmetrics data for potential users with interests in new trends, social media platforms and journal rankings. (4) We investigated the behavior of economic researchers using a survey by exploring the usefulness of different altmetrics on journal level while they make decisions for selecting one article for reading. According to the user evaluation results, we demonstrate that altmetrics are not well known and understood by the economic community. However, this does not mean that these indicators are not helpful at all to economists. Instead, it brings forward the problem of how to introduce altmetrics to the economic community in the right way using which characteristics (e.g., as visible numbers attached at library records or behind the library’s relevance ranking system). Considering the aforementioned findings of this thesis, we can suggest several forms of presenting altmetric information in library portals, using EconBiz as the proof-of-concept, with the intention to assist both researchers and libraries to identify relevant journals or articles (e.g., highly mentioned online and recently published) for their need and to cope with the information overload

Altmetrics for Digital Libraries

The volume of scientific literature is increasing and researchers have difficulties in estimating their quality and relevance. Library portals, therefore, are getting more relevant by using quality indicators to help researchers during their research process. With the growing presence of social media, altmetrics have been proposed as complementary indicators to traditional measures. Altmetrics can help to identify online attention and can appear much faster than citations. This study explores altmetrics for filtering relevant articles (in library portals) within the discipline of Economic and Business Studies literature. Firstly, it highlights the altmetrics presence from Mendeley and Altmetric.com for the journals in the above-mentioned disciplines. It presents correlations between citation and altmetrics on article and journal level, suggesting Mendeley counts as an alternative indicator to citations. Afterward, it investigates the use of altmetrics data for potential users with interests in new trends, social media platforms, and journal rankings. Lastly, it explores the behavior of economic researchers using a survey by discovering the usefulness of different altmetrics. With the findings of this study, several forms of altmetrics in library portals are discussed, using EconBiz as the proof-of-concept, to assist both researchers and libraries to identify relevant journals or articles and to cope with the information overload

Scholarly Impact: a Bibliometric and Altmetric study of the Journal of Community Informatics

Demonstrating scholarly impact is a matter of growing importance. This paper reports on a bibliometric and altmetric analysis conducted on the Journal of Community Informatics (JOCI). Besides the bibliometric analysis the study also looked into JOCI article-level metrics by comparing usage metrics (article views), alternative metrics (Mendeley readership), and traditional citation metrics (Google Scholar citations). The main contribution is to provide more insight into the metrics that could influence the citation impact in Community Informatics research. Furthermore, the study used article-level metrics data to identify, compare and rank the most impactful papers published in JOCI over a 12-year period

Altmetrics and societal impact measurements: Match or mismatch? A literature review

Can alternative metrics (altmetrics) data be used to measure societal impact? We wrote this literature overview of empirical studies in order to find an answer to this question. The overview includes two parts. The first part, “societal impact measurements”, explains possible methods and problems in measuring the societal impact of research, case studies for societal impact measurement, societal impact considerations at funding organizations, and the societal problems that should be solved by science. The second part of the review, “altmetrics”, addresses a major question in research evaluation, which is whether altmetrics are proper indicators for measuring the societal impact of research. In the second part we explain the data sources used for altmetrics studies and the importance of field-normalized indicators for impact measurements. This review indicates that it should be relevant for impact measurements to be oriented towards pressing societal problems. Case studies in which societal impact of certain pieces of research is explained seem to provide a legitimate method for measuring societal impact. In the use of altmetrics, field-specific differences should be considered by applying field normalization (in cross-field comparisons). Altmetrics data such as social media counts might mainly reflect the public interest and discussion of scholarly works rather than their societal impact. Altmetrics (Twitter data) might be especially fruitfully employed for research evaluation purposes, if they are used in the context of network approaches. Conclusions based on altmetrics data in research evaluation should be drawn with caution

Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research

This is an accepted manuscript of an article published by Springer in Scientometrics on 09/09/2016, available online: https://doi.org/10.1007/s11192-016-2118-8 The accepted version of the publication may differ from the final published version.This study compares Spanish and UK research in eight subject fields using a range of bibliometric and social media indicators. For each field, lists of Spanish and UK journal articles published in the year 2012 and their citation counts were extracted from Scopus. The software Webometric Analyst was then used to extract a range of altmetrics for these articles, including patent citations, online presentation mentions, online course syllabus mentions, Wikipedia mentions and Mendeley reader counts and Altmetric.com was used to extract Twitter mentions. Results show that Mendeley is the altmetric source with the highest coverage, with 80% of sampled articles having one or more Mendeley readers, followed by Twitter (34%). The coverage of the remaining sources was lower than 3%. All of the indicators checked either have too little data or increase the overall difference between Spain and the UK and so none can be suggested as alternatives to reduce the bias against Spain in traditional citation indexes

Can Amazon.com reviews help to assess the wider impacts of books?

University of Wolverhampto

Social media in scholarly communication : a review of the literature and empirical analysis of Twitter use by SSHRC doctoral award recipients

This report has been commissioned by the Social Sciences and Humanities Research Council (SSHRC) to analyze the role that social media currently plays in scholarly communication as well as to what extent metrics derived from social media activity related to scholarly content can be applied in an evaluation context. Scholarly communication has become more diverse and open with research being discussed, shared and evaluated online. Social media tools are increasingly being used in the research and scholarly communication context, as scholars connect on Facebook, LinkedIn and Twitter or specialized platforms such as ResearchGate, Academia.edu or Mendeley. Research is discussed on blogs or Twitter, while datasets, software code and presentations are shared on Dryad, Github, FigShare and similar websites for reproducibility and reuse. Literature is managed, annotated and shared with online tools such as Mendeley and Zotero, and peer review is starting to be more open and transparent. The changing landscape of scholarly communication has also brought about new possibilities regarding its evaluation. So-called altmetrics are based on scholarly social media activity and have been introduced to reflect scholarly output and impact beyond considering only peer-reviewed journal articles and citations within them to measure scientific success. This includes the measurement of more diverse types of scholarly work and various forms of impact including that on society. This report provides an overview of how various social media tools are used in the research context based on 1) an extensive review of the current literature as well as 2) an empirical analysis of the use of Twitter by the 2010 cohort of SSHRC Doctoral Award recipients was analyzed in depth. Twitter has been chosen as one of the most promising tools regarding interaction with the general public and scholarly communication beyond the scientific community. The report focuses on the opportunities and challenges of social media and derived metrics and attempts to provide SSHRC with information to develop guidelines regarding the use of social media by funded researchers as well support the informed used of social media metrics