A multi-dimensional analysis of usage counts, Mendeley readership, and citations for journal and conference papers

This study analyzed 16,799 journal papers and 98,773 conference papers published by IEEE Xplore in 2016 to investigate the relationships among usage counts, Mendeley readership, and citations through descriptive, regression, and mediation analyses. Differences in the relationship among these metrics between journal and conference papers are also studied. Results showed that there is no significant difference between journal and conference papers in the distribution patterns and accumulation rates of the three metrics. However, the correlation coefficients of the interrelationships between the three metrics were lower in conference papers compared to journal papers. Secondly, funding, international collaboration, and open access are positively associated with all three metrics, except for the case of funding on the usage metrics of conference papers. Furthermore, early Mendeley readership is a better predictor of citations than early usage counts and performs better for journal papers. Finally, we reveal that early Mendeley readership partially mediates between early usage counts and citation counts in the journal and conference papers. The main difference is that conference papers rely more on the direct effect of early usage counts on citations. This study contributes to expanding the existing knowledge on the relationships among usage counts, Mendeley readership, and citations in journal and conference papers, providing new insights into the relationship between the three metrics through mediation analysis.Comment: 23 pages, 7 figure

Issues in the Interpretation of “Altmetrics” Digital Traces: A Review

Researchers leave traces of their behavior during many stages of their research process. Parts of this process were formerly invisible. With scholarship moving online, we can now access various types of altmetrics digital traces such as reading, organizing, sharing, and discussing scientific papers, thus develop a more holistic story about researchers and their work. However, a lack of in-depth interpretation of altmetrics digital traces is observed. Therefore, this paper focuses on reviewing some of the existing altmetrics research, with a particular emphasis on the issues that need to be taken into consideration in the interpretation of altmetrics digital traces. Taking a preliminary step toward a guideline for more in-depth analysis of digital traces of scholarly acts, this review aims to bring attention to these issues to avoid misuse of altmetrics indicators

Studying the accumulation velocity of altmetric data tracked by Altmetric.com

This paper investigates the data accumulation velocity of 12 Altmetric.com data sources. DOI created date recorded by Crossref and altmetric event posted date tracked by Altmetric.com are combined to reflect the altmetric data accumulation patterns over time and to compare the data accumulation velocity of various data sources through three proposed indicators, including Velocity Index, altmetric half-life, and altmetric time delay. Results show that altmetric data sources exhibit different data accumulation velocity. Some altmetric data sources have data accumulated very fast within the first few days after publication, such as Reddit, Twitter, News, Facebook, Google+, and Blogs. On the opposite spectrum, research outputs are at relatively slow pace in accruing data on some data sources, like Policy documents, Peer review, Q&A, Wikipedia, Video, and F1000Prime. Most altmetric data sources' velocity degree also changes by document types, subject fields, and research topics. The type Review is slower in receiving altmetric mentions than Article, while Editorial Material and Letter are typically faster. In general, most altmetric data sources show higher velocity values in the fields of Physical Sciences and Engineering and Life and Earth Sciences. Within each field, there also exist some research topics that attract social attention faster than others.Merit, Expertise and Measuremen

Web indicators for research evaluation. Part 1: Citations and links to academic articles from the Web

The extensive use of the web by many sectors of society has created the potential for new wider impact indicators. This article reviews research about Google Scholar and Google Patents, both of which can be used as sources of impact indicators for academic articles. It also briefly reviews methods to extract types of links and citations from the web as a whole, although the indicators that these generate are now probably too broad and too dominated by automatically generated websites, such as library and publisher catalogues, to be useful in practice. More valuable web-based indicators can be derived from specific types of web pages that cite academic research, such as online presentations, course syllabi, and science blogs. These provide evidence that is easier to understand and use and less likely to be affected by unwanted types of automatically generated content, although they are susceptible to gaming

Identifying the Invisible Impact of Scholarly Publications: A Multi-Disciplinary Analysis Using Altmetrics

A thesis submitted in partial fulfilment of the requirements of the University of Wolverhampton for the degree of Doctor of Philosophy.The field of ‘altmetrics’ is concerned with alternative metrics for the impact of research publications using social web data. Empirical studies are needed, however, to assess the validity of altmetrics from different perspectives. This thesis partly fills this gap by exploring the suitability and reliability of two altmetrics resources: Mendeley, a social reference manager website, and Faculty of F1000 (F1000), a post- publishing peer review platform. This thesis explores the correlations between the new metrics and citations at the level of articles for several disciplines and investigates the contexts in which the new metrics can be useful for research evaluation across different fields. Low and medium correlations were found between Mendeley readership counts and citations for Social Sciences, Humanities, Medicine, Physics, Chemistry and Engineering articles from the Web of Science (WoS), suggesting that Mendeley data may reflect different aspects of research impact. A comparison between information flows based on Mendeley bookmarking data and cross-disciplinary citation analysis for social sciences and humanities disciplines revealed substantial similarities and some differences. This suggests that Mendeley readership data could be used to help identify knowledge transfer between scientific disciplines, especially for people that read but do not author articles, as well as providing evidence of impact at an earlier stage than is possible with citation counts. The majority of Mendeley readers for Clinical Medicine, Engineering and Technology, Social Science, Physics and Chemistry papers were PhD students and postdocs. The highest correlations between citations and Mendeley readership counts were for types of Mendeley users that often authored academic papers, suggesting that academics bookmark papers in Mendeley for reasons related to scientific publishing. In order to identify the extent to which Mendeley bookmarking counts reflect readership and to establish the motivations for bookmarking scientific papers in Mendeley, a large-scale survey found that 83% of Mendeley users read more than half of the papers in their personal libraries. The main reasons for bookmarking papers were citing in future publications, using in professional activities, citing in a thesis, and using in teaching and assignments. Thus, Mendeley bookmarking counts can potentially indicate the readership impact of research papers that have educational value for non-author users inside academia or the impact of research papers on practice for readers outside academia. This thesis also examines the relationship between article types (i.e., “New Finding”, “Confirmation”, “Clinical Trial”, “Technical Advance”, “Changes to Clinical Practice”, “Review”, “Refutation”, “Novel Drug Target”), citation counts and F1000 article factors (FFa). In seven out of nine cases, there were no significant differences between article types in terms of rankings based on citation counts and the F1000 Article Factor (FFa) scores. Nevertheless, citation counts and FFa scores were significantly different for articles tagged: “New finding” or “Changes to Clinical Practice”. This means that F1000 could be used in research evaluation exercises when the importance of practical findings needs to be recognised. Furthermore, since the majority of the studied articles were reviewed in their year of publication, F1000 could also be useful for quick evaluations

大学図書館における電子ジャーナルの閲読と引用のオブソレッセンス分析

筑波大学修士 (図書館情報学) 学位論文・平成26年3月25日授与 (32621号)201

Bibliometric approach to research in entrepreneurship

El emprendedor, genera un creciente interés que se sustenta en los valiosos beneficios económicos y sociales que produce, lo que ha supuesto casi sin excepción una de las principales motivaciones para su estudio. Las publicaciones sobre emprendimiento (entrepreneurship) han experimentado un crecimiento exponencial en los últimos años. El interés por el campo de investigación se encuentra en pleno auge. Progresivamente se ha generado un universo documental cada vez más inalcanzable tanto para investigadores como para responsables público-privados, incrementando la dificultad para interpretar, analizar y comprender aquellos hallazgos relevantes en contraposición de otros de naturaleza más prescindible. En esta tesitura surge la necesidad de plantear una investigación centrada en incluir el punto de vista bibliométrico para sentar las bases científicas de futuras investigaciones. La cuestión central que justifica la tesis es la necesidad de construir una referencia para impulsar el incremento y mejora de la calidad de los estudios bibliométricos sobre emprendimiento. Para ello, se realizan diferentes análisis: en primer lugar se caracteriza una muestra representativa de documentos altamente citados sobre emprendimiento; seguidamente se estudia el potencial que ofrecen métricas alternativas a las citas, vinculadas al consumo de información como posibles predictoras del impacto futuro de esos documentos; en tercer lugar se profundiza en la evolución temática que ha seguido la disciplina a través del análisis de su contenido; finalmente, se recopilan los principales estudios bibliométricos que han analizado la disciplina de manera global con especial atención a sus características técnicas, objetivos, limitaciones y conclusiones. La tesis presenta importantes implicaciones para el futuro, actúa como nexo de unión entre dos conocimientos habitualmente desconectados. Pretende ser una referencia significativa para académicos interesados en el estudio del emprendimiento mediante herramientas bibliométricas. Aporta como principal contribución la incorporación del enfoque bibliométrico a este tipo de investigación. Entre los principales hallazgos se presentan evidencias que indican que la imagen ofrecida hasta el momento no incorpora la naturaleza multidisciplinar del emprendimiento y podría estar distorsionada. A su vez, detecta una serie de problemas inherentes a su realización, así como la necesidad de incorporar los últimos avances en bibliometría y mejorar la colaboración entre expertos de ambos campos para intentar solventarlos y avanzar hacia futuros progresos