247 research outputs found
Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data
In this study, we address the question whether (and to what extent,
respectively) altmetrics are related to the scientific quality of papers (as
measured by peer assessments). Only a few studies have previously investigated
the relationship between altmetrics and assessments by peers. In the first
step, we analyse the underlying dimensions of measurement for traditional
metrics (citation counts) and altmetrics - by using principal component
analysis (PCA) and factor analysis (FA). In the second step, we test the
relationship between the dimensions and quality of papers (as measured by the
post-publication peer-review system of F1000Prime assessments) - using
regression analysis. The results of the PCA and FA show that altmetrics operate
along different dimensions, whereas Mendeley counts are related to citation
counts, and tweets form a separate dimension. The results of the regression
analysis indicate that citation-based metrics and readership counts are
significantly more related to quality, than tweets. This result on the one hand
questions the use of Twitter counts for research evaluation purposes and on the
other hand indicates potential use of Mendeley reader counts
How to improve the prediction based on citation impact percentiles for years shortly after the publication date?
The findings of Bornmann, Leydesdorff, and Wang (in press) revealed that the
consideration of journal impact improves the prediction of long-term citation
impact. This paper further explores the possibility of improving citation
impact measurements on the base of a short citation window by the consideration
of journal impact and other variables, such as the number of authors, the
number of cited references, and the number of pages. The dataset contains
475,391 journal papers published in 1980 and indexed in Web of Science (WoS,
Thomson Reuters), and all annual citation counts (from 1980 to 2010) for these
papers. As an indicator of citation impact, we used percentiles of citations
calculated using the approach of Hazen (1914). Our results show that citation
impact measurement can really be improved: If factors generally influencing
citation impact are considered in the statistical analysis, the explained
variance in the long-term citation impact can be much increased. However, this
increase is only visible when using the years shortly after publication but not
when using later years.Comment: Accepted for publication in the Journal of Informetrics. arXiv admin
note: text overlap with arXiv:1306.445
Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective
An extensive analysis of the presence of different altmetric indicators
provided by Altmetric.com across scientific fields is presented, particularly
focusing on their relationship with citations. Our results confirm that the
presence and density of social media altmetric counts are still very low and
not very frequent among scientific publications, with 15%-24% of the
publications presenting some altmetric activity and concentrating in the most
recent publications, although their presence is increasing over time.
Publications from the social sciences, humanities and the medical and life
sciences show the highest presence of altmetrics, indicating their potential
value and interest for these fields. The analysis of the relationships between
altmetrics and citations confirms previous claims of positive correlations but
relatively weak, thus supporting the idea that altmetrics do not reflect the
same concept of impact as citations. Also, altmetric counts do not always
present a better filtering of highly cited publications than journal citation
scores. Altmetrics scores (particularly mentions in blogs) are able to identify
highly cited publications with higher levels of precision than journal citation
scores (JCS), but they have a lower level of recall. The value of altmetrics as
a complementary tool of citation analysis is highlighted, although more
research is suggested to disentangle the potential meaning and value of
altmetric indicators for research evaluation
- …