17 research outputs found
Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data
In this study, we address the question whether (and to what extent,
respectively) altmetrics are related to the scientific quality of papers (as
measured by peer assessments). Only a few studies have previously investigated
the relationship between altmetrics and assessments by peers. In the first
step, we analyse the underlying dimensions of measurement for traditional
metrics (citation counts) and altmetrics - by using principal component
analysis (PCA) and factor analysis (FA). In the second step, we test the
relationship between the dimensions and quality of papers (as measured by the
post-publication peer-review system of F1000Prime assessments) - using
regression analysis. The results of the PCA and FA show that altmetrics operate
along different dimensions, whereas Mendeley counts are related to citation
counts, and tweets form a separate dimension. The results of the regression
analysis indicate that citation-based metrics and readership counts are
significantly more related to quality, than tweets. This result on the one hand
questions the use of Twitter counts for research evaluation purposes and on the
other hand indicates potential use of Mendeley reader counts
Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?
Evaluative bibliometrics compares the citation impact of researchers,
research groups and institutions with each other across time scales and
disciplines. Both factors - discipline and period - have an influence on the
citation count which is independent of the quality of the publication.
Normalizing the citation impact of papers for these two factors started in the
mid-1980s. Since then, a range of different methods have been presented for
producing normalized citation impact scores. The current study uses a data set
of over 50,000 records to test which of the methods so far presented correlate
better with the assessment of papers by peers. The peer assessments come from
F1000Prime - a post-publication peer review system of the biomedical
literature. Of the normalized indicators, the current study involves not only
cited-side indicators, such as the mean normalized citation score, but also
citing-side indicators. As the results show, the correlations of the indicators
with the peer assessments all turn out to be very similar. Since F1000 focuses
on biomedicine, it is important that the results of this study are validated by
other studies based on datasets from other disciplines or (ideally) based on
multi-disciplinary datasets.Comment: Accepted for publication in the Journal of Informetric