“When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences

The Social Sciences have long been struggling with quantitative forms of research assessment—insufficient coverage in prominent citation indices and overall lower citation counts than in STM subject areas have led to a widespread weariness regarding bibliometric evaluations among social scientists. Fueled by the rise of the social web, new hope is often placed on alternative metrics that measure the attention scholarly publications receive online, in particular on social media. But almost a decade after the coining of the term altmetrics for this new group of indicators, the uptake of the concept in the Social Sciences still seems to be low. Just like with traditional bibliometric indicators, one central problem hindering the applicability of altmetrics for the Social Sciences is the low coverage of social science publications on the respective data sources—which in the case of altmetrics are the various social media platforms on which interactions with scientific outputs can be measured. Another reason is that social scientists have strong opinions about the usefulness of metrics for research evaluation which may hinder broad acceptance of altmetrics too. We conducted qualitative interviews and online surveys with researchers to identify the concerns which inhibit the use of social media and the utilization of metrics for research evaluation in the Social Sciences. By analyzing the response data from the interviews in conjunction with the response data from the surveys, we identify the key concerns that inhibit social scientists from (1) applying social media for professional purposes and (2) making use of the wide array of metrics available. Our findings show that aspects of time consumption, privacy, dealing with information overload, and prevalent styles of communication are predominant concerns inhibiting Social Science researchers from using social media platforms for their work

“When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences

The Social Sciences have long been struggling with quantitative forms of research assessment—insufficient coverage in prominent citation indices and overall lower citation counts than in STM subject areas have led to a widespread weariness regarding bibliometric evaluations among social scientists. Fueled by the rise of the social web, new hope is often placed on alternative metrics that measure the attention scholarly publications receive online, in particular on social media. But almost a decade after the coining of the term altmetrics for this new group of indicators, the uptake of the concept in the Social Sciences still seems to be low. Just like with traditional bibliometric indicators, one central problem hindering the applicability of altmetrics for the Social Sciences is the low coverage of social science publications on the respective data sources—which in the case of altmetrics are the various social media platforms on which interactions with scientific outputs can be measured. Another reason is that social scientists have strong opinions about the usefulness of metrics for research evaluation which may hinder broad acceptance of altmetrics too. We conducted qualitative interviews and online surveys with researchers to identify the concerns which inhibit the use of social media and the utilization of metrics for research evaluation in the Social Sciences. By analyzing the response data from the interviews in conjunction with the response data from the surveys, we identify the key concerns that inhibit social scientists from (1) applying social media for professional purposes and (2) making use of the wide array of metrics available. Our findings show that aspects of time consumption, privacy, dealing with information overload, and prevalent styles of communication are predominant concerns inhibiting Social Science researchers from using social media platforms for their work. Regarding indicators for research impact we identify a widespread lack of knowledge about existing metrics, their methodologies and meanings as a major hindrance for their uptake through social scientists. The results have implications for future developments of scholarly online tools and show that researchers could benefit considerably from additional formal training regarding the correct application and interpretation of metrics

Evaluating altmetrics acts through their creators – how to advance?

<p>One of the grand challenges in the meaningful use and interpretation of altmetrics is the heterogeneity of the acts behind them (Haustein, 2016). On the one hand, the diversity of the online interactions as a measure of scholarly impact is part of what makes altmetrics such a promising complement to traditional impact measures. On the other hand, this diversity makes the interpretation of altmetrics a difficult endeavor, as altmetrics derived from different social media platforms are shaped by significantly different premises. Although different actions on those platforms are in many cases fundamentally different regarding both the respective user’s degree of involvement and intention, their scores are displayed side by side by altmetrics providers without much further explanation of their diverse premises. For example, bookmarking a publication in Mendeley has a substantially different meaning from writing a post about the same publication in Facebook. In order to account for semantic differences between acts from different sources for altmetrics, efforts have been made to classify interactions regarding the required degree of involvement (Haustein, Bowman, & Costas, 2016) or their stakeholders’ main use cases („NISO RP-25-2016, Outputs of the NISO Alternative Assessment Project - National Information Standards Organization“, 2016). One largely unexplored premise that should be considered when interpreting altmetrics are differences regarding the platforms’ userships – the users that are responsible for the interactions underlying altmetrics. Referring to past studies, the share of academics among the users interacting with scientific articles seems to vary considerably between platforms: while for example Jin-Cheon Na & Yingxin Estella Ye (2017) found a distinct predominance of non-academic users in discussions of psychological academic articles on Facebook, Vainio & Holmberg (2017) found researchers to be strongly represented among Twitter users responsible for tweeting scientific articles. And even for those platforms for which we can assume that the relevant share of interactions with scientific publications is committed by researchers such as Mendeley and ResearchGate (Sugimoto, Work, Larivière, & Haustein, 2016), there still might be considerable differences regarding the overall researchers’ professional experience, productivity in terms of traditional publications or represented fields of research between individual platforms. </p> <p>The differentiation between the semantics of acts in altmetrics – and their weighting based on this differentiation – leads to many questions regarding a sensible methodology and also its overall desirableness.</p> <p> </p> <p> </p