(Un)sophisticated reasoning? The integrative complexity of user-generated debates across political systems and online discussion arenas

This study is the first to compare the integrative complexity of online user comments across distinct political systems and in discussion arenas with different primary use functions. Integrative complexity is a psycho-linguistic construct that is increasingly used by communication scholars to study the argumentative quality of political debate contributions. It captures the sophistication of online user comments in terms of differentiation and integration, mapping whether a post contains different aspects or viewpoints related to an issue and the extent to which it draws conceptual connections between these. This study investigates user contributions on the public role of religion and secularism in society between August 2015 and July 2016 from Australia, the United States, Germany and Switzerland. In each country, it analyzes user posts from the a) website comment sections and b) public Facebook pages of mainstream news media, from the c) Facebook pages of partisan collective actors and alternative media, and from d) Twitter. Almost as many user contributions implicitly or explicitly differentiate various dimensions of or perspectives on an issue as express unidimensional, simplistic thoughts. Conceptual integration, however, is rare. The integrative complexity of online user comments is higher in consensus-oriented than in majoritarian democracies and in arenas that are used primarily for issue-driven, plural discussions rather than preference-driven, like-minded debates. This suggests that the accommodative public debate cultures of consensus-oriented political systems and interactions with individuals who hold different positions promote more argumentatively complex over simple online debate contributions

The integrative complexity of online user comments across different types of democracy and discussion arenas

This study is the first to compare the integrative complexity of online user comments across distinct democratic political systems and in discussion arenas with different primary use functions. Integrative complexity is a psycho-linguistic construct that is increasingly used by communication scholars to study the argumentative quality of political debate contributions. It captures the sophistication of online user comments in terms of differentiation and integration, mapping whether a post contains different aspects or viewpoints related to an issue and the extent to which it draws conceptual connections between these. This study investigates user contributions on the public role of religion and secularism in society between August 2015 and July 2016 from Australia, the United States, Germany, and Switzerland. In each country, it analyzes user posts from the (a) website comment sections and (b) public Facebook pages of mainstream news media, from the (c) Facebook pages of partisan collective actors and alternative media, and from (d) Twitter. Almost as many user contributions implicitly or explicitly differentiate various dimensions of or perspectives on an issue as express unidimensional, simplistic thoughts. Conceptual integration, however, is rare. The integrative complexity of online user comments is higher in consensus-oriented than in majoritarian democracies and in arenas that are used primarily for issue-driven, plural discussions rather than preference-driven, like-minded debates. This suggests that the accommodative public debate cultures of consensus-oriented political systems and interactions with individuals who hold different positions promote more argumentatively complex over simple online debate contributions

The (de)civilizing impact of an inclusive actor set in news articles on associated user debates

Online incivility has found its way into the mainstream. Searching for strategies to counter this development, research has focused on the mitigating effects of moderation or user identification. When people comment on articles in the website comment sections or on the Facebook pages of media outlets, content-related properties of the news can influence the incivility of these discussions. This study is among the first to investigate how deliberative attributes of an article influence the style of user-generated debates. It asks whether the inclusion of diverse actors in an article triggers more "toxic outrage" in online discussions and which actors may have a moderating influence. The findings suggest that toxic outrage in user comments is driven specifically by political controversy in the article. Few types of actors actually have a moderating influence on the style of online debates. This is particularly unexpected for periphery actors such as civil society groups or citizens whose inclusion into political discourses is highly valued by deliberative theory

What facilitates constructive engagement? A dictionary-based comparison of outrage and recognition across online platforms

This paper examines the style of user-generated debate on the divisive issue of the public role of religion and secularism in the US. In a dictionary-based comparison, we measure both outrage and recognition in comments on news websites, Facebook news pages, Facebook pages of partisan actors and alternative media, as well as on Twitter. To our knowledge, this is the first attempt to explicitly capture both negative and positive dimensions of mediated debate by computational means. Our results show that the style of user-generated debate is more outrageous and less recognitive on platforms which mix public and private contexts (Facebook) than on those which separate the two. Furthermore, we find that the style is more outrageous and less recognitive in issue-driven debates that evolve pluralistically around contentious issues (news websites & Facebook news pages) than in preference-driven discussions that bring together like-minded people. The findings indicate that separated contexts can foster constructive engagement among users and confirm that pluralistic debates are marked considerably by rude behavior. Future research should identify effective strategies to counter such negative tendencies and systematically compare the style of user-generated discourse across countries

Mannheim International News Discourse Data Set (MIND)

The MIND data set is a collection of news items spanning one year beginning 1st of August 2015 until the 31st of July 2016. It contains news items of over 110 sources from six countries on four continents. Selected were the most relevant political information sources in each country, for the research question "Religion and Secularism in the Society" in the categories News Website, Printed Newspaper and Blog. Collected was the complete output of each source (e.g. with the topics politics, society, economy, culture, while excluding the topics sport, lifestyle and weather)

Enhancing theory-informed dictionary approaches with “glass-box” machine learning: The case of integrative complexity in social media comments

Dictionary-based approaches to computational text analysis have been shown to perform relatively poorly, particularly when the dictionaries rely on simple bags of words, are not specified for the domain under study, and add word scores without weighting. While machine learning approaches usually perform better, they offer little insight into (a) which of the assumptions underlying dictionary approaches (bag-of-words, domain transferability, or additivity) impedes performance most, and (b) which language features drive the algorithmic classification most strongly. To fill both gaps, we offer a systematic assumption-based error analysis, using the integrative complexity of social media comments as our case in point. We show that attacking the additivity assumption offers the strongest potential for improving dictionary performance. We also propose to combine off-the-shelf dictionaries with supervised “glass box” machine learning algorithms (as opposed to the usual “black box” machine learning approaches) to classify texts and learn about the most important features for classification. This dictionary-plus-supervised-learning approach performs similarly well as classic full-text machine learning or deep learning approaches, but yields interpretable results in addition, which can inform theory development on top of enabling a valid classification