Using supervised machine learning to code policy issues: Can classifiers generalize across contexts?

Content analysis of political communication usually covers large amounts of material and makes the study of dynamics in issue salience a costly enterprise. In this article, we present a supervised machine learning approach for the automatic coding of policy issues, which we apply to news articles and parliamentary questions. Comparing computer-based annotations with human annotations shows that our method approaches the performance of human coders. Furthermore, we investigate the capability of an automatic coding tool, which is based on supervised machine learning, to generalize across contexts. We conclude by highlighting implications for methodological advances and empirical theory testing

Owning the issues of crime and immigration: the relation between immigration and crime news and anti-immigrant voting in 11 countries

It is still not well understood how the media affect anti-immigrant party voting. In this paper, we argue and demonstrate empirically that mere exposure to immigration- and crime-related news is positively related to the likelihood that a voter casts a vote for an anti-immigrant party. On the basis of a media content analysis (N = 20,084 news items) in combination with a voter panel survey (N = 17,014 respondents) conducted in 11 European countries we find for several anti-immigrant parties that - ceteris paribus - exposure to news about immigration or crime increases voters' probabilities to vote for these parties. We discuss our findings in light of prior research on issue ownership, and their implications for the role of the mass media in established democracies

Machine Learning-Based Content Analysis: Automating the analysis of frames and agendas in political communication research

We used machine learning to study policy issues and frames in political messages. With regard to frames, we investigated the automation of two content-analytical tasks: frame coding and frame identification. We found that both tasks can be successfully automated by means of machine learning techniques. Frame coding can be automated through supervised machine learning (SML). Results show that the performance of SML-based frame coding approaches the performance of human coders. Furthermore, we have shown that frames can be automatically identified through clustering, a form of unsupervised machine learning. We used this method to identify issue frames in the nuclear power debate. We found that automatically identified frames closely resemble frames that have been identified in previous studies, by means of qualitative approaches. In addition, we have shown that policy issues can be coded by means of SML as well as through semi-automatically created dictionaries. Again, automatic coding approaches the performance of human coders. Moreover, we demonstrated that SML and dictionary-based coding can be applied to different types of political messages (e.g., news articles and parliamentary records)

Do perceived poll results affect party preferences, or do party preferences affect perceived poll results?

It is well established in the literature that a party’s perceived standing in the polls affects voters’ probability to vote for that party. However, do voters perceive a parties’ poll performance in accurate ways, or is there a nonrandom error to poll perceptions? In this paper, we argue that poll perceptions are systematically biased. On the basis of data from a voter survey conducted in four countries (N=22,504) we find for most parties an interplay of poll perceptions and probabilities to vote. Indeed, we more often find probabilities to vote influencing poll perceptions than vice versa. This bias tends to be larger among the lower-educated, and among the less knowledgeable. We conclude by setting our findings in wider perspective and discussing the relevance to the research field, and society more generally

From newsworthiness to shareworthiness: How to predict news sharing based on article characteristics

People increasingly visit online news sites not directly, but by following links on social network sites. Drawing on news value theory and integrating theories about online identities and self-representation, we develop a concept of shareworthiness, with which we seek to understand how the number of shares an article receives on such sites can be predicted. Findings suggest that traditional criteria of newsworthiness indeed play a role in predicting the number of shares, and that further development of a theory of shareworthiness based on the foundations of newsworthiness can offer fruitful insights in news dissemination processes

Frames beyond words: Applying cluster and sentiment analysis to news coverage of the nuclear power issue

Methods to automatically analyze media content are advancing significantly. Among others, it has become increasingly popular to analyze the framing of news articles by means of statistical procedures. In this article, we investigate the conceptual validity of news frames that are inferred by a combination of k-means cluster analysis and automatic sentiment analysis. Furthermore, we test a way of improving statistical frame analysis such that revealed clusters of articles reflect the framing concept more closely. We do so by only using words from an article’s title and lead and by excluding named entities and words with a certain part of speech from the analysis. To validate revealed frames, we manually analyze samples of articles from the extracted clusters. Findings of our tests indicate that when following the proposed feature selection approach, the resulting clusters more accurately discriminate between articles with a different framing. We discuss the methodological and theoretical implications of our findings

Automatic thematic content analysis: finding frames in news

Framing in news is the way in which journalists depict an issue in terms of a 'central organizing idea.' Frames can be a perspective on an issue. We explore the automatic classification of four generic news frames: conflict, human interest, economic consequences, and morality. Complex characteristics of messages such as frames have been studied using thematic content analysis. Indicator questions are formulated, which are then manually coded by humans after reading a text and combined into a characterization of the message. We operationalize this as a classification task and, inspired by the way-of-working of media analysts, we propose a two-stage approach, where we first rate a news article using indicator questions for a frame and then use the outcomes to predict whether a frame is present. We approach human accuracy on almost all indicator questions and frames