62 research outputs found

    People rely on their attitudes more than the source when judging the accuracy of news stories on Facebook

    Get PDF
    The role of 'fake news' in shaping political behaviour has received extensive attention in recent years, with Facebook and other websites undertaking a number of measures to try and address the problem. Drawing on an experimental study during the 2017 German federal election campaign, Bernhard Clemm von Hohenberg illustrates that people rely far more on their pre-existing political attitudes when judging the accuracy of a story on Facebook than they do on the reputation of a news source

    Horseshoe patterns: visualizing partisan media trust in Germany

    Get PDF
    A trusted media is crucial for a politically informed citizenry, yet media trust has become fragile in many Western countries. An underexplored aspect is the link between media (dis)trust and populism. The authors visualize media trust across news outlets and partisanship in Germany, for both mainstream and “alternative” news sources. For each source, average trust is grouped by partisanship and sorted from left to right, allowing within-source comparisons. The authors find an intriguing horseshoe pattern for mainstream media sources, for which voters of both populist left-wing and right-wing parties express lower levels of trust. The underlying distribution of individual responses reveals that voters of the right-wing populist party are especially likely to “not at all” trust the mainstream outlets that otherwise enjoy high levels of trust. The media trust gap between populist and centrist voters disappears for alternative sources, for which trust is generally low

    Analysis of Web Browsing Data: A Guide

    Get PDF
    The use of individual-level browsing data, that is, the records of a person’s visits to online content through a desktop or mobile browser, is of increasing importance for social scientists. Browsing data have characteristics that raise many questions for statistical analysis, yet to date, little hands-on guidance on how to handle them exists. Reviewing extant research, and exploring data sets collected by our four research teams spanning seven countries and several years, with over 14,000 participants and 360 million web visits, we derive recommendations along four steps: preprocessing the raw data; filtering out observations; classifying web visits; and modelling browsing behavior. The recommendations we formulate aim to foster best practices in the field, which so far has paid little attention to justifying the many decisions researchers need to take when analyzing web browsing data.Die Verwendung von Browsing-Daten auf individueller Ebene, d.h. die Aufzeichnungen der Besuche einer Person bei Online-Inhalten ĂŒber einen Desktop- oder mobilen Browser, ist fĂŒr Sozialwissenschaftler*innen von zunehmender Bedeutung. Browsing-Daten haben Eigenschaften, die viele Fragen fĂŒr die statistische Analyse aufwerfen, doch bisher gibt es nur wenige praktische Anleitungen fĂŒr den Umgang mit ihnen. Nach Durchsicht bestehender Forschungsarbeiten und der Untersuchung von DatensĂ€tzen, die von vier Forschungsteams in sieben LĂ€ndern und ĂŒber mehrere Jahre hinweg gesammelt wurden, mit ĂŒber 14.000 Teilnehmenden und 360 Millionen Webbesuchen, leiten die Autor*innen Empfehlungen in vier Schritten ab: Vorverarbeitung der Rohdaten, Herausfiltern von Beobachtungen, Klassifizierung von Webbesuchen und Modellierung des Surfverhaltens

    Truth and Bias: Robust findings?

    No full text
    Differences in information processing across ideological groups have been a recurring theme in political science. The recent debate about “fake news” has brought attention to the question whether US liberals and conservatives differ in how they evaluate the truth of information. Researchers have asked, first, which side is better at discerning true from false information (truth discernment), and second, whether liberals and conservatives are driven to different degrees by the ideological congruence of information (assimilation bias). The paradigmatic designs to study these question require selecting or constructing informational stimuli. As I show empirically with two robustness tests and one extended replication of previous studies, this selection/construction necessarily affects the resulting (a)symmetries. When it is unclear how well the selection represents some universe of real-world information, we should thus be wary of results about asymmetries. Ideally, studies should find ways to randomly sample stimuli from the target population of information

    Large Language Models as a Substitute for Human Experts in Annotating Political Text

    No full text
    Large-scale text analysis has grown rapidly as a method in political science and beyond. To date, text-as-data methods rely on large volumes of human-annotated training examples, which places a premium on researcher resources. However, advances in large language models (LLMs) may make automated annotation increasingly viable. This paper tests the performance of GPT-4 across a range of scenarios relevant for analysis of political text. We compare GPT-4 coding with human expert coding of tweets and news articles across four variables (whether text is political, negativity, sentiment, and ideology) and across four countries (the United States, Chile, Germany, and Italy). GPT-4 coding is highly accurate, especially for shorter texts such as tweets, correctly classifying texts up to 95\% of the time. Performance drops for longer news articles, and very slightly for non-English text. We introduce a ``hybrid'' coding approach, in which disagreements of multiple GPT-4 runs are adjudicated by a human expert, which boosts accuracy. Finally, we explore downstream effects, finding that transformer models trained on hand-coded or GPT-4-coded data yield almost identical outcomes. Our results suggests that LLM-assisted coding is a viable and cost-efficient approach, although consideration should be given to task complexity

    Large language models as a substitute for human experts in annotating political text

    Get PDF
    Large-scale text analysis has grown rapidly as a method in political science and beyond. To date, text-as-data methods rely on large volumes of human-annotated training examples, which place a premium on researcher resources. However, advances in large language models (LLMs) may make automated annotation increasingly viable. This paper tests the performance of GPT-4 across a range of scenarios relevant for analysis of political text. We compare GPT-4 coding with human expert coding of tweets and news articles across four variables (whether text is political, its negativity, its sentiment, and its ideology) and across four countries (the United States, Chile, Germany, and Italy). GPT-4 coding is highly accurate, especially for shorter texts such as tweets, correctly classifying texts up to 95% of the time. Performance drops for longer news articles, and very slightly for non-English text. We introduce a ‘hybrid’ coding approach, in which disagreements of multiple GPT-4 runs are adjudicated by a human expert, which boosts accuracy. Finally, we explore downstream effects, finding that transformer models trained on hand-coded or GPT-4-coded data yield almost identical outcomes. Our results suggest that LLM-assisted coding is a viable and cost-efficient approach, although consideration should be given to task complexity
    • 

    corecore