Investigating selection bias of online surveys on coronavirus-related behavioral outcomes

The coronavirus SARS-CoV-2 outbreak has stimulated numerous online surveys that are mainly based on online convenience samples or commercial online access panels where participants select themselves. The results are, nevertheless, often generalized to the general population. In our paper we investigate the potential bias that is introduced by respondents' self-selection. The analysis is based on survey data of the "GESIS Panel Special Survey on the Coronavirus SARS-CoV-2 Outbreak in Germany", together with background information of the GESIS Panel. Our analyses show indication of a nonignorable amount of selection bias for measures of personality traits among online survey respondents. This provides some evidence that participating in an online survey and complying with measures that can minimize the risk of being infected with the SARS-CoV-2 virus are confounded. Hence, generalizing these results to the general population bears the risk of over or underestimating the share of the population that complies with specific measures

Recruitment strategies for a probability-based online panel: Effects of interview length, question sensitivity, incentives and interviewers

Probability-based online panels represent a comparably new and emerging form of data collection infrastructures. To date, there is little empirical evidence on online panel recruitment. This dissertation aims to fill the gap and contribute experimental evidence. The overall objective is to identify ways to optimize the telephone recruitment process of a probability-based online panel in Germany and derive practical recommendations. Referring to the Total Survey Error perspective (Groves & Lyberg, 2010) optimal is defined in the sense of maximizing the recruitment probability and online participation probability and minimizing the selection bias under given budget constraints. Based on the framework of survey participation (Groves & Couper, 1998), the four studies of this dissertation focus on several aspects of the recruitment process that researchers can decide upon and have control about. In three survey experiments, the effect of varying survey features on the success of the recruitment process is analyzed. The experimental factors are the length of the recruitment interview, the inclusion of a sensitive question, and incentives. In an additional analysis, the role of interviewers as an additional error source during the recruitment process is examined

Documentation of online surveys (Version 1.0)

This guideline on documentation of online surveys should provide guidance to researchers that aim at documenting the process of an online survey data collection. The collection of empirical data is an essential part of the scientific process. It is, therefore, important to comprehensively document how scientific data was collected. The information included in such a description will enable others to critically evaluate the data collection procedure and make replication studies possible. The guideline at hand attempts to provide a comprehensive list of aspects important for the documentation of an online survey: General aspects, details of the data collection preparation, the data collection phase, and the data processing are included in the guideline

Informing about Web Paradata Collection and Use (Version 1.0)

This survey guideline addresses the practical question of how best to inform survey participants about the collection and use of paradata in web surveys. We provide an overview of different personal and non-personal web paradata and the associated information and consent requirements. Best practices regarding the procedure, wording, and placement of non-personal web paradata information are discussed. In addition, we propose a sample wording for web paradata information in German and English

The role of the information environment during the first COVID-19 wave in Germany

The COVID-19 pandemic has been accompanied by intense debates about the role of the information environment. On the one hand, citizens learn from public information campaigns and news coverage and supposedly adjust their behaviours accordingly; on the other, there are fears of widespread misinformation and its detrimental effects. Analyzing the posts of the most important German information providers published via Facebook, this paper first identifies a uniform salience of subtopics related to COVID-19 across different types of information sources that generally emphasized the threats to public health. Next, using a large survey conducted with German residents during the first COVID-19 wave in March 2020 we investigate how information exposure relates to perceptions, attitudes and behaviours concerning the pandemic. Regression analyses show that getting COVID-19-related information from a multitude of sources has a statistically significant and positive relationship with public health outcomes. These findings are consistent even across the ideological left/right spectrum and party preferences. These consistent correlational results demonstrate that during the first wave of COVID-19, a uniform information environment went hand in hand with a cautious public and widely accepted mitigation measures. Nonetheless, we discuss these findings against the backdrop of an increased politicization of public-health measures during later COVID-19 waves

Fieldwork Monitoring in Practice: Insights from 17 Large-scale Social Science Surveys in Germany

This study provides a synopsis of the current fieldwork monitoring practices of large-scale surveys in Germany. Based on the results of a standardized questionnaire, the study summarizes fieldwork monitoring indicators used and fieldwork measures carried out by 17 large-scale social sciences surveys in Germany. Our descriptive results reveal that a common set of fieldwork indicators and measures exist on which the studied surveys rely. However, it also uncovers the need for additional design-specific indicators. Finally, it underlines the importance of a close cooperation between survey representatives and fieldwork agencies to optimize processes in fieldwork monitoring in the German survey context. The article concludes with implications for fieldwork practice

The long-term impact of different offline population inclusion strategies in probability-based online panels: Evidence from the German Internet Panel and the GESIS Pane

While online panels offer numerous advantages, they are often criticized for excluding the offline population. Therefore, some probability-based online panels have developed offline population inclusion strategies. Two dominant approaches prevail: providing internet equipment and offering an alternative survey participation mode. We investigate the impact of these approaches on two probability-based online panels in Germany: the German Internet Panel, which provides members of the offline population with internet equipment, and the GESIS Panel, which offers members of the offline population to participate via postal mail surveys. In addition, we explore the impact of offering an alternative mode only to non-internet users versus also offering the alternative mode to internet users who are unwilling to provide survey data online. Albeit lower recruitment and/or panel wave participation probabilities among offliners than onliners, we find that including the offline population has a positive long-term impact on sample accuracy in both panels. In the GESIS Panel, the positive impact is particularly strong when offering the alternative participation mode to non-internet users and internet users who are unwilling to provide survey data online

Hygienegesellschaften als Experimentiergesellschaften? Corona als Herausforderung für Strukturen resilienten Experimentierens

Die explosive Verbreitung des Corona-Virus über die Welt brachte ein Realexperiment globaler Dimension mit offenem Ausgang hervor. Gesellschaften formieren sich, kosmopolitisch verschränkt, als Hygienegesellschaften. Zwischen striktem Lockdown und minimaler Begrenzung gehen sie je eigene experimentelle Wege. Dabei wird der Raum des Experimentellen bisher eher defensiv gestaltet, was in der Konsequenz zur Begrenzung experimenteller Räume sowie zur schleichenden Delegitimierung von Kontrollpolitiken der Pandemie führt. Die These dieses Beitrags lautet, dass solche Legitimationsprobleme umso eher gelöst werden können, je eher Strukturen resilienten Experimentierens entworfen und institutionalisiert werden. Ein wichtiges Ergebnis der Überlegungen ist, dass die resiliente Gestaltung experimenteller Räume zur Bewältigung solchen Krisengeschehens nur gelingen kann, wenn neben top-down verordneten Kontrollen sich eine bottom- up-Kultur sozialen Experimentierens entfalten kann. Dies setzt auf Seiten der Zivilgesellschaft ebenso wie von staatlicher Seite die Fähigkeit und den Willen zu Partizipation und Kooperation voraus. Entsprechende Ansatzpunkte werden dafür identifiziert. The dynamic spread of the corona virus has triggered an open-ended real-world experiment of global dimensions. Being under siege by Corona Pandemic, societies are forming themselves as hygiene societies. While cosmopolitically intertwined, they create their own experimental paths, changing between strict lockdown and minimal intervention strategies. With regard to experimental spaces, disease policies still follow a defensive strategy. This leads not only to the limitation of experimental processes but rather to a creeping delegitimization of health control policies. This paper stresses that societies could better cope with urging legitimation problems if structures of resilient experimentation would be designed and appropriately institutionalized. Moreover, implementing resilient designs of experimental spaces, to successfully cope with pandemic crises, requires a vivid bottom-up culture of social experimentation to complement current top-down control measures. However, this presupposes the willingness as well as the capability to participate and cooperate on both sides, the civil society and the state. Correspondingly, appropriate starting points will be identified.  (editorial reviewed

Enhancing Participation in Probability-Based Online Panels: Two Incentive Experiments and Their Effects on Response and Panel Recruitment

This article investigates how mail-based online panel recruitment can be facilitated through incentives. The analysis relies on two incentive experiments and their effects on panel recruitment, and the intermediate participation in the recruitment survey. The experiments were implemented in the context of the German Emigration and Remigration Panel Study and encompass two samples of randomly sampled persons. Tested incentives include a conditional lottery, conditional monetary incentives, and the combination of unconditional money-in-hand with conditional monetary incentives. For an encompassing evaluation of the link between incentives and panel recruitment, the article further assesses the incentives’ implications for demographic composition and panel recruitment unit costs. Multivariate analysis indicates that low combined incentives (€5/€5) or, where unconditional disbursement is unfeasible, high conditional incentives (€20) are most effective in enhancing panel participation. In terms of demographic bias, low combined incentives (€5/€5) and €10 conditional incentives are the favored options. The budget options from the perspective of panel recruitment include the lottery and the €10 conditional incentive which break-even at net sample sizes of 1000