Representativeness and response quality of survey data

In the social sciences, research is often based on findings from survey data. Common research topics examine political behavior, societal attitudes and opinions, as well as personal values. However impactful the results might be, research and policy debates based on survey data rely on the assumption that the survey data are of high enough quality to be able to draw inferences from the data to a broader population. My dissertation covers potential errors of survey data with regard to representativeness and response quality from different perspectives

The utility of auxiliary data for survey response modeling: Evidence from the German Internet Panel

Auxiliary data are becoming more important as nonresponse rates increase and new fieldwork monitoring and respondent targeting strategies develop. In many cases, auxiliary data are collected or linked to the gross sample to predict survey response. If the auxiliary data have high predictive power, the response models can meaningfully inform survey operations as well as post-survey adjustment procedures. In this paper, I examine the utility of different sources of auxiliary data (sampling frame data, interviewer observations, and micro-geographic area data) for modeling survey response in a probability-based online panel in Germany. I find that the utility of each of these data sources is challenged by a number of concerns (scarcity, missing data, transparency issues, and high levels of aggregation) and that none of the auxiliary data are associated with survey response to any substantial degree

Response quality in nonprobability and probability-based online panels

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based online panels with identical fieldwork periods and questionnaires in Germany. Three response quality indicators typically associated with survey satisficing are assessed: straight-lining in grid questions, item nonresponse, and midpoint selection in visual design experiments. Our results show that there is significantly more straight-lining in the nonprobability online panels than in the probability-based online panels. However, contrary to our expectations, there is no generalizable difference between nonprobability online panels and probability-based online panels with respect to item nonresponse. Finally, neither respondents in nonprobability online panels nor respondents in probability-based online panels are significantly affected by the visual design of the midpoint of the answer scale

Recruiting a Probability-Based Online Panel via Postal Mail: Experimental Evidence

Once recruited, probability-based online panels have proven to enable high-quality and high-frequency data collection. In ever faster-paced societies and, recently, in times of pandemic lockdowns, such online survey infrastructures are invaluable to social research. In absence of email sampling frames, one way of recruiting such a panel is via postal mail. However, few studies have examined how to best approach and then transition sample members from the initial postal mail contact to the online panel registration. To fill this gap, we implemented a large-scale experiment in the recruitment of the 2018 sample of the German Internet Panel (GIP) varying panel recruitment designs in four experimental conditions: online-only, concurrent mode, online-first, and paper-first. Our results show that the online-only design delivers higher online panel registration rates than the other recruitment designs. In addition, all experimental conditions led to similarly representative samples on key socio-demographic characteristics

Measurement equivalence in probability and nonprobability online panels

Nonprobability online panels are commonly used in the social sciences as a fast and inexpensive way of collecting data in contrast to more expensive probability-based panels. Given their ubiquitous use in social science research, a great deal of research is being undertaken to assess the properties of nonprobability panels relative to probability ones. Much of this research focuses on selection bias, however, there is considerably less research assessing the comparability (or equivalence) of measurements collected from respondents in nonprobability and probability panels. This article contributes to addressing this research gap by testing whether measurement equivalence holds between multiple probability and nonprobability online panels in Australia and Germany. Using equivalence testing in the Confirmatory Factor Analysis framework, we assessed measurement equivalence in six multi-item scales (three in each country). We found significant measurement differences between probability and nonprobability panels and within them, even after weighting by demographic variables. These results suggest that combining or comparing multi-item scale data from different sources should be done with caution. We conclude with a discussion of the possible causes of these findings, their implications for survey research, and some guidance for data users.publishedVersio

Measurement instruments for fast and frequent data collection during the early phase of COVID-19 in Germany: reflections on the Mannheim Corona Study

The outbreak of the COVID-19 pandemic has led to a vast increase in the demand for fast, frequent, and multi-faceted data to study the impact of the pandemic on people’s lives. Existing data collection infrastructures had to be adapted quickly during the early phase of the pandemic to meet this data demand. Our research group contributed to this by conducting the Mannheim Corona Study (MCS), a longitudinal probability-based online survey, in a daily rotating panel design that took place from March 20 through July 10, 2020. The fast-and-frequent panel data collection design of the MCS had numerous consequences for designing its questionnaires and choosing its measurement instruments. This included designing new instruments on the fly in the ever-changing pandemic environment, making efficient use of limited questionnaire space, and deciding on measurement frequencies in a structured manner under uncertain external conditions. In this report, we document the MCS approach to choosing measurement instruments fit for the purpose of fast and frequent data collection during the early phase of COVID-19 in Germany. We particularly highlight three examples of measurement instruments in the MCS and reflect on their measurement properties. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s42409-022-00030-5

The Mannheim Corona Study: Life in Germany in a state of emergency : Report for March 20 to June 30, 2020 ; English translation of the original report (in German)

We are all affected by the corona pandemic and the measures taken by the federal government to decelerate the spread of the virus. At the University of Mannheim, we have developed a survey instrument over the past eight years which we can use to quickly examine the social implications of the pandemic in Germany. To the best of our knowledge, our study is the only one in Germany which can assess how the corona crisis is affecting people’s lives in Germany on a daily basis. That is why we consider it as our social duty to contribute to a better understanding of how the corona crisis is affecting the population and to inform the public as well as decision-makers in politics and business about current developments

The Mannheim Corona study: Life in Germany in a state of emergency : Report for March 20 to July 09, 2020

We are all affected by the corona pandemic and the measures taken by the federal government to decelerate the spread of the virus. At the University of Mannheim, we have developed a survey instrument over the past eight years which we can use to quickly examine the social implications of the pandemic in Germany. To the best of our knowledge, our study is the only one in Germany which can assess how the corona crisis is affecting people’s lives in Germany on a daily basis. That is why we consider it as our social duty to contribute to a better understanding of how the corona crisis is affecting the population and to inform the public as well as decision-makers in politics and business about current developments. The Mannheim Corona Study started on Friday, March 20. The study is carried out daily and reports on life in Germany during the corona crisis. We examine social and economic aspects (such as childcare, employment situations, and disposable income), the influence of political measures on social interactions, anxiety as well as public acceptance of the measures taken to contain the pandemic. Between 411 und 643 (on average 489) respondents take part in the study every day

The Mannheim Corona study: life in Germany in a state of emergency : report for March 20 to May 28, 2020, English translation of the original report (in German)

We are all affected by the corona pandemic and the measures taken by the federal government to decelerate the spread of the virus. At the University of Mannheim, we have developed a survey instrument over the past eight years which we can use to quickly examine the social implications of the pandemic in Germany. To the best of our knowledge, our study is the only one in Germany which can assess how the corona crisis is affecting people’s lives in Germany on a daily basis. That is why we consider it as our social duty to contribute to a better understanding of how the corona crisis is affecting the population and to inform the public as well as decision-makers in politics and business about current developments. The Mannheim Corona Study started on Friday, March 20. The study is carried out daily and reports on life in Germany during the corona crisis. We examine social and economic aspects (such as childcare, employment situations, and disposable income), the influence of political measures on social interactions, anxiety as well as public acceptance of the measures taken to contain the pan

Die Mannheimer Corona-Studie: Schwerpunktbericht zu Erwerbstätigkeit und Kinderbetreuung

Die Corona-Pandemie verändert das Leben und den Alltag der Deutschen. In diesem Schwerpunktbericht der Mannheimer Corona-Studie richten wir den Blick auf die Arbeitswelt und darauf wie die Kinder in Deutschland betreut werden. Welche Auswirkungen auf Beruf und Erwerbstätigkeit sehen wir rund drei Wochen nachdem die ersten Maßnahmen zur Pandemiebekämpfung eingeführt wurden? Und wie gelingt es den Menschen in Deutschland neben der Arbeit die Betreuung ihrer Kinder zu organisieren wenn Kindertagesstätten und Schulen geschlossen sind? Zur Beantwortung dieser Fragen stellen wir die Ergebnisse der täglich durchgeführten Erhebung vom 20. März bis einschließlich 2. April 2020 vor. Wir zeigen, wer nach wie vor beim Arbeitgeber vor Ort arbeitet, wer im Home-Office und wer in Kurzarbeit. Wer ist von Freistellung oder sogar Arbeitslosigkeit betroffen? Insbesondere untersuchen wir, welche soziale Gruppen - also beispielsweise Personen mit wenig Bildung oder niedrigem Einkommen - von den wirtschaftlichen Auswirkungen der Krise betroffen sind. In Bezug auf die Kinderbetreuung differenzieren wir nach verschiedenen Betreuungsformen innerhalb und außerhalb des Haushalts. Hier untersuchen wir auch, wie sich Eltern die Betreuungsarbeit untereinander aufteilen