Web Paradata in Survey Research (Version 1.0)

Paradata are information about the primary survey data collection process. This guide is intended for survey practitioners who want to collect and use paradata in web surveys. The guideline focuses on a typology and possible applications of web paradata and practical implications regarding the collection, post-processing, and documentation of web paradata.Paradaten sind Informationen über den Erhebungsprozess von Umfragedaten. Dieser Leitfaden richtet sich an Umfragepraktiker, die Paradaten in Online-Umfragen erheben und auswerten möchten. Der Leitfaden konzentriert sich dabei auf die Typologie und mögliche Anwendungen von Web-Paradaten und praktische Implikationen hinsichtlich der Sammlung, Nachbearbeitung und Dokumentation von Web-Paradaten

Context effects in question evaluation via web probing: exploring the interaction of open-ended and closed survey questions

Web probing is a method of question evaluation that employs probing techniques from cognitive interviews in self-administered web surveys. Context effects describe when a question produces a different answer depending on the context in which it is asked, such as the order of the questions. In web probing, the order of survey and probing questions may impact the data collected by both question types. This thesis establishes a model of context effects in web probing, examines it in a series of empirical studies, and discusses the implications for state-of-the art question evaluation via web probing

An experimental test of the effectiveness of cognitive interviewing in pretesting questionnaires

Pretesting survey questions via cognitive interviewing is based on the assumptions that the problems identified by the method truly exist in a later survey and that question revisions based on cognitive interviewing findings produce higher-quality data than the original questions. In this study, we empirically tested these assumptions in a web survey experiment (n = 2,200). Respondents received one of two versions of a question on self-reported financial knowledge: either the original draft version, which was pretested in ten cognitive interviews, or a revised version, which was modified based on the results of the cognitive interviews. We examined whether the cognitive interviewing findings predicted problems encountered in the web survey and whether the revised question version was associated with higher content-related and criterion-related validity than the draft version. The results show that cognitive interviewing is effective in identifying real question problems, but not necessarily in fixing survey questions and improving data quality. Overall, our findings point to the importance of using iterative pretesting designs, that is, carrying out multiple rounds of cognitive interviews and also testing the revisions to ensure that they are indeed of higher quality than the draft questions

Questionnaire design decisions when transitioning from an interviewer-administered to a self-administered online mode (Version 1.0)

Large-scale surveys are increasingly moving from (face-to-face or telephone-based) interviewer-administered to self-administered online modes. To ensure high measurement quality and maximum comparability and equivalence between the source questionnaire and its adaptation - across modes as well as across survey waves - various aspects of question design must be considered and several decisions need to be made concerning question presentation and wording. This survey guideline summarizes good practices on how to transition questionnaires from interviewer- to self-administered web surveys and gives recommendations and examples for major adaptation issues as well as general questionnaire design elements relevant to web surveys. In this context, we focus on the switch from an interviewer-based to an online mode (although mixed-mode designs are also conceivable).Groß angelegte Studien werden zunehmend von interviewerbasierten (persönlichen oder telefonischen) Befragungen auf selbstverwaltete Onlineerhebungen umgestellt. Um eine hohe Datenqualität und ein Höchstmaß an Vergleichbarkeit und Äquivalenz zwischen dem Ausgangsfragebogen und seiner Adaptation - sowohl über verschiedenen Modi als auch Erhebungswellen hinwe - zu gewährleisten, müssen verschiedene Aspekte des Fragebogendesigns berücksichtigt und eine Reihe von Entscheidungen in Bezug auf die Präsentation und Formulierung von Fragen getroffen werden. Dieser Erhebungsleitfaden fasst bewährte Methoden zur Umwandlung der Fragebögen von interviewgestützten zu selbstgestützten Onlineerhebungen zusammen und gibt Empfehlungen und Beispiele für wichtige Anpassungsfragen sowie zu allgemeinen Gestaltungselementen von Fragebögen, die für Onlineumfragen relevant sind. In diesem Zusammenhang konzentrieren wir uns auf den Wechsel von einem interviewergestützten zu einem Onlinemodus (obwohl auch gemischte Modi denkbar sind)

Islamist attitudes among young Muslims in Germany: Cognitive Pretest

Radicalized Islamist attitudes or expressions of opinion do not automatically lead to corresponding violent acts among young people from Islamist socialization milieus. But they contribute to the legitimacy of violent activities, because any violence against other groups is always dependent on such patterns of attitude, which imply a devaluation of foreign groups, which is given by the radicalized Islamist attitudes towards persons of other faiths. The research project "Islamist attitudes among young Muslims in Germany" examines the patterns and social backgrounds, including the actors who promote such legitimation-providing attitudes among adolescents. In order to prepare the survey, selected parts of the questionnaire should be subjected to a cognitive pretest under methodological and questionnaire-related aspects and revised based on the test results. The questionnaire was developed in German and translated into Turkish and Arabic, among other languages. The aim of the cognitive pretest was to test both the original questionnaire and the translations. To test the Turkish and Arabic questionnaires, simultaneous interpreters were to be added to the cognitive interviews via video conference. For this purpose, the GESIS pretest laboratory was commissioned to carry out the cognitive pretest

Internet of Things: Cognitive Online-Pretest - October-November 2018

The Pretest lab of GESIS Leibniz Institute for the Social Sciences (Germany) was assigned the task to pretest a module on the Internet of Things (IoT) to be included, if deemed appropriate, in future waves of the Community Survey on ICT Use in Households and by lndividuals conducted by the European Commission. The aim of the cognitive pretest was to optimize the questions prior to their potential administration in the actual survey and thereby to improve data quality. The inclusion of a module on IoT in the Community Survey on ICT Use in Households and by lndividuals is of interest to DG CNECT in order to monitor the evolution and adoption of recent technological solutions in this domain. GESIS tested an English and a German version of the module with British and German respondents (n=120) in a web survey. In total, respondents received open and closed probes for four items. Some of the items do not need to be changed but the results indicate for a few questions that the answer op-tions, the question order, or the German translations need to be improved. The pretest report includes recommendation to further refine the questions of the module

European Working Conditions Survey (EWCS): Cognitive Pretest

This report contains the results of the post test of the 6th EWCS, carried out by GESIS - Leibniz Institute for the Social Sciences (Germany). The aim is to deliver insights for the interpretation of the re-sults of the 6th EWCS, and to deliver input for the development and revision of the 7th EWCS. Two major research questions are addressed. The first is to establish differences in comprehension of questions between employed and self-employed respondents. Secondly, it examines potential measurement issues related to the concept of employment and the main paid job criterion for workers in atypical working situations. For the post test a mixed-mode design was used. A cognitive online pretest in the UK, Germany, and Poland was carried out with a total of 365 employed and self-employed respondents. Face-to-face interviews with 32 respondents in Germany and Poland delivered insights on atypical working situa-tions, such as multi-activity or precarious employment

European Working Conditions Survey 2024: Cognitive Pretest

Eurofound is preparing to transition the European Working Conditions Survey (EWCS) from a computer-assisted personal interview (CAPI) to self-administered online data collection. GESIS was contracted to adapt the CAPI master questionnaire, translate the adapted online questionnaire into German and Polish, and cognitively pretest the online questionnaire using cross-cultural cognitive interviews and web probing. The main goal of cognitive testing was assessing the comparability of the adapted online questionnaire to the original face-to-face questionnaire. Further objectives were testing whether the online questionnaire is suitable for cross-cultural research and applies gender-sensitive language

Translation of established public health measurement instruments into Arabic and Dari (ENSURE)

Im Rahmen des DFG-geförderten Forschungsprojektes "Ensuring Valid Comparisons of Self-Reports in Heterogenous Populations and Marginalized Groups (ENSURE)" (Projektnummer 409654512) an der Professur für empirische Sozialforschung (Institut für Soziologie, Technische Universität Dresden) werden bereits etablierte Messinstrumente quantitativer Befragungen in der Gesundheitsforschung einem kognitiven Pretest unterzogen, um sicherzustellen, dass sie auch in anderen Sprach- und Kulturkreisen eingesetzt werden können (sprach- und kulturübergreifende Messinvarianz). Die Zielgruppe stellen in Deutschland lebenden Personen mit Fluchterfahrung (Fluchtmigration) aus Syrien, dem Irak und Afghanistan dar. Getestet wurden die Fragen mithilfe von kognitiven Remote-Interviews unter Einsatz von per Video zugeschalteten Konsekutivdolmetschern in den Sprachen arabisch und dari