Postal Surveys (Version 2.0)

Postal surveys are an important data collection mode in social science survey research. One key task when implementing postal surveys is to motivate the potential respondents to participate. This is done by means of the cover letter, the design of the questionnaire, and repeated contacts. Methodological aspects of the implementation of postal surveys are discussed against the background of theories that explain target persons’ willingness to participate

Rating-Scale Labeling in Online Surveys: An Experimental Comparison of Verbal and Numeric Rating Scales with Respect to Measurement Quality and Respondents' Cognitive Processes

Unlike other data collection modes, the effect of labeling rating scales on reliability and validity, as relevant aspects of measurement quality, has seldom been addressed in online surveys. In this study, verbal and numeric rating scales were compared in split-ballot online survey experiments. In the first experiment, respondents’ cognitive processes were observed by means of eye tracking, that is, determining the respondent’s fixations in different areas of the screen. In the remaining experiments, data for reliability and validity analysis were collected from a German adult sample. The results show that respondents needed more fixations and more time to endorse a category when a rating scale had numeric labels. Cross-sectional reliability was lower and some hypotheses with respect to the criterion validity could not be supported when numeric rating scales were used. In conclusion, theoretical considerations and the empirical results contradict the current broad usage of numeric scales in online surveys

Design of Rating Scales in Questionnaires (Version 2.0)

Rating scales are among the most important and most frequently used instruments in social science data collection. There is an extensive body of methodological research on the design of rating scales and on their (psycho-)metric properties. In this contribution, the authors address individual aspects of questionnaire construction with regard to rating scales. A brief overview of the current state of research is given, a number of accounts of practical experiences are presented, and - to the extent that this is possible - recommendations for the design of rating scales are given

Question Wording (Version 2.0)

This contribution provides an overview of the basic rules that should be observed when formulating survey questions. The structure of the chapter is oriented towards the cognitive response process that respondents undergo when answering survey questions. This process involves a total of four tasks: understanding the question text, retrieving information from memory, forming a judgment, and formulating a response in the format provided. Using practical examples, the authors demonstrate how questions can be formulated in such a way that it makes it easier for respondents to handle these four tasks

Reasons for refusals, their collection in surveys and interviewer impact

"Refusals are a significant source of non-response in surveys. During field periods of some surveys reasons for refusals are collected in call record data (as part of para-data). This article presents a study employing a content analysis of open-ended comments on reasons for refusals collected by interviewers in a survey of the German population (ALLBUS). We analysed the reasons for refusals contained in these comments, as well as to what extent these comments include information about factors relevant to participation in surveys. Additionally, we analysed the impact of interviewer characteristics – gender, age, education and experience – on data collection using various multilevel multinomial models. The results show that interviewer comments provide typical reasons for refusals, as well as specific information about target persons, their environment and the survey process. Interviewers' age and education influenced the collection of reasons for refusals. At the same time interviewer variances (obtained through multinomial multilevel models) were very high, showing that interviewers prefer to report certain reasons for refusals. The highest interviewer level variances were obtained for providing no comments at all. To improve data quality and reduce high interviewer impact, we suggest using improved standardised instruments to collect reasons for refusals. Codings based on a categorisation scheme which we developed for our content analysis show high reliability (kappa = .81). Thus, this scheme can be used as a basis for developing such standardised instruments." (author's abstract

Schriftlich-postalische Befragung (Version 1.1)

Schriftlich-postalische Befragungen stellen eine wichtige Art der Datenerhebung in der sozialwissenschaftlichen Umfrageforschung dar. Eine zentrale Aufgabe bei der Durchführung schriftlichpostalischer Befragungen besteht darin, die potentiellen Befragten zur Teilnahme zu motivieren, was durch das Anschreiben, die Gestaltung des Fragebogens und die wiederholten Kontakte erfolgt. Vor dem Hintergrund der Theorien, die die Teilnahmebereitschaft erklären, werden methodische Aspekte bei der Durchführung schriftlich-postalischer Befragungen erörtert

Standardisierte Erfassung von Verweigerungsgründen in face-to-face-Umfragen

"Verweigerungen stellen einen beträchtlichen Anteil systematischer Ausfälle in Umfragen dar. Einige Umfragen erheben Verweigerungsgründe während des Datenerhebungsprozesses. Neben einer Zusatzinformation zum Ablauf der Datenerhebung liefern diese Daten auch Informationen, die zur Reduzierung der Verweigerungsraten genutzt werden können. Allerdings fehlen derzeit standardisierte Instrumente zur Erfassung von Verweigerungsgründen, die eine zuverlässige Datenerhebung sichern würden. In diesem Artikel stellen wir ein standardisiertes Instrument – ein Kategorienschema – zur Klassifizierung der Verweigerungsgründe vor, das als Teil einer Inhaltsanalyse entwickelt wurde. Als Datenbasis nutzten wir offen erhobene Angaben der Interviewer in Kontaktprotokollen im ALLBUS 2008. Die Intercoder-Reliabilität der Codierung beträgt 0,84 und ist zufriedenstellend. Abschließend geben wir Hinweise zur Nutzung des Kategorienschemas für die Codierung der offen erhobenen Daten sowie zur Datenerhebung direkt durch den Interviewer im Feld." (Autorenreferat)"Refusals are a considerable source of nonresponse in surveys. During the field period, some surveys collect reasons for refusals as survey para-data. In addition to providing information for research these data can also provide information about nonrespondents that can be used to reduce refusal rates. However, there is a lack of standardized instruments of data collection, the fact of which declines reliability, validity, and objectivity of the data collected. This article presents a standardized instrument – a categorization scheme – for classifying reasons for refusals. The instrument was developed using content analysis of open-ended comments by interviewers in the German General Social Survey (ALLBUS 2008). The interrater-reliability of the developed categorization scheme is .84 and thus satisfying. We give some suggestions on how the categorization scheme can be used by surveyors as well as by interviewers when collecting reasons for refusals." (author's abstract

Internet of Things: Cognitive Online-Pretest - October-November 2018

The Pretest lab of GESIS Leibniz Institute for the Social Sciences (Germany) was assigned the task to pretest a module on the Internet of Things (IoT) to be included, if deemed appropriate, in future waves of the Community Survey on ICT Use in Households and by lndividuals conducted by the European Commission. The aim of the cognitive pretest was to optimize the questions prior to their potential administration in the actual survey and thereby to improve data quality. The inclusion of a module on IoT in the Community Survey on ICT Use in Households and by lndividuals is of interest to DG CNECT in order to monitor the evolution and adoption of recent technological solutions in this domain. GESIS tested an English and a German version of the module with British and German respondents (n=120) in a web survey. In total, respondents received open and closed probes for four items. Some of the items do not need to be changed but the results indicate for a few questions that the answer op-tions, the question order, or the German translations need to be improved. The pretest report includes recommendation to further refine the questions of the module

Frageformulierung (Version 1.1)

Dieser Beitrag gibt einen Überblick über grundsätzliche Regeln, die bei der Formulierung von Survey-Fragen berücksichtigt werden sollten. Die Gliederung des Kapitels orientiert sich dabei am kognitiven Antwortprozess, den Befragte bei der Beantwortung von Survey-Fragen durchlaufen und der insgesamt vier Aufgaben umfasst: Verstehen des Fragetextes, Informationsgewinnung, Urteilsbildung und Formatierung der Antwort. Anhand von praktischen Beispielen wird illustriert, wie man mit Hilfe der Frageformulierung Befragten die Bewältigung dieser vier Aufgaben erleichtern kann