Questions Administered by Telephone or In Person: Differences in Interviewer-Respondent Interactions

When choosing a mode for data collection of computer-assisted surveys, a researcher has three main options available: the computer-assisted telephone interview (CATI), the computer-assisted personal interview (CAPI) or a web interview (i.e., a self-adminstered interview). Generally, CAPI allows for collecting most complex data, of the highest quality, but only when interviewers are well-trained and effort is made to monitor and manage interviewers during field work. This higher data quality in CAPI interviews may be due to the finding that presence of an interviewer reduces the amount of respondents’ satisficing behaviors (i.e., not investing the required effort to provide meaningful answers, see Heerwegh 2008). An interviewer can motivate respondents, presumably by means of rapport (Garbarski et al 2016) to invest the amount of cognitive effort required to answer questions thoughtfully. However, with interviewers administering the survey their social presence may also give rise to respondents’ reflection of social consequences of providing information to a relative stranger. By looking at response distributions it has been shown that social desirability bias and satisficing are more prevalent in CATI than in CAPI (see Holbrook et al 2003), and lowest in (self-administered) Web interviews. To fully explain such effects, it makes sense to study actual behaviors in interviewer-respondent interactions. In this approach, generally referred to as interaction coding or behavior coding (see Ongena and Dijkstra, 2006), interactions are systematically evaluated on deviations of the so-called paradigmatic sequence. A paradigmatic sequence is the interaction as intended by the researcher, with a sequence consisting of only two or three actions: the interviewer reading the question exactly as worded, the respondent providing an answer that exactly matches one of the response options, and optionally, an interviewer acknowledgement. Any deviation from this sequence may indicate problems in the questionnaire or the interviewing procedure. However, to our knowledge, CATI and CAPI interactions have never been systematically compared. By analyzing interactions of 60 CATI and 54 CAPI interviews that originated from a mixed-mode experiment using the European Social Survey questionnaire (Haan 2015), we found mixed differences with respect to behaviors in CATI and CAPI interactions. For example, interviewer laughter appeared to be more common in CATI than in CAPI, but apologetic utterances such as ‘sorry’ occurred equally often in both modes. Furthermore, a significant difference was found in the number of words uttered. Question-answer sequences contained more words in CATI than in CAPI. This is partly explained by the fact that for many questions in the CAPI survey show cards were used. Lack of show cards in CATI extends interactions due to less efficient communication about response alternatives. Further analysis showed that respondents in CATI had more difficulty in formulating their response than in CAPI. These task-related issues may contribute to the effect of decreased trust and motivation of respondents in CATI interviews, and may subsequently explain the increased level of satisficing and social desirability bias in this survey mode compared to CAPI

Chapter 12: Differences in interaction quantity and conversational flow in CAPI and CATI interviews. Appendix 12

Appendix 12A Differences in Length of Interviewer-Respondent Interactions in CAPI and CATI Interviews Table A12A.1 Response Rrates (AAPOR RR1) Table A12A.2 Descriptive Statistics for Questions and Respondent Characteristics Table A12A.3 Means and Correlations Trimmed number of Turns and Events in CAPI and CATI by Question Characteristics Table A12A.4 Means and Correlations Trimmed number of Turns and Events by Question Characteristic

An Evaluation study of the Event History Calender

To obtain valid and reliable data in survey interviews, it is generally acknowledged that the interviewer's behavior should be standardized in order to provide each respondent with the same stimulus and prevent the interviewer affecting the answers of the respondent (Fowler & Mangione, 1990). Conducting an interview using an event history calendar (EHC), however, usually requires the interviewer to be much more flexible. Although completely structured interviews with an EHC have been done (Furstenberg, Brooks-Gunn, & Morgan, 1987), the more common approach is to use a semistructured format with an initial prescribed question for each (sub)domain (Freedman, Thornton, Camburn, Alwin, & Young-DeMarco, 1988). In such an approach, the interviewer is required to react to the respondent's answers, which in turn may produce unwanted interviewer variance and thus may endanger the quality of the data. Thus, interviewers on the one hand have to pose the questions as worded in the questionnaire, but on the other hand have to probe, for example asking whether a particular situation has changed, using their own words and based on their own judgment given the answers of the respondent. The flexible approach is especially expected to improve data quality with regard to resolving inconsistencies and checking the respondent's memory (Belli, Shay, & Stafford, 2001). However, such freedom may cause the interviewer also to perform nonstandardized behavior in cases where standardized behavior may have been more appropriate (e.g., using suggestive probes; see Smit, Dijkstra, & Van der Zouwen, 1997). Moreover, administering an EHC is quite a complex task. Interviewers have to grasp the essential ideas underlying the EHC method and apply techniques like parallel probing in an adequate way, whereas filling in the calendar itself is much less straightforward than filling in a usual questionnaire from a survey interview. In this chapter we present two studies in the context of a large tobacco epidemiology study evaluating interviewer performance in administering an EHC. The first study concerns a paper-and-pencil version of the EHC; the second one a computer-assisted version. First we describe the underlying principles of the EHC and the task of the interviewer in more detail

Reaching Hard-to-Survey Populations: Mode Choice and Mode Preference

This study assesses the effect of response-mode choices on response rates, and responsemode preferences of hard-to-survey populations: young adults, full-time workers, big city inhabitants, and non-Western immigrants. Using address-based sampling, a stratified sample of 3,496 households was selected. The first group of sample members was contacted face to face and could choose between a CAPI and web response mode. The second group, contacted by telephone, could choose between CATI and web. The third group, contacted by telephone, was randomly allocated to a response mode. Our address-based sampling technique was successful in reaching most of the hard-to-survey groups. Insufficient numbers of non- Western immigrants were reached; therefore this group was excluded from our analyses. In our mixed-effect models, no significant effects on the willingness to participate were found for mode choice. We found that full-time workers and young adults were significantly more likely to choose web over CAPI when contacted face to face

Shaking hands in a busy waiting room:The effects of the surveyor’s introduction and people present in the waiting room on the response rate

Although waiting room surveys are frequently conducted, methodological studies on this topic are scarce. Behaviour of surveyors in waiting rooms can easily be controlled, and these surveys also allow for collection of paradata; relevant information on the circumstances of a request to participate in survey research. In this paper, we present the results of an experiment systematically manipulating surveyor’s handshakes and verbal introduction of their names. Patients scheduled for radiological examinations were approached to take part in a survey. An observer noted circumstances in the waiting room (CT or MRI) such as the number of people present.In the CT waiting room, willingness to participate was higher when no other people were filling out the survey than when there were other people filling out the survey. Thus, scarcity effects seemed to play a major role in the decision to participate. In addition, a patient waiting alone was more likely to fully complete the questionnaire, than patients accompanied by one or more caregivers. There was no effect of the surveyor’s handshake or verbal name introduction on survey participation, which is a fortunate outcome in light of social distances measures fighting COVID-19

Do alumni practise what you teach? Impact of Science master-tracks: preparation for academic careers versus preparation for societal-oriented careers

Career development of Science students seems dependent on more than disciplinary education, such as the nature and focus of career preparation. In this study we compare graduate outcomes of science students at the University of Groningen, completing two master-level program streams with different focal points, by using career progress surveys from alumni (n=242) with self-reported societal development data. The first program is a classical research-oriented master (ROM), that aims to prepare students for a career within academia. The second program, called Science, Business and Policy (SBP) focusses on a societal profile and aims to prepare for a career outside academia. SBP is different from ROM because it has a practical and multidisciplinary orientation (vs a theoretical and mono-disciplinary orientation in ROM) and it includes a long work-placement outside academia, a form of work based learning. The differences in the tracks translate to differences in career paths. Both profiles resulted in good but distinct career opportunities, corresponding to alumni’s discipline and sector. Although ROM-alumni felt especially prepared for an academic career, only a quarter actually continued in academia and many we employed in education. In contrast, most SBP-alumni were employed in business or policy. Targeting specific job preparation seems more successful with a societal profile. Reflecting on skills, in both groups research-related skills decreased during careers while soft skills increased. Alumni were generally satisfied with their first job and also with their current job, with SBP-alumni scoring significantly higher on satisfaction with income, status, appreciation and perspective. It can be concluded that diversification in educational profiles changes the societal career paths of science students