Modelling final outcome and length of call sequence to improve efficiency in interviewer call scheduling

Survey practitioners are increasingly interested in how best to use paradata to improve data collection processes. One particular question is if it is possible to identify early on during fieldwork sample cases that may require a long time, and therefore a lot of financial and staff resources, until interviewing is completed. More specifically, we aim to identify cases with long unsuccessful call sequences. This paper models call record data predicting final call outcome and length of a call sequence. Separate binary and joint multinomial logistic models for the two outcomes are presented, accounting for the clustering of households within interviewers. Of particular interest is to identify explanatory variables that predict final outcome and length of a call sequence. The study uses data from Understanding Society, a large-scale UK longitudinal survey. The work has implications for responsive and adaptive survey designs. The results indicate that modelling outcome and length of a call sequence jointly improves the fit of the model. Outcomes of previous calls, in particular from the most recent call, are highly predictive. The timing of calls and interviewer observation variables, although significant in the models, only slightly improve the predictive power

Using prior wave information and paradata: Can they help to predict response outcomes and call sequence length in a longitudinal study?

In recent years the use of paradata for nonresponse investigations has risen significantly. One key question is how useful paradata, including call record data and interviewer observations, from the current and previous waves of a longitudinal study, as well as previous wave survey information, are in predicting response outcomes in a longitudinal context. This paper aims to address this question. Final response outcome and sequence length (the number of calls/visits to a household) are modelled both separately and jointly for a longitudinal study. Being able to predict length of call sequence and response can help to improve both adaptive and responsive survey designs and to increase efficiency and effectiveness of call scheduling. The paper also identifies the impact of different methodological specifications of the models, for example different specifications of the response outcomes. Latent class analysis is used as one of the approaches to summarise call outcomes in sequences. To assess and compare the models in their ability to predict, indicators derived from classification tables, ROC (Receiver Operating Curves), discrimination and prediction are proposed in addition to the standard approach of using the pseudo R2 value, which is not a sufficient indicator on its own. The study uses data from Understanding Society, a large-scale longitudinal survey in the UK. The findings indicate that basic models (including geographic, design and survey data from the previous wave), although commonly used in predicting and adjusting for nonresponse, do not predict the response outcome well. Conditioning on previous wave paradata, including call record data, interviewer observation data and indicators of change, improve the fit of the models. A significant improvement can be observed when conditioning on the most recent call outcome, which may indicate that the nonresponse process predominantly depends on the most current circumstances of a sample unit

Disentangling the complex association between female genital cutting and HIV among Kenyan women

Female genital cutting (FGC) is a widespread cultural practice in Africa and the Middle East, with a number of potential adverse health consequences for women. It was hypothesised by Kun (1997) that FGC increases the risk of HIV transmission through a number of different mechanisms. Using the 2003 data from the Kenyan Demographic and Health Survey (KDHS), this study investigates the potential association between FGC and HIV. The 2003 KDHS provides a unique opportunity to link the HIV test results with a large number of demographic, social, economic and behavioural characteristics of women, including women’s FGC status. It is hypothesised that FGC increases the risk of HIV infection if HIV/AIDS is present in the community. A multilevel binary logistic regression technique is used to model the HIV status of women, controlling for selected individual characteristics of women and interaction effects. The results demonstrate evidence of a statistically significant association between FGC and HIV, after controlling for the hierarchical structure of the data, potential confounding factors, and interaction effects. The results show that women who had had FGC and a younger or the same age first union partner have higher odds of being HIV positive than women with a younger or same age first union partner but without FGC; whereas women who had had FGC and an older first union partner have lower odds of being HIV positive than women with an older first union partner but without FGC. The findings suggest the behavioural pathway of association between FGC and HIV as well as an underlying complex interplay of bio-behavioural and social variables being important in disentangling the association between FGC and HIV

The interviewer contribution to variability in response times in face-to-face interview surveys

Survey researchers have consistently found that interviewers make a small but systematic contribution to variability in response times. However, we know little about what the characteristics of interviewers are that lead to this effect. In this study, we address this gap in understanding by linking item-level response times from wave 3 of the UK Household Longitudinal Survey (UKHLS) to data from an independently conducted survey of interviewers. The linked data file contains over three million records and has a complex, hierarchical structure with response latencies nested within respondents and questions, which are themselves nested within interviewers and areas. We propose the use of a cross-classified mixed-effects location scale model to allow for the decomposition of the joint effects on response times of interviewers, areas, questions, and respondents. We evaluate how interviewer demographic characteristics, personality, and attitudes to surveys and to interviewing affect the length of response latencies and present a new method for producing interviewer-specific intra-class correlations of response times. Hence, the study makes both methodological and substantive contributions to the investigation of response times

Do respondents using smartphones produce lower quality data? Evidence from the UK Understanding Society mixed-device survey

We live in a digital age with high level of use of technologies. Surveys have started adopting technologies including smartphones for data collection. There is a move towards online data collection in the UK, including an ambition to collect 75% of household responses online in the UK 2021 Census. Major social household surveys in the UK have either transitioned to online data collection or are in the process of preparation for the transitioning. The Covid-19 pandemic forced rapid transitions to online data collection for many social surveys globally, with this mode of data collection being the only possibility at the moment. There are still concerns regarding allowing respondents to use smartphones to respond to surveys and not much is known about data quality produced by respondents using smartphones for survey completion in the UK context. This paper uses the first available in the UK, large scale mixed-device survey, Understanding Society Wave 8 where 40% of the sample were assigned to online mode of data collection. It allows comparison of data quality between different devices within the online mode of data collection with a special focus on smartphones. This analysis is very timely and fills the gap in knowledge. Descriptive analysis and then various regressions are used depending on the outcome variables to study data quality indicators associated with different devices in the online part of the survey. The following data quality indicators are assessed: break-off rates, item nonresponse, response style indicators, completion times, differential reporting indicators including self-reporting of risky behaviours, and consent to data linkage. Comparisons to limited results available in the UK are drawn. The results suggest that even in the context of non-optimised for smartphone questionnaire, we should not be concerned about respondents using smartphones for future social surveys, even for longer surveys such as the Understanding Society, as break off rates are very low and data quality between devices is not very different

Do interviewers moderate the effect of monetary incentives on response rates in household interview surveys?

As citizens around the world become ever more reluctant to respond to survey interview requests, incentives are playing an increasingly important role in maintaining response rates. In face-to-face surveys, interviewers are the key conduit of information about the existence and level of any incentive offered and, therefore, potentially moderate the effectiveness with which an incentive translates nonproductive addresses into interviews. Yet, while the existing literature on the effects of incentives on response rates is substantial, little is currently known about the role of interviewers in determining whether or not incentives are effective. In this article, we apply multilevel models to three different face-to-face interview surveys from the United Kingdom, which vary in their sample designs and incentive levels, to assess whether some interviewers are more successful than others in using incentives to leverage cooperation. Additionally, we link the response outcome data to measures of interviewer characteristics to investigate whether interviewer variability on this dimension is systematically related to level of experience and demographic characteristics. Our results show significant and substantial variability between interviewers in the effectiveness of monetary incentives on the probability of cooperation across all three surveys. However, none of the interviewer characteristics considered are significantly associated with more or less successful interviewers

Mixed-device online surveys in the UK

There is a move towards online data collection in the UK, including the plan to collect 75% of responses online in the 2021 Census. Online survey response is complicated by respondents using different devices. So far, no research has been conducted in the UK to study characteristics of people using different devices in mixed-device online surveys. This analysis uses all publicly available UK social surveys with an online component: Understanding Society Innovation Panel, Community Life Survey, European Social Survey, 1958 National Child Development Study, and the Second Longitudinal Study of Young People in England. Bivariate analysis and logistic regressions are used to study significant correlates of device use in online surveys. The results of bivariate analysis suggest that age, gender, marital status, employment status, religion, household size, children in household, household income, number of cars, and frequency of internet use are significantly associated with device used across surveys. The associations with age, gender, employment status, household size and education are consistent with the findings from other countries. The knowledge about characteristics of respondents using different devices in online surveys in the UK will help to understand better the response process in online surveys and to target certain subgroups more effectively

Survey Data Collection Network (SDC-Net): The impact of Covid-19 on survey data collection methods in the social sciences

This is the final report of the Survey Data Collection Network (SDC-Net). SDC-Net was a network of UK-based academic and non-academic partners including government departments, third sector and commercial research organisations, academics and major ESRC investments to share knowledge and collaborate in the area of survey data collection in social surveys as well as in setting the research agenda in the field. The network operated between December 2021 and April 2023. The Principal Investigator was Olga Maslovskaya (University of Southampton) and the Co-Investigators are Gabriele Durrant (University of Southampton and NCRM), Lisa Calderwood (UCL), Gerry Nicolaas (NatCen) and Laura Wilson (ONS). The network activities were funded by the ESRC via the project “The impact of Covid-19 on survey data collection methods in the Social Sciences” as an additional funding stream of the ESRC-funded UK National Centre for Research Methods (NCRM). The network included 107 members. The list of the organisations of the network members can be found in Appendix 1. Tim Hanson, who is the Head of ESS Questionnaire Design and Fieldwork in the European Social Survey (ESS), Ben Humberstone, who is the Head of Population Studies in Kantar Public, Sam Clemens, who is the Head of Probability Survey in Ipsos-Mori as well as Debrah Harding, who is the Managing Director of the Market Research Society (MRS), were project partners. The ESRC recognised the importance of the activities of the previous network GenPopWeb2 which was also funded by the ESRC and the activities of SDC-Net were the continuation of the GenPopWeb2 with the wider scope addressing not only issues associated with online data collection in social surveys but the wider area of survey data collection in the UK