Impact of mixed modes on measurement errors and estimates of change in panel data

Mixed mode designs are receiving increased interest as a possible solution for saving costs in panel surveys, although the lasting effects on data quality are unknown. To better understand the effects of mixed mode designs on panel data we will examine its impact on random and systematic error and on estimates of change. The SF12, a health scale, in the Understanding Society Innovation Panel is used for the analysis. Results indicate that only one variable out of 12 has systematic differences due to the mixed mode design. Also, four of the 12 items overestimate variance of change in time in the mixed mode design. We conclude that using a mixed mode approach leads to minor measurement differences but it can result in the overestimation of individual change compared to a single mode design

Evaluating mode differences in longitudinal data. Moving to a mixed mode paradigm of survey methodology

Collecting and combining data using multiple modes of interview (e.g., face-to- face, telephone, Web) is becoming common practice in survey agencies. This is also true for longitudinal studies, a special type of survey that applies questionnaires repeatedly to the same respondents. In this PhD I investigate if and how collecting information using different modes can impact data quality in panel studies. Chapters 2 and 3 investigate how a sequential telephone - face-to-face mixed mode design can bias reliability, validity and estimates of change compared to a single mode. In order to achieve this goal I have used an experimental design from the Understanding Society Innovation Panel. The analyses have shown that there are only small differences in reliability and validity between the two modes but estimates of change might be overestimated in the mixed modes design. Chapter 4 investigates the measurement differences between face-to-face, telephone and Web on three scales: depression, physical activity and religiosity. We use a quasi-experimental (cross-over) design in the Health and Retirement Study. The results indicate systematic differences between interviewer modes and Web. We propose social desirability and recency as possible explanations. In Chapter 5 we investigate using the Understanding Innovation Panel if the extra contact by email leads to increased propensity to participate in a sequential Web - face-to-face design. Using the experimental nature of our data we show that the extra contact by email in the mixed mode survey does not increase participation likelihood. One of the main difficulties in the research of (mixed) modes designs is separating the effects of selection and measurement of the modes. Chapter 6 tackles this issue by proposing equivalence testing, a statistical approach to control for measurement differences across groups, as a front-door approach to disentangle these two. A simulation study shows that this approach works and highlights the bias when the two main assumptions don't hold

King Carol I and Crown Prince Ferdinand’s Visit to Craiova (1-3 October 1890)

The end of the 19th century, more precisely the year 1890, represented for the people of Craiova the opportunity to witness a new royal visit, this time a very special one, because for the first time the Crown Prince Ferdinand visited the city of Craiova with his uncle, King Carol I. The visit lasted 3 days, 1–3 October, being perhaps one of the most eventful visits, at least in terms of the very busy schedule. Many schools were visited, both girls’ and boys’ schools, as well as hospitals or prisons were among the objectives of the visit. The Craiova society was involved in the big event, with demonstrations taking place in which many people participated. There was also criticism in the anti–dynastic press of the time, many aspects of the organization of the visit being attacked. The visit remains one of reference for Craiova, the magnitude of the event echoing in the press and the documents of the time

The Impact of Nurse Continuity on Biosocial Survey Participation

Biological measurements (or biomeasures) are increasingly being collected in large longitudinal biosocial surveys, enabling researchers to exploit the advantages of social science data with objective health measures to better understand how health and social behaviour interact over time. However, not all survey respondents are willing to take part in the biomeasure component of biosocial surveys, even when the measures are administered by certified medical professionals, such as nurses. Thus, understanding factors which affect participation in biomeasure collection is essential for making valid biosocial inferences about the population. Previous research has shown that interviewer continuity can be useful for optimizing longitudinal survey participation, but it is yet unknown if nurse continuity impacts the likelihood of participation in biomeasure collection. We investigated the impact of nurse continuity on nonresponse to biomeasure collection in waves 4 and 6 of the English Longitudinal Study of Ageing (ELSA). Using cross-classified multilevel models, we find that switching nurses between waves does not negatively impact participation in biomeasure collection, and sometimes can be beneficial, particularly for previous wave nonrespondents. The practical implication is that biosocial surveys may not need to employ strict nurse continuity protocols to maximize participation in subsequent waves of biomeasure data collection

Chapter 16: Investigating the Use of Nurse Paradata in Understanding Nonresponse to Biological Data Collection. Appendix 16

Appendix 16A: Descriptive statistics for available call records Figure A16A.1 Histograms of wave 2 nurse visit call length by call status with outlier of 691 minutes removed from the ‘any interviewing done’ category Figure A16A.2 Histograms of wave 3 nurse visit call length by call status with outlier of 464 minutes removed from the ‘any interviewing done’ category Figure A16A.3 Bar chart of total number of calls per household at UKHLS wave 2 nurse visit Figure A16A.4 Bar chart of total number of calls per household at UKHLS wave 3 nurse visit Figure A16A.5 Histogram of total nurse visit time in minutes – wave 2 Figure A16A.6 Histogram of total nurse visit time in minutes minus the blood sample modules – wave 2 Appendix 16B: Full Results from Regression Models Corresponding to Tables 16.4-16.12 in Main Body Table A16B.1 Results from cross-classified multilevel logistic regression models for the log-odds of participating in the wave 2 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.4 in main body Table A16B.2 Results from cross-classified logistic regression models of the log-odds of consenting to the blood sample in the wave 2 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.5 in main body Table A16B.3 Results from cross-classified logistic regression models of the log-odds of obtaining the blood sample in the wave 2 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.6 in main body. Table A16.4 Results from cross-classified logistic regression models of the log-odds of participating in the wave 3 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.7 in main body. Table A16B.5 Results from cross-classified logistic regression models of the log-odds of consenting to the blood sample in the wave 3 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.8 in main body. Table A16B.6 Results from cross-classified multilevel logistic regression models of the log-odds of obtaining the blood sample in the wave 3 nurse visit using nurse characteristics and paradata variables. Corresponds to table 16.9 in main body. Table A16B.7 Results from cross-classified logistic regression models of the log-odds of participating in the wave 3 nurse visit using nurse performance indicators. Corresponds to table 16.10 in main body. Table A16B.8 Results from cross-classified logistic regression models of the log-odds of consenting to the blood sample in the wave 3 nurse visit using nurse performance indicators. Corresponds to table 16.11 in main body. Table A16B.9 Results from cross-classified logistic regression models of the log-odds of obtaining the blood sample in the wave 3 nurse visit using nurse performance indicators. Corresponds to table 16.12 in main body. Appendix 16C: Quantile-Quantile plot of nurse residuals for the model of having call records in wave 2 Figure A16C.1 Quantile-Quantile plot of nurse residuals with 95% confidence intervals from the model of having call records in wave 2. The green oval indicates the nurses with above-average call recording and the red oval indicates the nurses with below-average call recording

Investigating the Use of Nurse Paradata in Understanding Nonresponse to Biological Data Collection

The recent collection of biological data in large-scale sample surveys has opened up new possibilities for research into the interactions between physical and social mechanisms in the general population. Whilst the possibilities are undoubtedly exciting, these data can create additional challenges from the viewpoints of both collection and analysis. In particular, the extra burden of biological data collection can lead to increased incidences of nonresponse, potentially affecting the quality of the data and the robustness of results from subsequent analysis. Where the two-stage nurse visit survey design is used, such as in Understanding Society (UKHLS) and the English Longitudinal Study of Ageing (ELSA), the possible effects of the assigned nurse on patterns of nonresponse also need to be considered. In order to address nonresponse to biological data collection, researchers can build response propensity models and use these to adapt future data collection and/or produce post-survey adjustments such as weights. However, variables included in the models must be available for both respondents and nonrespondents in the sample. A recent branch of research concerns the use of new forms of data collected during the survey process, known as paradata, for this purpose. Paradata can include variables such as call histories, response timings and interviewer observations, and may provide a cost-effective way to deal with nonresponse in analyses which use survey data. In this paper, we use paradata collected during the nurse visit in UKHLS to investigate nonresponse to biological data collection and also to examine and explain the effects of the nurse. Preliminary results from UKHLS wave 2 have shown that clustering by nurse is present in responses to three conditional stages: participation in the nurse visit, consent to the blood sample and obtaining the blood sample. When quantifying the nurse effects, we estimated cross-classified multilevel models for the likelihood to respond to each of these stages to take account of the effects of geographical area. The models included a comprehensive set of explanatory variables for the respondents and households as well as data from call records. Given the amount of missing paradata, we also used the availability of call record data to estimate nurse performance indicators that could be used in response propensity models for the following wave of biological data collection. Results indicate that the nurse performance indicators derived from the availability of call record paradata are predictive of the likelihood of a sample member to participate in the first stage of the nurse visit in the following wave, given nurse age and experience. This was not the case for the subsequent two stages (consent to and obtaining the blood sample). This suggests that the recording of call histories by nurses in surveys may be an indication of their performance in the interviewer-type tasks involved in biological data collection, such as making contact and gaining cooperation from sample members. It also shows how these new forms of data may help to explain nurse effects in models of response to biological data collection

Interviewer Effects in Biosocial Survey Measurements

From SAGE Publishing via Jisc Publications RouterHistory: epub 2021-03-11Publication status: PublishedIncreasingly surveys are using interviewers to collect objective health measures, also known as biomeasures, to replace or supplement traditional self-reported health measures. However, the extent to which interviewers affect the (im)precision of biomeasurements is largely unknown. This article investigates interviewer effects on several biomeasures collected in three waves of the National Social Life, Health, and Aging Project (NSHAP). Overall, we find low levels of interviewer effects, on average. This nevertheless hides important variation with touch sensory tests being especially high with 30% interviewer variation, and smell tests and timed balance/walk/chair stands having moderate interviewer variation of around 10%. Accounting for contextual variables that potentially interact with interviewer performance, including housing unit type and presence of a third person, failed to explain the interviewer variation. A discussion of these findings, their potential causes, and their implications for survey practice is provided

Estimating stochastic survey response errors using the multitrait‐multierror model

From Wiley via Jisc Publications RouterHistory: received 2018-09-17, rev-recd 2021-01-26, accepted 2021-05-30, pub-electronic 2021-10-12Article version: VoRPublication status: PublishedFunder: ESRC National Centre for Research Methods, University of Southampton; Id: http://dx.doi.org/10.13039/501100000613; Grant(s): R121711Abstract: Surveys are well known to contain response errors of different types, including acquiescence, social desirability, common method variance and random error simultaneously. Nevertheless, a single error source at a time is all that most methods developed to estimate and correct for such errors consider in practice. Consequently, estimation of response errors is inefficient, their relative importance is unknown and the optimal question format may not be discoverable. To remedy this situation, we demonstrate how multiple types of errors can be estimated concurrently with the recently introduced ‘multitrait‐multierror’ (MTME) approach. MTME combines the theory of design of experiments with latent variable modelling to estimate response error variances of different error types simultaneously. This allows researchers to evaluate which errors are most impactful, and aids in the discovery of optimal question formats. We apply this approach using representative data from the United Kingdom to six survey items measuring attitudes towards immigrants that are commonly used across public opinion studies

Estimating stochastic survey response errors using the multitrait‐multierror model

Surveys are well known to contain response errors of different types, including acquiescence, social desirability, common method variance and random error simultaneously. Nevertheless, a single error source at a time is all that most methods developed to estimate and correct for such errors consider in practice. Consequently, estimation of response errors is inefficient, their relative importance is unknown and the optimal question format may not be discoverable. To remedy this situation, we demonstrate how multiple types of errors can be estimated concurrently with the recently introduced ‘multitrait-multierror’ (MTME) approach. MTME combines the theory of design of experiments with latent variable modelling to estimate response error variances of different error types simultaneously. This allows researchers to evaluate which errors are most impactful, and aids in the discovery of optimal question formats. We apply this approach using representative data from the United Kingdom to six survey items measuring attitudes towards immigrants that are commonly used across public opinion studies