Memory Gaps in the American Time Use Survey. Investigating the Role of Retrieval Cues and Respondents’ Level of Effort

Unaccounted respondent memory gasp- i.e., those activity gaps that are attributed by interviewers to respondents\u27 memory failure- have serious implications for data quality. We contribute to the existing literature by investigating interviewing dynamics using paradata, distinguishing temporary memory gaps, which can be resolved during the interview, from enduring memory gaps, which cannot be resolved. We investigate factors that are associated with both kinds of memory gaps and how different response strategies are associated with data quality. We investigate two hypotheses that are associated with temporary and enduring memory gaps. The motivated cuing hypothesis posits that respondents who display more behaviors related to the presence and use of retrieval cues throughout the survey will resolve temporary memory gaps more successfully compared to respondents displaying fewer such behaviors. This should result in overall lower levels of enduring memory gaps. The lack of effort hypothesis suggests that respondent who are less eager to participate in the survey will expend less cognitive effort to resolve temporary memory gaps compared to more motivated respondents. This should then result in a positive association with enduring memory gaps and no association with temporary memory gaps. Using survey and paradata from the 2010 ATUS, our analyses indicate that, as hypothesized, behaviors indicating the use of retrieval cues are positively associated with temporary memory gaps and negatively associated with enduring memory gaps. Motivated respondents experiencing memory difficulties overcome what otherwise would result in enduring memory gaps more successfully compared to other respondents. Indicators of lack of effort, such as whether or not the respondent initially refused to participate in the survey, are positively associated with enduring memory gaps suggesting that reluctant respondents do not resolve memory gaps. The paper concludes with a discussion of implications for survey research

WELLBEING AND DATA QUALITY IN THE AMERICAN TIME USE SURVEY (ATUS) FROM A TOTAL SURVEY ERROR PERSPECTIVE

In this dissertation, I seek to develop a tool for the enhancement of time-use and wellbeing measures from a total survey error perspective. In particular, I evaluate the quality of the time use data produced in the American Time Use Survey (ATUS), by exploring its indicators and identifying its main predictors, including interview rapport. Results from these analyses are then used to evaluate the extent to which certain variables correlate, as predicted, with expected levels of wellbeing. The first specific objective was to investigate the data quality of the 2010 ATUS by constructing a data quality index. In my dissertation, data quality was operationalized as the degree of completeness with which the ATUS diary was completed. The second objective was to examine whether interview rapport predicts data quality. Interview rapport is understood as the conversational interaction quality that contributes to motivate respondents to be more thorough in their reports and to help them access the required information. Finally, the third objective was to assess the predictive power of activitybased wellbeing measures and other variables assumed to affect overall wellbeing, controlling for the impact of data quality in the prediction model. Two factors of data quality were found through a confirmatory factor analysis model, one related to the degree of motivation to report and the second one related to memory processes that impact the level of accuracy and detail of activity reports. Gender, age, and education are significant predictors of both factors. Through a structural model, it was also found that interview rapport predicts motivation and memory though in opposite directions: While rapport appears to benefit the motivation to respond, it can be detrimental to remembering details. Finally, when predicting overall health (taken as a proxy for wellbeing), it was found that only when controlling for the memory-related data quality factor was there a relative increase in the amount of variance explained, although it was not a practically significant increase. Further research would be helpful in validating the measurement of data quality and rapport constructs, as well as to more efficiently incorporate data quality and rapport in the prediction of wellbeing

NCRN Meeting Spring 2016: Evaluating Data quality in Time Diary Surveys Using Paradata

Presented at the NCRN Meeting Spring 2016 in Washington DC on May 9-10, 2016; see http://www.ncrn.info/event/ncrn-spring-2016-meetingOver the past decades, time use researchers have been increasingly interested in analyzing wellbeing in tandem with the use of time (Juster and Stafford, 1985; Krueger et al, 2009). Many methodological issues have arose in this endeavor, including the concern about the quality of the time use data. Survey researchers have increasingly turned to the analysis of paradata to better understand and model data quality. In particular, it has been argued that paradata may serve as proxy of the respondents’ cognitive response process, and can be used as an additional tool to assess the impact of data generation on data quality. In this presentation, data quality in the American Time Use Survey (ATUS) will be assessed through the use of paradata and survey responses. Specifically, I will talk about a data quality index I have created, which includes measures of different types of ATUS errors (e.g. low number of reported activities, failures to report an activity), and paradata variables (e.g. response latencies, incompletes). The overall objective of this study is to contribute to data quality assessment in the collection of timeline data from national surveys by providing insights on those interviewing dynamics that most impact data quality. These insights will help to improve future instruments and training of interviewers, as well as to reduce costs.NSF Grant 1507241 (NCRN Coordinating Office) and 1132015 (to University of Nebraska-Lincoln

Memory Gaps in the American Time Use Survey. Investigating the Role of Retrieval Cues and Respondents’ Level of Effort

Unaccounted respondent memory gasp- i.e., those activity gaps that are attributed by interviewers to respondents\u27 memory failure- have serious implications for data quality. We contribute to the existing literature by investigating interviewing dynamics using paradata, distinguishing temporary memory gaps, which can be resolved during the interview, from enduring memory gaps, which cannot be resolved. We investigate factors that are associated with both kinds of memory gaps and how different response strategies are associated with data quality. We investigate two hypotheses that are associated with temporary and enduring memory gaps. The motivated cuing hypothesis posits that respondents who display more behaviors related to the presence and use of retrieval cues throughout the survey will resolve temporary memory gaps more successfully compared to respondents displaying fewer such behaviors. This should result in overall lower levels of enduring memory gaps. The lack of effort hypothesis suggests that respondent who are less eager to participate in the survey will expend less cognitive effort to resolve temporary memory gaps compared to more motivated respondents. This should then result in a positive association with enduring memory gaps and no association with temporary memory gaps. Using survey and paradata from the 2010 ATUS, our analyses indicate that, as hypothesized, behaviors indicating the use of retrieval cues are positively associated with temporary memory gaps and negatively associated with enduring memory gaps. Motivated respondents experiencing memory difficulties overcome what otherwise would result in enduring memory gaps more successfully compared to other respondents. Indicators of lack of effort, such as whether or not the respondent initially refused to participate in the survey, are positively associated with enduring memory gaps suggesting that reluctant respondents do not resolve memory gaps. The paper concludes with a discussion of implications for survey research

Driver education and teen crashes and traffic violations in the first two years of driving in a graduated licensing system

Our primary research question was whether teens obtaining their intermediate-level provisional operators permit (POP) in a graduated driver licensing (GDL) environment through driver education differed in crashes and traffic violations from teens who obtained their POP by completing a supervised driving certification log without taking driver education. A descriptive epidemiological study examining a census of all teen drivers in Nebraska (151,880 teens, 48.6% girls, 51.4% boys) during an eight year period from 2003 to 2010 was conducted. The driver education cohort had significantly fewer crashes, injury or fatal crashes, violations, and alcohol-related violations than the certification log cohort in both years one and two of driving following receipt of the POP. Hierarchical logistic regression was conducted, controlling for gender, race/ethnicity, median household income, urban–rural residence, and age receiving the POP. In both year one and two of driving, teens in the certification log cohort had higher odds of a crash, injury or fatal crash, violation, or alcohol-related violation. Findings support that relative to a supervised driving certification log approach, teens taking driver education are less likely to be involved in crashes or to receive a traffic violation during their first two years of driving in an intermediate stage in a graduated driver licensing system. Because teen crash and fatality rates are highest at ages 16–18, these reductions are especially meaningful. Driver education appears to make a difference in teen traffic outcomes at a time when risk is highest

Memory Gaps in the American Time Use Survey: Are Respondents Forgetful or is There More to it?

Unaccounted respondent memory gaps - i.e., those activity gaps that are attributed by interviewers to respondents' memory failure - have serious implications for data quality. We contribute to the existing literature by investigating interviewing dynamics using paradata, distinguishing temporary memory gaps, which can be resolved during the interview, from enduring memory gaps, which cannot be resolved. We investigate factors that are associated with both kinds of memory gaps and how different response strategies are associated with data quality. We investigate two hypotheses that are associated with temporary and enduring memory gaps. The motivated cuing hypothesis posits that respondents who display more behaviors related to the presence and use of retrieval cues throughout the survey will resolve temporary memory gaps more successfully compared to respondents displaying fewer such behaviors. This should result in overall lower levels of enduring memory gaps. The lack of effort hypothesis suggests that respondent who are less eager to participate in the survey will expend less cognitive effort to resolve temporary memory gaps compared to more motivated respondents. This should then result in a positive association with enduring memory gaps and no association with temporary memory gaps. Using survey and paradata from the 2010 ATUS, our analyses indicate that, as hypothesized, behaviors indicating the use of retrieval cues are positively associated with temporary memory gaps and negatively associated with enduring memory gaps. Motivated respondents experiencing memory difficulties overcome what otherwise would result in enduring memory gaps more successfully compared to other respondents. Indicators of lack of effort, such as whether or not the respondent initially refused to participate in the survey, are positively associated with enduring memory gaps suggesting that reluctant respondents do not resolve memory gaps. The paper concludes with a discussion of implications for survey research