Purpose This study examined data equivalency and loss to follow-up rates from Internet and interactive voice response (IVR) system surveys in a prospective-cohort study.  Methods 475 limited-service restaurant workers participating in the 12-week study were given a choice to report their weekly slipping experience by either IVR or Internet. Demographic differences, loss to follow-up, self-reported rates of slipping, and selection of first and last choices were compared.  Results Loss to follow-up rates were slightly higher for those choosing the IVR mode. Rates of slipping and selection of first and last choices were not significantly different between survey modes. Propensity to choose an Internet survey decreased with increasing age, and was the lowest among Spanish speakers (5%) and those with less than a high school education (14%).  Conclusions Studies relying solely on Internet-based data collection may lead to selective exclusion of certain populations. Findings suggest that Internet and IVR may be combined as survey modalities within longitudinal studies

Brennan, Melanye J.

Chang, Wen-Ruey

Courtney, Theodore K.

Huang, Yueng-Hsiang

Lombardi, David A.

Perry, Melissa J.

Verma, Santosh K.

English

George Washington University: Health Sciences Research Commons (HSRC)

Himmelfarb Health Sciences Library, The George Washington UniversityHealth Sciences Research CommonsEnvironmental and Occupational Health FacultyPublications Environmental and Occupational Health1-2014Internet and telephonic IVR mixed-mode surveyfor longitudinal studies: Choice, retention, and dataequivalencySantosh K. VermaUniversity of MassachusettsTheodore K. CourtneyHarvard School of Public HealthDavid A. LombardiHarvard School of Public HealthWen-Ruey ChangLiberty Mutual Research Institute for Safety, Hopkinton, MAYueng-Hsiang HuangLiberty Mutual Research Institute for Safety, Hopkinton, MASee next page for additional authorsFollow this and additional works at: http://hsrc.himmelfarb.gwu.edu/sphhs_enviro_facpubsPart of the Environmental Public Health Commons, and the Occupational Health and IndustrialHygiene CommonsThis Journal Article is brought to you for free and open access by the Environmental and Occupational Health at Health Sciences Research Commons.It has been accepted for inclusion in Environmental and Occupational Health Faculty Publications by an authorized administrator of Health SciencesResearch Commons. For more information, please contact hsrc@gwu.edu.Recommended CitationVerma, S.K., Courtney, T.K., Lombardi, D., Chang, W., Huang, Y. et al. (2014). Internet and telephonic IVR mixed-mode survey forlongitudinal studies: Choice, retention, and data equivalency. Annals of Epidemiology, 24(1), 72-74.AuthorsSantosh K. Verma, Theodore K. Courtney, David A. Lombardi, Wen-Ruey Chang, Yueng-Hsiang Huang,Melanye J. Brennan, and Melissa J. PerryThis journal article is available at Health Sciences Research Commons: http://hsrc.himmelfarb.gwu.edu/sphhs_enviro_facpubs/94Brief communicationInternet and telephonic IVR mixed-mode survey for longitudinal studies:choice, retention, and data equivalencyqSantosh K. Verma ScD a,b,*, Theodore K. Courtney MS a,c, David A. Lombardi PhD a,c,Wen-Ruey Chang PhD d, Yueng-Hsiang Huang PhD e, Melanye J. Brennan MS a, Melissa J. Perry ScD faCenter for Injury Epidemiology, Liberty Mutual Research Institute for Safety, Hopkinton, MassachusettsbDepartment of Family Medicine and Community Health, University of Massachusetts Medical School, Worcester, Massachusettsc Environmental and Occupational Medicine and Epidemiology Program, Department of Environmental Health, Harvard School of Public Health, Boston, MassachusettsdCenter for Physical Ergonomics, Liberty Mutual Research Institute for Safety, Hopkinton, MassachusettseCenter for Behavioral Sciences, Liberty Mutual Research Institute for Safety, Hopkinton, MassachusettsfDepartment of Environmental and Occupational Health, The George Washington University, School of Public Health and Health Services, 2100 M Street, NW Suite 203A, Washington,DC 20037a r t i c l e i n f oArticle history:Received 25 June 2013Accepted 8 October 2013Available online 14 October 2013Keywords:Interactive voice responseInternetIVRLongitudinalMixed modeRestaurantsSurveySurvey methodFallSlipsInjurya b s t r a c tPurpose: This study examined data equivalency and loss to follow-up rates from Internet and interactivevoice response (IVR) system surveys in a prospective-cohort study.Methods: 475 limited-service restaurant workers participating in the 12-week study were given a choiceto report their weekly slipping experience by either IVR or Internet. Demographic differences, loss tofollow-up, self-reported rates of slipping, and selection of first and last choices were compared.Results: Loss to follow-up rates were slightly higher for those choosing the IVR mode. Rates of slippingand selection of first and last choices were not significantly different between survey modes. Propensityto choose an Internet survey decreased with increasing age, and was the lowest among Spanish speakers(5%) and those with less than a high school education (14%).Conclusions: Studies relying solely on Internet-based data collection may lead to selective exclusion ofcertain populations. Findings suggest that Internet and IVR may be combined as survey modalities withinlongitudinal studies. 2014 The Authors. Published by Elsevier Inc. All rights reserved.IntroductionTelephonic interactive voice response (IVR) systems andInternet surveys have provided new and efficient methods of lon-gitudinal data collection. Several longitudinal studies have usedeither Internet or IVR surveys to collect follow-up data [1e11].However, few longitudinal studies have used a mixed-mode ofInternet and IVR surveys to follow participants [12]. Using only onesurvey mode may lead to biased sample selection, due to techno-logical barriers for certain participants, and/or biased results.Providing a choice of multiple survey modes (Internet and IVR)has been shown to increase the response rate in cross-sectionalsurveys [13,14]; however, retention and data equivalency bet-ween the two methods for longitudinal studies has not been wellstudied.In a 12-week prospective cohort study of risk factors for slipping,we used Internet and IVR surveys to collect participants’ weeklyself-reported slip experiences and other key information related toslipping and falling. We present a comparison of participant surveymode selection by sociodemographic characteristics. We alsoexamined self-reported rates of slipping, loss to follow-up, andprimacy/recency effects by survey mode.MethodsFour hundred seventy-five workers were recruited from 36limited-service restaurants across the United States in the yearsq This is an open-access article distributed under the terms of the CreativeCommons Attribution-NonCommercial-No Derivative Works License, which per-mits non-commercial use, distribution, and reproduction in any medium, providedthe original author and source are credited.* Corresponding author. Center for Injury Epidemiology, Liberty Mutual ResearchInstitute for Safety, 71 Frankland Rd., Hopkinton, MA 01748. Tel.: (508) 497 0213;fax: (508) 435 0482.E-mail address: Santosh.Verma@LibertyMutual.com (S.K. Verma).Contents lists available at ScienceDirectAnnals of Epidemiologyjournal homepage: www.annalsofepidemiology.org1047-2797/$ e see front matter  2014 The Authors. Published by Elsevier Inc. All rights reserved.http://dx.doi.org/10.1016/j.annepidem.2013.10.004Annals of Epidemiology 24 (2014) 72e742007 and 2008. Study methods have been previously described indetail [15].Weekly surveyAfter completing an in-person baseline survey at the restaurant,participants were asked to report their slip experience weekly overthe following 12 weeks. Participants were given a choice ofreporting their weekly experience by IVR, Internet, or by mailingpostal questionnaire forms. All studymaterials weremade availablein English, Spanish, and Portuguese.Each week after the enrollment, participants reported thenumber of hours worked and their slip experience in the previousweek at work. If they experienced one or more slips in the previousweek, they also reported the circumstances around the first four ofthose slips, including the time and presence of contamination onthe floor.Survey modesParticipants choosing IVR were provided with a toll-free tele-phone number, a unique identifier and instructions on how to usethe IVR system. Each week an e-mail was sent to participantschoosing Internet with a link to their weekly survey that includedtheir unique identifier and the language of choice. For both IVRand Internet surveys, numerical responses were quality-checkedagainst preset ranges and data were automatically time- anddate-stamped and entered into a spreadsheet. These data werechecked daily and, if a participant did not complete their weeklysurvey on the due date, a reminder call was made, typically the nextbusiness day, by a study team member to remind the participant toreport the data through the chosen survey mode.Paper forms and stamped envelopes were given to participantswho were not comfortable with either IVR or Internet. Participantswho chose paper surveys were excluded from further analysis dueto small sample size (n ¼ 6).Statistical analysisGeneralized estimating equations models with compoundsymmetry covariance structure were used to compare (1) choice ofInternet versus IVR mode by selected demographic characteristicsand (2) survey completion (the percent who completed at least onesurvey and the mean number of completed surveys among theseparticipants), the reported rate of slipping, and selection of the firstand the last choice among those who chose Internet and IVRsurveys.ResultsOf the 475 participants, 315 chose the IVR (66.3%), 154 chose theInternet survey (32.4%). Choice of the Internet survey mode overthe IVR mode decreased with increasing age (P < .05, See Appendix1). Spanish speakers (as compared with English speakers), Blacknon-Hispanic (as compared with White non-Hispanic), and par-ticipants without any college education were less likely to choosethe Internet over the IVR (P < .05). No significant difference wasobserved by gender.The proportion of participants completing at least one weeklysurvey was slightly higher among the Internet survey group (pro-portion ratio, 1.08; 95% confidence interval, 1.02e1.14; Table 1). Rateof slipping did not differ significantly by survey method in eitherthe univariate or multivariable model. Selection of the first and thelast choice (primacy or recency effect) also did not differ signifi-cantly by survey mode.DiscussionChoiceIn this study, young adults, those with some college educationand those who spoke English or Portuguese, were more likely tochoose the Internet survey. Similar results were reported by Bex-elius et al. [16]. The availability of the Internet has increasedsignificantly in recent years, and many studies have used or areplanning to use online surveys to collect longitudinal data [1e3].However, approximately 19% of American adults still do not use theInternet according to the Pew Internet and American Life Survey2012 data [17]. The Pew report also found that Internet use waslower among older adults, African American and Hispanics, andthose with no high school education [17]. A similar digital dividealong the lines of age, education level, and socioeconomic statusexists in Europe [18]. Thus, a longitudinal study relying solely on anInternet survey mode may have lower participation from thesegroups. The IVR is also a survey method of choice for participantswho have difficulty reading and writing, as receptive vocabularytypically exceeds reading vocabulary [19,20].Retention and data equivalencyLoss to follow-up was slightly higher for the IVR mode ascompared with the Internet mode. However, the difference wassmall and the increase in the response rate due to choice betweenmultiple modes may outweigh this marginal disadvantage of theIVR mode. Sinadinovic et al. [21], in 2011, found that higherresponse rates resulted when the respondents were given a choicebetween both Internet and IVR (43.2% and 46.6%, respectively) ascomparedwith when only the Internet or the IVRmodewas offered(38.1% and 33.9%, respectively).We did not observe a significant difference in the reported rateof slipping by survey mode. Other studies using both the InternetTable 1Loss to follow-up, rate of slipping, and probability of choosing the first choice and thelast choice by survey modedresults from multivariable generalized estimatingequations modelsMultivariable modelEffect estimate* 95% Confidence intervalParticipants completing at least one weekly surveyInternet 1.08y 1.02e1.14Telephone (IVR) 1Difference in the mean number of weekly surveys for those who completed atleast one surveyInternet 0.49z 0.02e0.97Telephone (IVR) 0Rate of slippingInternet 0.84x 0.63e1.13Telephone (IVR) 1Floor contaminants, first choice (liquid)Internet 1.15y 0.94e1.42Telephone (IVR) 1Floor contaminants, last choice (do not know/other)Internet 0.95y 0.60e1.49Telephone (IVR) 1Time of slip, first choice (6e11 AM)Internet 1.42y 0.95e2.13Telephone (IVR) 1Time of slip, last choice (9 PM to 6 AM)Internet 1.09y 0.58e2.03Telephone (IVR) 1* Adjusted for age, gender, education, language, and ethnicity.y Proportion ratio.z Difference in mean.x Rate ratio.S.K. Verma et al. / Annals of Epidemiology 24 (2014) 72e74 73and IVR have similarly found no difference in responses by surveymode [14,16]. However, some questions may be more sensitive tomodal differences. For example, one study found that respondentsto the aural modes were more likely than were respondents to thevisual modes to give extreme positive responses to scale-typequestions [22].Response choices for nominal variables may differ by visual andaural surveymodedue to primacy or recency effect [22,23]. However,the probability of selecting the first or the last choice did not differ bysurveymode in this study. Similarly, a reviewof 82experiments foundno consistent pattern for primacy or recency effects [24].Studies relying solely on the Internet may lead to selectiveexclusion of older adults, lower educational groups, and racial-ethnic minorities. For a short repetitive survey, we did not findany evidence suggesting lack of data equivalency between the twomodes, and it should be possible to combine data collected usingthe IVR and Internet surveys in longitudinal studies.References[1] Turner C, Bain C, Schluter PJ, Yorkston E, Bogossian F, McClure R, et al. Cohortprofile: the nurses and midwives e-cohort studyda novel electronic longi-tudinal study. Int J Epidemiol 2009;38(1):53e60.[2] Russell CW, Boggs DA, Palmer JR, Rosenberg L. Use of a web-based ques-tionnaire in the Black Women’s Health Study. Am J Epidemiol 2011;172(11):1286e91.[3] van Gelder MM, Bretveld RW, Roeleveld N. Web-based questionnaires: thefuture in epidemiology? Am J Epidemiol 2010;172(11):1292e8.[4] Arcury TA, Mills T, Marin AJ, Summers P, Quandt SA, Rushing J, et al. Worksafety climate and safety practices among immigrant Latino residential con-struction workers. Am J Ind Med 2012;55(8):736e45.[5] Wiebe DJ, Blackstone MM, Mollen CJ, Culyba AJ, Fein JA. Self-reportedviolence-related outcomes for adolescents within eight weeks of emergencydepartment treatment for assault injury. J Adolesc Health 2011;49(4):440e2.[6] Magid DJ, Ho PM, Olson KL, Brand DW, Welch LK, Snow KE, et al. A multimodalblood pressure control intervention in 3 healthcare systems. Am J Manag Care2011;17(4):e96e103.[7] Statland JM, Wang Y, Richesson R, Bundy B, Herbelin L, Gomes J, et al. Aninteractive voice response diary for patients with non-dystrophic myotonia.Muscle Nerve 2011;44(1):30e5.[8] Blackstone MM, Wiebe DJ, Mollen CJ, Kalra A, Fein JA. Feasibility of an inter-active voice response tool for adolescent assault victims. Acad Emerg Med2009;16(10):956e62.[9] Payne KA, Caro JJ, Daley WL, Khan ZM, Ishak KJ, Stark K, et al. The design of anobservational study of hypertension management, adherence and pressurecontrol in Blood Pressure Success Zone Program participants. Int J Clin Pract2008;62(9):1313e21.[10] ShawWS, Verma SK. Data equivalency of an interactive voice response systemfor home assessment of back pain and function. Pain Res Manag 2007;12(1):23e30. Spring.[11] Corkrey R, Parkinson L. Interactive voice response: review of studies 1989-2000. Behav Res Methods Instrum Comput 2002;34(3):342e53.[12] Johnson SM, LeVine P. Self-reported treatment impressions and satisfaction ofpapulopustular rosacea patients treated with doxycycline, USP, 40 mg cap-sules. J Drugs Dermatol 2011;10(12):1376e81.[13] Greene J, Speizer H, Wiitala W. Telephone and web: mixed-mode challenge.Health Serv Res 2008;43(1 Pt 1):230e48.[14] Sinadinovic K, Wennberg P, Berman AH. Population screening of risky alcoholand drug use via Internet and Interactive Voice Response (IVR): a feasibilityand psychometric study in a random sample. Drug Alcohol Depend 2010;114(1):55e60.[15] Verma SK, ChangWR, Courtney TK, Lombardi DA, Huang YH, Brennan MJ, et al.A prospective study of floor surface, shoes, floor cleaning and slipping in U.S.limited-service restaurant workers. Occup Environ Med 2011;68(4):279e85.Oct 10, 2010. [Epub ahead of print].[16] Bexelius C, Merk H, Sandin S, Nyren O, Kuhlmann-Berenzon S, Linde A, et al.Interactive Voice Response and web-based questionnaires for population-based infectious disease reporting. Eur J Epidemiol 2010;25(10):693e702[Article].[17] Pew Internet & American Life Survey. Demographics of Internet users. 2012[cited 2013 3/11]; Available from: http://www.pewinternet.org/Static-Pages/Trend-Data-(Adults)/Whos-Online.aspx.[18] Vandoninck S, Roe K. The digital divide in Flanders: disappearance orpersistence?. Berlin, Allemagne: De Gruyter; 2008.[19] Crow JT. Receptive vocabulary acquisition for reading comprehension. TheModern Language Journal 1986;70(3):242e50.[20] Henriksen B. Three dimensions of vocabulary development. Studies in SecondLanguage Acquisition 1999;21(02):303e17.[21] Sinadinovic K, Wennberg P, Berman AH. Population screening of risky alcoholand drug use via Internet and Interactive Voice Response (IVR): A feasibilityand psychometric study in a random sample. Drug and Alcohol Dependence2011;114(1):55e60.[22] Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, et al. Response rateand measurement differences in mixed-mode surveys using mail, telephone,interactive voice response (IVR) and the Internet. Social Science Research2009;38(1):1e18.[23] Krosnick JA, Alwin DF. An evaluation of a cognitive theory of response-ordereffects in survey measurement. Public Opinion Quarterly 1987;51(2):201e19. June 20, 1987.[24] Dillman DA, Brown TL, Carlson JE, Carpenter EH, Lorenz FO, Mason R, et al.Effects of category order on answers in mail and telephone surveys. ruralsociology. Rural Sociology 1995;60(4):674e87.Appendix 1Number of participants in each category of selected demographic characteristics,proportion of participants choosing the Internet-basedmethod, and proportion ratioand 95% confidence intervals showing propensity to choose Internet-based methodversus IVR-based methodDemographicCharacteristicsn Internet(%)UnivariatemodelMultivariablemodelPR (95% CI) PR (95% CI)Age (y)14e19 115 46.96 1 120e29 152 32.24 0.70 (0.51e0.96) 0.54 (0.38e0.76)30e39 79 24.05 0.52 (0.33e0.82) 0.44 (0.29e0.67)40e49 78 34.62 0.74 (0.48e1.13) 0.55 (0.37e0.82)50þ 44 11.36 0.23 (0.08e0.68) 0.17 (0.06e0.49)GenderWomen 310 32.58 0.98 (0.74e1.29) 1.04 (0.79e1.36)Men 159 33.33 1 1EducationLess than high school 80 13.75 0.29 (0.16e0.53) 0.35 (0.21e0.61)High school student 78 46.15 0.94 (0.66e1.34) 0.50 (0.36e0.70)High school graduate 184 24.46 0.50 (0.36e0.70) 0.51 (0.37e0.70)Some college or above 127 48.82 1 1EthnicityBlack non-Hispanic 91 24.18 0.62 (0.41e0.93) 0.56 (0.38e0.82)Hispanic 77 20.78 0.55 (0.29e1.06) 0.93 (0.53e1.64)Other 42 35.71 0.92 (0.55e1.53) 0.78 (0.46e1.30)White non-Hispanic 259 39.00 1 1LanguageSpanish survey 40 5.00 0.13 (0.06e0.30) 0.21 (0.08e0.59)Portuguese survey 13 53.85 1.54 (1.07e2.21) 1.76 (1.33e2.34)English survey 416 34.86 1 1S.K. Verma et al. / Annals of Epidemiology 24 (2014) 72e7474

Internet and telephonic IVR mixed-mode survey for longitudinal studies: Choice, retention, and data equivalency

https://hsrc.himmelfarb.gwu.edu/cgi/viewcontent.cgi?article=1093&context=sphhs_enviro_facpubs

Internet and telephonic IVR mixed-mode survey for longitudinal studies: Choice, retention, and data equivalency

Abstract

Similar works

Full text

Available Versions

George Washington University: Health Sciences Research Commons (HSRC)