Design Effects in the Transition to Web-Based Surveys

Innovation within survey modes should always be mitigated by concerns about survey quality and in particular sampling, coverage, nonresponse, and measurement error. This is as true today with the development of web surveying as it was in the 1970s when telephone surveying was being developed. This paper focuses on measurement error in web surveys. Although Internet technology provides significant opportunities for innovation in survey design, systematic research has yet to be conducted on how most of the possible innovations might affect measurement error, leaving many survey designers “out in the cold.” This paper summarizes recent research to provide an overview of how choosing the web mode affects the asking and answering of questions. It starts with examples of how question formats used in other survey modes perform differently in the web mode. It then provides examples of how the visual design of web surveys can influence answers in unexpected ways and how researchers can strategically use visual design to get respondents to provide their answers in a desired format. Finally, the paper concludes with suggested guidelines for web survey design

Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer Boxes and Providing Extra Verbal Instructions Improve Response Quality?

Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys

Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer Boxes and Providing Extra Verbal Instructions Improve Response Quality?

Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys

Comparing Check-All and Forced-Choice Question Formats in Web Surveys

For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys

eHealth Literacy and Web 2.0 Health Information Seeking Behaviors Among Baby Boomers and Older Adults

Background: Baby boomers and older adults, a subset of the population at high risk for chronic disease, social isolation, and poor health outcomes, are increasingly utilizing the Internet and social media (Web 2.0) to locate and evaluate health information. However, among these older populations, little is known about what factors influence their eHealth literacy and use of Web 2.0 for health information. Objective: The intent of the study was to explore the extent to which sociodemographic, social determinants, and electronic device use influences eHealth literacy and use of Web 2.0 for health information among baby boomers and older adults. Methods: A random sample of baby boomers and older adults (n=283, mean 67.46 years, SD 9.98) participated in a cross-sectional, telephone survey that included the eHealth literacy scale (eHEALS) and items from the Health Information National Trends Survey (HINTS) assessing electronic device use and use of Web 2.0 for health information. An independent samples t test compared eHealth literacy among users and non-users of Web 2.0 for health information. Multiple linear and logistic regression analyses were conducted to determine associations between sociodemographic, social determinants, and electronic device use on self-reported eHealth literacy and use of Web 2.0 for seeking and sharing health information. Results: Almost 90% of older Web 2.0 users (90/101, 89.1%) reported using popular Web 2.0 websites, such as Facebook and Twitter, to find and share health information. Respondents reporting use of Web 2.0 reported greater eHealth literacy (mean 30.38, SD 5.45, n=101) than those who did not use Web 2.0 (mean 28.31, SD 5.79, n=182), t217.60=−2.98, P=.003. Younger age (b=−0.10), more education (b=0.48), and use of more electronic devices (b=1.26) were significantly associated with greater eHealth literacy (R2 =.17, R2adj =.14, F9,229=5.277, P<.001). Women were nearly three times more likely than men to use Web 2.0 for health information (OR 2.63, Wald= 8.09, df=1, P=.004). Finally, more education predicted greater use of Web 2.0 for health information, with college graduates (OR 2.57, Wald= 3.86, df =1, P=.049) and post graduates (OR 7.105, Wald= 4.278, df=1, P=.04) nearly 2 to 7 times more likely than non-high school graduates to use Web 2.0 for health information. Conclusions: Being younger and possessing more education was associated with greater eHealth literacy among baby boomers and older adults. Females and those highly educated, particularly at the post graduate level, reported greater use of Web 2.0 for health information. More in-depth surveys and interviews among more diverse groups of baby boomers and older adult populations will likely yield a better understanding regarding how current Web-based health information seeking and sharing behaviors influence health-related decision making

The Ursinus Weekly, March 11, 1971

ProTheatre schedules 9 Spring productions • Student Life Committee suggests twelve semester open houses; President Pettit approves half • Union pilot committee seeks new members • Seniors plan \u2771 Showboat • Gymnasium site excavated • Pi Nu sponsors annual Songfest • Festival of the Arts • Blood drive • Editorial: Where do we go from here? • Focus: Tom Mack • Movie critic: What\u27s at the movies (and worth seeing) • Letters to the editor: Freaks out; Sterling studies; Excellent issue; G.P. knocks U.S.G.A. • Hamlet • Robin and the other hoods • Inside track • Hoopmen complete season • Snellbelles protest lack of floortime • Sports cornerhttps://digitalcommons.ursinus.edu/weekly/1140/thumbnail.jp

The Ursinus Weekly, May 20, 1971

U.C. Century II fund drive commences; Goal for \u2770-\u2775 set at 5.5 million • Samuel L. Gandy speaks June 6th • Pancoast takes sabbatical leave • Ursinus gives merit scholarships • Women\u27s curfews change; Weekend hours abolished • What has become of APEs? • Editorial: Apathy kills • Focus: Selby Nera • U.S.G.A. wrap-up 1970-71: A year in pursuit of progress • Appearance & reality? • Double standard is alive and living in Collegeville • Movie critic: Little big man • Student Union and Bomberger renovation to begin soon • Faculty portrait: Dr. Ray Schultz • U.C. English comprehensive to present times-one view • U.C. President William Pettit reflects on past year, his first as President • Mrs. Watson has perfected the art of winning • After four years • Varsity golf: Linksmen finish 6-8 • D. Larson pitches 3 hitter • Final examination schedulehttps://digitalcommons.ursinus.edu/weekly/1147/thumbnail.jp

Reliability and Validity of the Telephone-Based eHealth Literacy Scale Among Older Adults: Cross-Sectional Survey

Background: Only a handful of studies have examined reliability and validity evidence of scores produced by the 8-item eHealth literacy Scale (eHEALS) among older adults. Older adults are generally more comfortable responding to survey items when asked by a real person rather than by completing self-administered paper-and-pencil or online questionnaires. However, no studies have explored the psychometrics of this scale when administered to older adults over the telephone. Objective: The objective of our study was to examine the reliability and internal structure of eHEALS data collected from older adults aged 50 years or older responding to items over the telephone. Methods: Respondents (N=283) completed eHEALS as part of a cross-sectional landline telephone survey. Exploratory structural equation modeling (E-SEM) analyses examined model fit of eHEALS scores with 1-, 2-, and 3-factor structures. Subsequent analyses based on the partial credit model explored the internal structure of eHEALS data. Results: Compared with 1- and 2-factor models, the 3-factor eHEALS structure showed the best global E-SEM model fit indices (root mean square error of approximation=.07; comparative fit index=1.0; Tucker-Lewis index=1.0). Nonetheless, the 3 factors were highly correlated (r range .36 to .65). Item analyses revealed that eHEALS items 2 through 5 were overfit to a minor degree (mean square infit/outfit values <1.0; t statistics less than –2.0), but the internal structure of Likert scale response options functioned as expected. Overfitting eHEALS items (2-5) displayed a similar degree of information for respondents at similar points on the latent continuum. Test information curves suggested that eHEALS may capture more information about older adults at the higher end of the latent continuum (ie, those with high eHealth literacy) than at the lower end of the continuum (ie, those with low eHealth literacy). Item reliability (value=.92) and item separation (value=11.31) estimates indicated that eHEALS responses were reliable and stable. Conclusions: Results support administering eHEALS over the telephone when surveying older adults regarding their use of the Internet for health information. eHEALS scores best captured 3 factors (or subscales) to measure eHealth literacy in older adults; however, statistically significant correlations between these 3 factors suggest an overarching unidimensional structure with 3 underlying dimensions. As older adults continue to use the Internet more frequently to find and evaluate health information, it will be important to consider modifying the original eHEALS to adequately measure societal shifts in online health information seeking among aging populations.Open Access Fundin