Design Effects in Web Surveys: Comparing Trained and Fresh Respondents

In this paper we investigate whether there are differences in design effects between trained and fresh respondents. In three experiments, we varied the number of items on a screen, the choice of response categories, and the layout of a five point rating scale. We find that trained respondents are more sensitive to satisficing and select the first acceptable response option more often than fresh respondents. Fresh respondents show stronger effects with regard to verbal and nonverbal cues than trained respondents, suggesting that fresh respondents find it more difficult to answer questions and pay more attention to the details of the response scale in interpreting the question.professional respondents;questionnaire design;items per screen;response categories;layout

Design of Web Questionnaires: The Effect of Layout in Rating Scales

This article shows that respondents gain meaning from visual cues in a web survey as well as from verbal cues (words).We manipulated the layout of a five point rating scale using verbal, graphical, numerical, and symbolic language. This paper extends the existing literature in four directions: (1) all languages (verbal, graphical, numeric, and symbolic) are individually manipulated on the same rating scale, (2) a heterogeneous sample is used, (3) in which way personal characteristics and a respondent's need to think and evaluate account for variance in survey responding is analyzed, and (4) a web survey is used.Our experiments show differences due to verbal and graphical language but no effects of numeric or symbolic language are found.Respondents with a high need for cognition and a high need to evaluate are affected more by layout than respondents with a low need to think or evaluate.Furthermore, men, the elderly, and the highly educated are the most sensible for layout effects.web survey;questionnaire lay out;context effects;need for cognition;need to evaluate

Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?

With the current growth in broadband penetration, Internet is likely to be the data collection mode of choice for stated preference research in the not so distant future. However, little is known about how this survey mode may influence data quality and welfare estimates. In a first controlled field experiment to date as part of a national contingent valuation (CV) survey estimating willingness to pay (WTP) for biodiversity protection plans, we assign two groups sampled from the same panel of respondents either to an Internet or in-person (in-house) interview mode. Our design is better able than previous studies to isolate measurement effects from sample composition effects. We find little evidence of social desirability bias in the in-person interview setting or satisficing (shortcutting the response process) in the Internet survey. The share of “don’t knows”, zeros and protest responses to the WTP question with a payment card is very similar between modes. Equality of mean WTP between samples cannot be rejected. Considering equivalence, we can reject that mean WTP from the in-person sample is more than 30% higher. Results are quite encouraging for the use of Internet in CV as stated preferences do not seem to be significantly different or biased compared to in-person interviews.Internet; contingent valuation; interviews; survey mode; willingness to pay

Mode effect or question wording? Measurement error in mixed mode surveys.

[Dataset available: http://hdl.handle.net/10411/17672]

Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes

Internet is quickly becoming the survey mode of choice for stated preference (SP) surveys in environmental economics. However, this choice is being made with relatively little consideration of its potential influence on survey results. This paper reviews the theory and emerging evidence of mode effects in the survey methodology and SP literatures, summarizes the findings, and points out implications for Internet SP practice and research. The SP studies that compare Internet with other modes do generally not find substantial difference. The majority of welfare estimates are equal; or somewhat lower for the Internet surveys. Further, there is no clear evidence of substantially lower quality or validity of Internet responses. However, the degree of experimental control is often low in comparative studies across survey modes, and they often confound measurement and sample composition effects. Internet offers a huge potential for experimentation and innovation in SP research, but when used to derive reliable welfare estimates for policy assessment, issues like representation and nonresponse bias for different Internet panels should receive more attention.Internet; survey mode; contingent valuation; stated preferences

An Online Tutor for Astronomy: The GEAS Self-Review Library

We introduce an interactive online resource for use by students and college instructors in introductory astronomy courses. The General Education Astronomy Source (GEAS) online tutor guides students developing mastery of core astronomical concepts and mathematical applications of general astronomy material. It contains over 12,000 questions, with linked hints and solutions. Students who master the material quickly can advance through the topics, while under-prepared or hesitant students can focus on questions on a certain topic for as long as needed, with minimal repetition. Students receive individual accounts for study and course instructors are provided with overview tracking information, by time and by topic, for entire cohorts of students. Diagnostic tools support self-evaluation and close collaboration between instructor and student, even for distance learners. An initial usage study shows clear trends in performance which increase with study time, and indicates that distance learners using these materials perform as well as or better than a comparison cohort of on-campus astronomy students. We are actively seeking new collaborators to use this resource in astronomy courses and other educational venues.Comment: 15 pages, 9 figures; Vogt, N. P., and A. S. Muise. 2015. An online tutor for general astronomy: The GEAS self-review library. Cogent Education, 2 (1