Conducting Research on the Internet:: Online Survey Design, Development and Implementation Guidelines

Using the Internet to conduct quantitative research presents challenges not found in conventional research. Some of our knowledge concerning the effective design and use of paper-based surveys does translate into electronic formats. However, electronic surveys have distinctive technological, demographic and response characteristics that affect how they should be designed, when they can be used and how they can be implemented. Survey design, subject privacy and confidentiality, sampling and subject solicitation, distribution methods and response rates and survey piloting are critical methodological components that must be addressed in order to conduct sound online research. This paper focuses on those distinctive characteristics. It reviews the current literature on the subject of electronic surveys and presents guidelines for designing, developing and implementing them, particularly web-based surveys. This paper argues that Web-based surveys are superior to email surveys in many aspects, but that email combined, perhaps with offline media, is an excellent vehicle for inviting individuals to participate in Web-based surveys. The application of these guidelines are demonstrated through the authors’ current research involving defining the nature of “non-public participation” (commonly referred to as lurking) in online discussion groups. Guidelines do not eliminate the many “trade-off” decisions required in the use of online surveys

New Technology and Travel Surveys: The Way Forward

Traditional survey methods are fast reaching their “use by” dates. It is much harder for researchers to contact households through the telephone as a result of call screening and answering machine devices. In relation to face to face interviews, housing estates and buildings are increasingly becoming fenced off and protected by security systems making it virtually impossible for the researcher to enter the premises, if not expected, as well as the fact that more often than not, householders are not available at the time when the survey is being conducted. In addition, the over use of marketing surveys has led people to believe that every survey they are asked to complete is of this type. Together, these difficulties have led to rising item and unit non-response, and consequently, rising unit costs. This is not only a phenomena associated with travel surveys, but rather the entire realm of social science research. Development of the World Wide Web has had some dramatic impacts on the global environment, in relation to communication, information and research. The development of web based surveys (internet-based or e-mail), is commonly seen as a combative measure to rising costs and the declining response problem faced by most survey practitioners. Despite the phenomenal uptake of this technology by industry and individuals, some people are still to embrace this new medium. This poses some interesting questions for researchers wanting to utilise this technology to combat low response rates. One of the major concerns today, in relation to web based surveys, is sample bias. Internet users are usually of higher socio-economic status. In addition, people who have access to the internet are not always certain of their ability to utilise the internet, let alone complete a web based survey. These are important issues that need to be considered if web based surveys are to be used

Measuring Customer Satisfaction on the Internet

Based on the Expectancy Disconfirmation Model as the underlying construct, methods to measure customer satisfaction with products and the steps to be undertaken in the research process are investigated. The measurement of Derived Satisfaction using (dis)confirmation was identified to be the appropriate approach to CS measurement. Prior research has also shown that during the research process, several points specific to CS measurements need to be accounted for. The Internet services currently used by marketing and social researchers include E-mail, mailinglists, newsgroups, Internet Chat, the World Wide Web (WWW) and Virtual Worlds. Virtual Worlds, being most advantageous for observational studies, are not useful for customer satisfaction research. Virtual Communities, in turn, have some promising characteristics for future use. Internet research methods based on these services include E-mail surveys and WWW-surveys. Common advantages of E-mail- and WWW-surveys include administrative and response speed, cost savings and global reach of respondents. Their greatest common disadvantage is the non-representativeness of the respondents for the larger population as well as their self-selection. Unless access is restricted to a known population, probability sampling is impossible when using the World Wide Web. Based on these insights, the Internet was found to be an advantageous medium for customer satisfaction studies only if specific conditions are met. Companies need to investigate on a case-by-case basis if the online measurement of customer satisfaction is possible in their specific situation. The recommendations were summarized in a decision-making framework. The results of a survey among market research agencies show that practitioners are to a large extent aware of the limitations within which the Internet can be used for customer satisfaction surveys. However, especially WWW-surveys sometimes are conducted in a way that does not lead to representative results.management information;

Automated Stellar Spectral Classification and Parameterization for the Masses

Stellar spectroscopic classification has been successfully automated by a number of groups. Automated classification and parameterization work best when applied to a homogeneous data set, and thus these techniques primarily have been developed for and applied to large surveys. While most ongoing large spectroscopic surveys target extragalactic objects, many stellar spectra have been and will be obtained. We briefly summarize past work on automated classification and parameterization, with emphasis on the work done in our group. Accurate automated classification in the spectral type domain and parameterization in the temperature domain have been relatively easy. Automated parameterization in the metallicity domain, formally outside the MK system, has also been effective. Due to the subtle effects on the spectrum, automated classification in the luminosity domain has been somewhat more difficult, but still successful. In order to extend the use of automated techniques beyond a few surveys, we present our current efforts at building a web-based automated stellar spectroscopic classification and parameterization machine. Our proposed machinery would provide users with MK classifications as well as the astrophysical parameters of effective temperature, surface gravity, mean abundance, abundance anomalies, and microturbulence.Comment: 5 pages; to appear in The Garrison Festschrift conference proceeding

Recruiting Young and Urban Groups into a Probability-Based Online Panel by Promoting Smartphone Use

A sizable minority of all web surveys are nowadays completed on smartphones. People who choose a smartphone for Internet-related tasks are different from people who mainly use a PC or tablet. Smartphone use is particularly high among the young and urban. We have to make web surveys attractive for smartphone completion in order not to lose these groups of smartphone users. In this paper we study how to encourage people to complete surveys on smartphones in order to attract hard-to-reach subgroups of the population. We experimentally test new features of a survey-friendly design: we test two versions of an invitation letter to a survey, a new questionnaire lay-out, and autoforwarding. The goal of the experiment is to evaluate whether the new survey design attracts more smartphone users, leads to a better survey experience on smartphones and results in more respondents signing up to become a member of a probability-based online panel. Our results show that the invitation letter that emphasizes the possibility for smartphone completion does not yield a higher response rate than the control condition, nor do we find differences in the socio-demographic background of respondents. We do find that slightly more respondents choose a smartphone for survey completion. The changes in the layout of the questionnaire do lead to a change in survey experience on the smartphone. Smartphone respondents need 20% less time to complete the survey when the questionnaire includes autoforwarding. However, we do not find that respondents evaluate the survey better, nor are they more likely to become a member of the panel when asked at the end of the survey. We conclude with a discussion of autoforwarding in web surveys and methods to attract smartphone users to web surveys

Conditional distributions of frame variables and voting behaviour in probability‐based surveys and opt‐in panels

Probability-based web surveys are increasingly challenged by decreasing response rates and high costs. A cheap and convenient solution is to use ‘opt-in’ online panels, which are based on non-probability samples. However, the quality of the data such panels produce is subject to debate. To improve our understanding in this regard, especially in the Swiss context, we compare conditional distributions of sociodemographic variables and voting behaviour of two probability-based web surveys and three opt-in panels. Indeed, point estimates in opt-in panels are well studied, but bivariate relationships between variables, arguably more important for researchers in political science research, have received less attention. Our analysis has the advantage of most variables of interest being included in the sampling frame and thus the true values are known for each conditional distribution. Our results show a lack of consistency and reproducibility in the results from opt-in panels, which leads us to recommend care when using this type of data

How Do Internet-Related Characteristics Affect Whether Members of a German Mixed-Mode Panel Switch from the Mail to the Web Mode?

In recent years, several longitudinal studies have transitioned from an interviewer-administered to a mixed-mode design, using the internet as one of the modes of data collection. However, a substantial proportion of panelists are reluctant to participate in web surveys when offered a choice in an ongoing mixed-mode panel. We still know little about the characteristics of panel members that drive them to comply with the request to complete surveys via the internet. This study aims to fill this gap by investigating how internet-related characteristics are linked to the willingness of panelists to switch from the mail mode to the web. We use data from multiple waves of the GESIS Panel, a probability-based mixed-mode panel in Germany (N = 5734). A web-push intervention motivated 28% of 1364 panelists of the mail mode to complete the survey online in a single wave and 70% of these 380 short-term switchers to switch to the web mode permanently. We measured indicators of internet use, internet skills, and attitudes toward the internet as potential mechanisms of this short-term and long-term mode switching in the two waves before the intervention. Our results suggest that internet use and internet skills affect respondents’ willingness to switch modes in a single wave. For these short-term switchers, however, none of the internet-related characteristics could explain mode switching in the long term. We also present self-reported reasons by panelists for not accepting the offer to switch modes that correspond to these findings. The results of this study can be used to develop effective push-to-web methods for longitudinal mixed-mode surveys

Mobile Web Surveys: a First Look at Measurement, Nonresponse, and Coverage Errors.

This dissertation focuses on the use of smartphones for Web surveys. The current state of knowledge about whether respondents are willing and able to accurately record their answers when using such devices is evolving, but far from complete. The primary purpose of my research is therefore to investigate the implications of this new mode for various sources of error using a Total Survey Error (TSE) perspective. Each chapter reports on a different aspect of a mode experiment that I designed to compare the effect of completion device (smartphone vs. computer) on survey errors. The experiment was carried out using the LISS panel (Longitudinal Internet Studies for the Social Sciences), a probability-based Web panel administered by CentERdata at Tilburg University in the Netherlands. The first analysis (Chapter 2) compares response quality in the two modes. When using smartphones, respondents in this study really were more mobile and more engaged with the other people and other tasks compared to when using computers. Despite this, response quality – conscientious responding and disclosure of sensitive information – was equivalent between the two modes of data collection. The second analysis (Chapter 3) investigates the causes of nonresponse in the mobile Web version of the experiment. I found that several social, psychological, attitudinal, and behavioral measures are associated with nonresponse. These include factors known to influence participation decisions in other survey modes such as personality traits, civic engagement, and attitudes about surveys as well as factors that may be specific to this mode, including smartphone use, social media use, and smartphone e-mail use. The third analysis (Chapter 4) estimates multiple sources of error simultaneously in the mobile Web version of the experiment. Errors are estimated as a mode effect against the conventional Web survey, which serves as the benchmark. I find few overall mode effects and no evidence whatsoever of measurement effects, but a significant impact of non-coverage bias for over one-third of the estimates. Collectively, these findings suggest that non-observation errors (i.e., coverage and nonresponse), not measurement errors, are the largest obstacle to the adoption of mobile Web surveys for population-based inference.PhDSurvey MethodologyUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/116722/1/antoun_1.pd

General and alcohol-related social media use and mental health: A large-sample longitudinal study

The current study aimed to investigate if general and alcohol-related social media use predicts symptoms of depression and anxiety. Students in Bergen, Norway, participated in a Web-based survey during fall 2015 (T1) and a follow-up survey during fall 2016 (T2). A total of 5217 participated in both surveys. Crude and adjusted linear regression analyses were conducted to investigate if social media use at T1 predicted depression and anxiety at T2. Several social media use variables (e.g., using Instagram) were positively associated with depression and anxiety over time, but these associations became non-significant when covariates were controlled for. Number of online friends was inversely related to depression whereas using Twitter was positively related to anxiety at T2, when covariates were controlled for. The effect sizes of the observed associations were all very small. The current study found little support for a relationship between social media use and mental health.publishedVersio