Webdatanet : Innovation and quality in web-based data collection

The article discusses the development of WEBDATANET established in 2011 which aims to create a multidisciplinary network of web-based data collection experts in Europe. Topics include the presence of 190 experts in 30 European countries and abroad, the establishment of web-based teaching and discussion platforms and working groups and task forces. Also discussed is the scope of the research carried by WEBDATANET. In light of the growing importance of web-based data in the social and behavioral sciences, WEBDATANET was established in 2011 as a COST Action (IS 1004) to create a multidisciplinary network of web-based data collection experts: (web) survey methodologists, psychologists, sociologists, linguists, economists, Internet scientists, media and public opinion researchers. The aim was to accumulate and synthesize knowledge regarding methodological issues of web-based data collection (surveys, experiments, tests, non-reactive data, and mobile Internet research), and foster its scientific usage in a broader community

Impact on the R-indicator and the Nonresponse Bias of Targeted Fieldwork: A Simulation Study with the Swiss ESS 2010

No responsive designs have been implemented yet in Switzerland but the use of a population register to draw the sample of the most recent surveys conducted by FORS would allow such a fieldwork implementation. The hard-to-contact and reluctant respondents often follow the usual suspect pattern: single, male, non- (Swiss) citizen, leaving in urban areas. A targeted fieldwork could help to increase the representativity of the sample and hopefully to reduce nonresponse bias. The ESS 2010 Swiss was one of the first FORS survey for which the sample was drawn from a population register that contains basic socio-demographic information on all sample members (marital status, nationality, age, gender, urbanization). This allowed us to study the representativity of the respondent sample, using the R-indicator. We found that after all fieldwork efforts, Swiss citizens, people leaving in urban areas and married people were still overrepresented, as well as people for whom a telephone number could be match using a commercial telephone directory (AZ direct). In Switzerland, not four as required by the ESS but five face-to-face visit attempts are compulsory for each sample members. We will consider the respondents that completed the ESS questionnaire after these 5 first visits as the core respondent sample. We then define the nonrespondent in two groups: low and high response propensity groups. In a first step, we define non-Swiss citizens, and non-married leaving in urban areas as the low propensity group. This splits the sample of nonrespondents after the first five visits roughly in two groups. We then simulate different attribution of the fieldwork effort to the different group, going from no targeted fieldwork at all (100%-100%) to a responsive design that would concentrate all the effort on the high hanging fruit (0%-100%). We then compare data quality indicator like response rate, R-indicator, and maximal absolute bias when different proportion of the high response propensity are include (0%, 25%, 50%, 75% and 100%), mimicking different targeted fieldwork. We then repeat the same analysis where we define the low propensity sample has the one without a telephone number. This variable may seem not relevant but is very discriminating in Switzerland for predicting participation.status: publishe

Using post-stratification or propensity score nonresponse adjustment: bias correction and precision lost – a case study with the Swiss ESS 2012 data

Nonresponse bias in social survey is a well-known and well-studied issue. Many technics are developed to reduce this source of error. During the data collection period different initiatives ranging from incentives, refusal conversion, extra contact attempts, mixing modes, or developing a responsive/targeted fieldwork can help increase participation and even attract respondents with different characteristic to reduce the bias. After the data has been collected, depending on the available paradata, nonresponse adjustment weights can be calculated in the hope to correct for the remaining bias. The problem with the use of weights, especially when response rates are rather low is the loss in precision that it can cause. Indeed, if the nonresponse adjustment weights vary a lot, the confidence interval of point estimates will become larger. The increase in standard error due to the application of weights can in some case counter-balance the decrease in bias. Moreover, a good nonresponse adjustment scheme is based on variables that highly correlate with the propensity to response, such variables are only rarely available. For this reason, developing nonresponse adjustment and the variables used to do so have to be thought of carefully. In this paper, we will study two nonresponse adjustments for the ESS 2012 survey in Switzerland. The first one will be based on socio-demographical variables from the population register from which the sample is drawn. Such paradata are commonly used in post-survey nonresponse adjustment as they are often the only data available but are known to have a low correlation with the propensity to answer as well as with many key variables. In a second step, data from the nonresponse follow-up survey that has been conducted shortly after the main ESS 2012 data collection will also be utilized to construct post-survey nonresponse adjustment. The nonresponse follow-up survey is designed to collect information about respondents that correlates highly with the propensity to answer and should lead to a efficient nonresponse adjustment. The effect on precision of the use of such weights could however be substantial. Moreover, the core of the nonrespondents that did not participate to either survey cannot be corrected for. Our aim is to compare these two methods, assessing the effect on estimates and hopefully on the bias as well as on the precision of these estimates by applying a bootstrapping.status: publishe

Identifying Pertinent Variables for Non-response Follow-Up Surveys. Lessons Learned from 4 Cases in Switzerland

All social surveys suffer from different types of errors, of which one of the most studied is non-response bias. Non-response bias is a systematic error that occurs because individuals differ in their accessibility and propensity to participate in a survey according to their own characteristics as well as those from the survey itself. The extent of the problem heavily depends on the correlation between response mechanisms and key survey variables. However, non-response bias is difficult to measure or to correct for due to the lack of relevant data about the whole target population or sample. In this paper, non-response follow-up surveys are considered as a possible source of information about non-respondents. Non-response follow-ups, however, suffer from two methodological issues: they themselves operate through a response mechanism that can cause potential non-response bias, and they pose a problem of comparability of measure, mostly because the survey design differs between main survey and non-response follow-up. In order to detect possible bias, the survey variables included in non-response surveys have to be related to the mechanism of participation, but not be sensitive to measurement effects due to the different designs. Based on accumulated experience of four similar non-response follow-ups, we studied the survey variables that fulfill these conditions. We differentiated socio-demographic variables that are measurement-invariant but have a lower correlation with non-response and variables that measure attitudes, such as trust, social participation, or integration in the public sphere, which are more sensitive to measurement effects but potentially more appropriate to account for the non-response mechanism. Our results show that education level, work status, and living alone, as well as political interest, satisfaction with democracy, and trust in institutions are pertinent variables to include in non-response follow-ups of general social surveys. - See more at: https://ojs.ub.uni-konstanz.de/srm/article/view/6138#sthash.VFOJNptl.dpufstatus: publishe

Identifying pertinent variables for nonresponse follow-up surveys: lessons learned from four cases in Switzerland

"All social surveys suffer from different types of errors, of which one of the most studied is non-response bias. Non-response bias is a systematic error that occurs because individuals differ in their accessibility and propensity to participate in a survey according to their own characteristics as well as those from the survey itself. The extent of the problem heavily depends on the correlation between response mechanisms and key survey variables. However, non-response bias is difficult to measure or to correct for due to the lack of relevant data about the whole target population or sample. In this paper, non-response follow-up surveys are considered as a possible source of information about non-respondents. Non-response follow-ups, however, suffer from two methodological issues: they themselves operate through a response mechanism that can cause potential non-response bias, and they pose a problem of comparability of measure, mostly because the survey design differs between main survey and non-response follow-up. In order to detect possible bias, the survey variables included in non-response surveys have to be related to the mechanism of participation, but not be sensitive to measurement effects due to the different designs. Based on accumulated experience of four similar non-response follow-ups, we studied the survey variables that fulfill these conditions. We differentiated socio-demographic variables that are measurement-invariant but have a lower correlation with non-response and variables that measure attitudes, such as trust, social participation, or integration in the public sphere, which are more sensitive to measurement effects but potentially more appropriate to account for the non-response mechanism. Our results show that education level, work status, and living alone, as well as political interest, satisfaction with democracy, and trust in institutions are pertinent variables to include in non-response follow-ups of general social surveys." (author's abstract