Current Knowledge and Considerations Regarding Survey Refusals: Executive Summary of the AAPOR Task Force Report on Survey Refusals

The landscape of survey research has arguably changed more significantly in the past decade than at any other time in its relatively brief history. In that short time, landline telephone ownership has dropped from some 98 percent of all households to less than 60 percent; cell-phone interviewing went from a novelty to a mainstay; address-based designs quickly became an accepted method of sampling the general population; and surveys via Internet panels became ubiquitous in many sectors of social and market research, even as they continue to raise concerns given their lack of random selection. Among these widespread changes, it is perhaps not surprising that the substantial increase in refusal rates has received comparatively little attention. As we will detail, it was not uncommon for a study conducted 20 years ago to have encountered one refusal for every one or two completed interviews, while today experiencing three or more refusals for every one completed interview is commonplace. This trend has led to several concerns that motivate this Task Force. As refusal rates have increased, refusal bias (as a component of nonresponse bias) is an increased threat to the validity of survey results. Of practical concern are the efficacy and cost implications of enhanced efforts to avert initial refusals and convert refusals that do occur. Finally, though no less significant, are the ethical concerns raised by the possibility that efforts to minimize refusals can be perceived as coercive or harassing potential respondents. Indeed, perhaps the most important goal of this document is to foster greater consideration by the reader of the rights of respondents in survey research

Recruitment using mobile telephones in an Irish general population sexual health survey: challenges and practical solutions

BACKGROUND: Non-coverage of households without a landline telephone is a major concern of telephone survey researchers. Sampling mobile telephone users in national surveys is vital in order to gain access to the growing proportion of households that use mobile telephones extensively or exclusively. The complex logistics of conducting surveys with mobile telephones have been discussed in the literature. This paper outlines the actual challenges encountered during a recent national sexual health survey in Ireland, which utilized a mobile telephone sampling frame to recruit approximately half of the sample. METHOD: The 2010 Irish Contraception and Crisis Pregnancy Survey (ICCP-2010) is a nationally representative sample of adults aged 18-45 years living in Ireland (n = 3002; 1416 recruited by landline telephone and 1586 recruited by mobile telephone). The overall response rate for the survey was 69% (79% for the landline telephone strand; 61% for the mobile telephone strand). All interviews were conducted using computer-assisting telephone interviewing. RESULTS: During the 18-week fieldwork period, five main challenges relating to the use of mobile telephones were encountered: (1) explaining to respondents how random digit dialling works in relation to mobile telephones; (2) establishing the respondent\u27s eligibility; (3) calling the respondent with the Caller ID blocked or withheld; (4) calling the respondent when they are in any number of locations or situations; and (5) explaining to respondents the importance of refusal conversion calls for the response rate calculation. Details of how the survey protocols and procedures were monitored and adapted throughout the study to ensure a high response rate are outlined. CONCLUSION: It is undeniably more challenging to recruit respondents using mobile telephones as opposed to landline telephones. Respondents are generally not familiar with being contacted on their personal mobile telephone for the purposes of being recruited for a research study. The main challenge for survey methodologists and interviewers is to devise simple protocols to explain to respondents why they are being contacted on a mobile telephone. Recommendations for survey researchers interested in using this methodological approach in the future are discussed

Assessing Order Effects in Online Community-based Health Forums

Measuring the quality of health content in online health forums is a challenging task. The majority of the existing measures are based on evaluations of forum users and may not be reliable. We employed machine learning techniques, text mining methods, and Big Data platforms to construct four measures of textual quality to automatically determine the similarity of a given answer to professional answers. We then used them to assess the quality of 66,888 answers posted on Yahoo! Answers Health section. All four measures of textual quality revealed a higher quality for asker-selected best answers indicating that askers, to some extent, have a proper judgment to select the best answers. We also studied the presence of order effects in online health forums. Our results suggest that the textual quality of the first answer positively influences the mean textual quality of the subsequent answers and negatively influences the quantity of subsequent answers

Evaluating the quality of social work supervision in UK children's services: comparing self-report and independent observations

Understanding how different forms of supervision support good social work practice and improve outcomes for people who use services is nearly impossible without reliable and valid evaluative measures. Yet the question of how best to evaluate the quality of supervision in different contexts is a complicated and as-yet-unsolved challenge. In this study, we observed 12 social work supervisors in a simulated supervision session offering support and guidance to an actor playing the part of an inexperienced social worker facing a casework-related crisis. A team of researchers analyzed these sessions using a customized skills-based coding framework. In addition, 19 social workers completed a questionnaire about their supervision experiences as provided by the same 12 supervisors. According to the coding framework, the supervisors demonstrated relatively modest skill levels, and we found low correlations among different skills. In contrast, according to the questionnaire data, supervisors had relatively high skill levels, and we found high correlations among different skills. The findings imply that although self-report remains the simplest way to evaluate supervision quality, other approaches are possible and may provide a different perspective. However, developing a reliable independent measure of supervision quality remains a noteworthy challenge

Opinion climates and immigrant political action: a cross-national study of 25 European democracies

We develop a model of immigrant political action that connects individual motivations to become politically involved with the context in which participation takes place. The article posits that opinion climates in the form of hostility or openness toward immigrants shape the opportunity structure for immigrant political engagement by contributing to the social costs and political benefits of participation. We argue that friendly opinion climates toward immigrants enable political action among immigrants, and facilitate the politicization of political discontent. Using survey data from the European Social Survey (ESS) 2002 to 2010 in 25 European democracies, our analyses reveal that more positive opinion climates-at the level of countries and regions-increase immigrant political engagement, especially among immigrants dissatisfied with the political system. However, this effect is limited to uninstitutionalized political action, as opinion climates have no observable impact on participation in institutionalized politics

Social environmental impacts on survey cooperation

Social environmental influences on survey cooperation are explored using data from six national household surveys in the United States matched to 1990 decennial census data. Consistent with the past literature on prosocial behavior, cooperation rates in these six surveys are found to be lower in urban, densely populated, high crime rate areas. Measures of social cohesion show no evidence of influencing cooperation. The influence of the environmental variables is then observed after introducing statistical controls for household structure, race, age of household members, presence of children, and socioeconomic attributes of households. Over half of the measured influence of the environmental variables is explained by these household-level attributes. These findings have practical import for survey administrators and are informative for the construction of a theory of survey participation.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43559/1/11135_2004_Article_BF00153986.pd