Current Knowledge and Considerations Regarding Survey Refusals: Executive Summary of the AAPOR Task Force Report on Survey Refusals

The landscape of survey research has arguably changed more significantly in the past decade than at any other time in its relatively brief history. In that short time, landline telephone ownership has dropped from some 98 percent of all households to less than 60 percent; cell-phone interviewing went from a novelty to a mainstay; address-based designs quickly became an accepted method of sampling the general population; and surveys via Internet panels became ubiquitous in many sectors of social and market research, even as they continue to raise concerns given their lack of random selection. Among these widespread changes, it is perhaps not surprising that the substantial increase in refusal rates has received comparatively little attention. As we will detail, it was not uncommon for a study conducted 20 years ago to have encountered one refusal for every one or two completed interviews, while today experiencing three or more refusals for every one completed interview is commonplace. This trend has led to several concerns that motivate this Task Force. As refusal rates have increased, refusal bias (as a component of nonresponse bias) is an increased threat to the validity of survey results. Of practical concern are the efficacy and cost implications of enhanced efforts to avert initial refusals and convert refusals that do occur. Finally, though no less significant, are the ethical concerns raised by the possibility that efforts to minimize refusals can be perceived as coercive or harassing potential respondents. Indeed, perhaps the most important goal of this document is to foster greater consideration by the reader of the rights of respondents in survey research

Recruitment using mobile telephones in an Irish general population sexual health survey: challenges and practical solutions

BACKGROUND: Non-coverage of households without a landline telephone is a major concern of telephone survey researchers. Sampling mobile telephone users in national surveys is vital in order to gain access to the growing proportion of households that use mobile telephones extensively or exclusively. The complex logistics of conducting surveys with mobile telephones have been discussed in the literature. This paper outlines the actual challenges encountered during a recent national sexual health survey in Ireland, which utilized a mobile telephone sampling frame to recruit approximately half of the sample. METHOD: The 2010 Irish Contraception and Crisis Pregnancy Survey (ICCP-2010) is a nationally representative sample of adults aged 18-45 years living in Ireland (n = 3002; 1416 recruited by landline telephone and 1586 recruited by mobile telephone). The overall response rate for the survey was 69% (79% for the landline telephone strand; 61% for the mobile telephone strand). All interviews were conducted using computer-assisting telephone interviewing. RESULTS: During the 18-week fieldwork period, five main challenges relating to the use of mobile telephones were encountered: (1) explaining to respondents how random digit dialling works in relation to mobile telephones; (2) establishing the respondent\u27s eligibility; (3) calling the respondent with the Caller ID blocked or withheld; (4) calling the respondent when they are in any number of locations or situations; and (5) explaining to respondents the importance of refusal conversion calls for the response rate calculation. Details of how the survey protocols and procedures were monitored and adapted throughout the study to ensure a high response rate are outlined. CONCLUSION: It is undeniably more challenging to recruit respondents using mobile telephones as opposed to landline telephones. Respondents are generally not familiar with being contacted on their personal mobile telephone for the purposes of being recruited for a research study. The main challenge for survey methodologists and interviewers is to devise simple protocols to explain to respondents why they are being contacted on a mobile telephone. Recommendations for survey researchers interested in using this methodological approach in the future are discussed

Water quality monitoring records for estimating tap water arsenic and nitrate: a validation study

<p>Abstract</p> <p>Background</p> <p>Tap water may be an important source of exposure to arsenic and nitrate. Obtaining and analyzing samples in the context of large studies of health effects can be expensive. As an alternative, studies might estimate contaminant levels in individual homes by using publicly available water quality monitoring records, either alone or in combination with geographic information systems (GIS).</p> <p>Methods</p> <p>We examined the validity of records-based methods in Washington State, where arsenic and nitrate contamination is prevalent but generally observed at modest levels. Laboratory analysis of samples from 107 homes (median 0.6 μg/L arsenic, median 0.4 mg/L nitrate as nitrogen) served as our "gold standard." Using Spearman's rho we compared these measures to estimates obtained using only the homes' street addresses and recent and/or historical measures from publicly monitored water sources within specified distances (radii) ranging from one half mile to 10 miles.</p> <p>Results</p> <p>Agreement improved as distance decreased, but the proportion of homes for which we could estimate summary measures also decreased. When including all homes, agreement was 0.05-0.24 for arsenic (8 miles), and 0.31-0.33 for nitrate (6 miles). Focusing on the closest source yielded little improvement. Agreement was greatest among homes with private wells. For homes on a water system, agreement improved considerably if we included only sources serving the relevant system (ρ = 0.29 for arsenic, ρ = 0.60 for nitrate).</p> <p>Conclusions</p> <p>Historical water quality databases show some promise for categorizing epidemiologic study participants in terms of relative tap water nitrate levels. Nonetheless, such records-based methods must be used with caution, and their use for arsenic may be limited.</p

Study of Radiologic Technologists’ Perceptions of Picture Archiving and Communication System (PACS) Competence and Educational Issues in Western Australia

Although the implementation of picture archiving and communication system (PACS) could increase productivity of radiology departments, this depends on factors such as the PACS competence of radiologic technologists (RTs). The purpose of this study was to investigate the RTs’ perceptions of PACS competence and educational issues in Western Australia (WA). A hardcopy questionnaire was distributed to WA RTs for obtaining their perceptions of PACS competence and educational issues. Descriptive (percentage of frequency, mean and standard deviation) and inferential statistics (t test and analysis of variance) were used to analyze the responses of the multiple choice and five-point scale questions from the returned questionnaires. The questionnaire response rate was 57.7 % (173 out of 300). The mean values of all PACS competence questions except questions 2e–g are in the range of 3.9–4.9, i.e., around competent to very competent. Participants indicated they received adequate PACS training (mean 3.8). Statistically significant variables influencing RTs’ perceptions of their PACS competence and educational issues including the age (p < 0.01), gender (p < 0.05), years of practice (p < 0.005–0.05), primary duty (p < 0.05), medical imaging qualification (p < 0.001), general computer skills (p < 0.001), and type of PACS education received (p < 0.001–0.05). The WA RTs indicated that they were competent in using the modality workstation, PACS and radiology information system, and received adequate training. However, future PACS education programs should be tailored to different RTs’ groups. For example, multiple training modules might be necessary to support the PACS competence development of older RTs and those with lower general computer literacy

Assessing Order Effects in Online Community-based Health Forums

Measuring the quality of health content in online health forums is a challenging task. The majority of the existing measures are based on evaluations of forum users and may not be reliable. We employed machine learning techniques, text mining methods, and Big Data platforms to construct four measures of textual quality to automatically determine the similarity of a given answer to professional answers. We then used them to assess the quality of 66,888 answers posted on Yahoo! Answers Health section. All four measures of textual quality revealed a higher quality for asker-selected best answers indicating that askers, to some extent, have a proper judgment to select the best answers. We also studied the presence of order effects in online health forums. Our results suggest that the textual quality of the first answer positively influences the mean textual quality of the subsequent answers and negatively influences the quantity of subsequent answers

Evaluating the quality of social work supervision in UK children's services: comparing self-report and independent observations

Understanding how different forms of supervision support good social work practice and improve outcomes for people who use services is nearly impossible without reliable and valid evaluative measures. Yet the question of how best to evaluate the quality of supervision in different contexts is a complicated and as-yet-unsolved challenge. In this study, we observed 12 social work supervisors in a simulated supervision session offering support and guidance to an actor playing the part of an inexperienced social worker facing a casework-related crisis. A team of researchers analyzed these sessions using a customized skills-based coding framework. In addition, 19 social workers completed a questionnaire about their supervision experiences as provided by the same 12 supervisors. According to the coding framework, the supervisors demonstrated relatively modest skill levels, and we found low correlations among different skills. In contrast, according to the questionnaire data, supervisors had relatively high skill levels, and we found high correlations among different skills. The findings imply that although self-report remains the simplest way to evaluate supervision quality, other approaches are possible and may provide a different perspective. However, developing a reliable independent measure of supervision quality remains a noteworthy challenge

Opinion climates and immigrant political action: a cross-national study of 25 European democracies

We develop a model of immigrant political action that connects individual motivations to become politically involved with the context in which participation takes place. The article posits that opinion climates in the form of hostility or openness toward immigrants shape the opportunity structure for immigrant political engagement by contributing to the social costs and political benefits of participation. We argue that friendly opinion climates toward immigrants enable political action among immigrants, and facilitate the politicization of political discontent. Using survey data from the European Social Survey (ESS) 2002 to 2010 in 25 European democracies, our analyses reveal that more positive opinion climates-at the level of countries and regions-increase immigrant political engagement, especially among immigrants dissatisfied with the political system. However, this effect is limited to uninstitutionalized political action, as opinion climates have no observable impact on participation in institutionalized politics

Social environmental impacts on survey cooperation

Social environmental influences on survey cooperation are explored using data from six national household surveys in the United States matched to 1990 decennial census data. Consistent with the past literature on prosocial behavior, cooperation rates in these six surveys are found to be lower in urban, densely populated, high crime rate areas. Measures of social cohesion show no evidence of influencing cooperation. The influence of the environmental variables is then observed after introducing statistical controls for household structure, race, age of household members, presence of children, and socioeconomic attributes of households. Over half of the measured influence of the environmental variables is explained by these household-level attributes. These findings have practical import for survey administrators and are informative for the construction of a theory of survey participation.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43559/1/11135_2004_Article_BF00153986.pd