Assessing Trends and Decomposing Change in Nonresponse Bias: The Case of Bias in Cohort Distributions

Survey research is still confronted by a trend of increasing nonresponse rates. In this context, several methodological advances have been made to stimulate participation and avoid bias. Yet, despite the growing number of tools and methods to deal with nonresponse, little is known about whether nonresponse biases show similar trends as nonresponse rates and what mechanisms (if any) drive changes in bias. Our article focuses on biases in cohort distributions in the U.S. and German general social surveys from 1980 to 2012 as one of the key variables in the social sciences. To supplement our cross-national comparison of these trends, we decompose changes into within-cohort change (WCC) and between-cohort change. We find that biases in cohort distributions have remained relatively stable and at a relatively low level in both countries. Furthermore, WCC (i.e., survey climate) accounts for the major part of the change in nonresponse bias

The Interorganizational Relationships of a Public Welfare Agency

The American social welfare field is best characterized as a highly decentralized sphere of activity in which autonomous organizations define and pursue their goals in a fairly independent fashion. The complex nature of modern social problems, however, requires concerted action by a variety of organizations if effective solutions are to be developed. This conflict between the structural nature of the welfare field and the demands of the problems to be addressed has meant that social welfare planners have had to be concerned with the conditions affecting the willingness of independent organizations to engage in cooperative activities with each other. The purposes of the present paper are twofold: (1) To identify some of the major variables that affect the interorganizational activities of social welfare organizations; and (2) to describe the actual interorganizational patterns of one such organization, a county board of public assistance

Understanding Respondents' Attitudes Toward Web Paradata Use

The collection and use of paradata is gaining in importance, especially in web surveys. From a research ethics’ perspective, respondents should be asked for their consent to the collection and use of web paradata. In this context, a positive attitude toward paradata use has been deemed to be a prerequisite for respondents’ willingness to share their paradata. The present study aimed to identify factors affecting respondents’ attitudes toward paradata use. Our findings revealed that adequately informing survey respondents about what paradata are and why they are used was an important determinant of their attitudes toward paradata use. Moreover, we found that respondents with a positive attitude toward the survey were more likely to have a favorable opinion of paradata use. Our findings suggest that a thorough understanding of the factors that contribute to a positive attitude toward paradata use provides the basis for improved paradata consent procedures, which in turn will increase rates of consent to paradata use and help attenuate the risk of consent bias in web surveys

The Influence of a Multidisciplinary Scientific Research Experience on Teachers Views of Nature of Science

This study examined a professional development project for K-12 science teachers that engaged participants in an authentic scientific investigation along with explicit-reflective attention to nature of science (NOS). The Views of Nature of Science (VNOS) and Views of Scientific Inquiry (VOSI) Questionnaires (Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002; Schwartz, Lederman, & Thompson, 2001) were used to examine the relationship between teachers’ views of NOS and specific aspects of the professional development project. Results of the study show that teachers’ views of NOS were influenced by the multidisciplinary, primarily non-experimental research that they engaged in, the opportunity to observe interactions of scientists from different disciplines, and explicit classroom activities and discussions regarding NOS

Early and Late Participation during the Field Period: Response Timing in a Mixed-Mode Probability-Based Panel Survey

Reluctance of respondents to participate in surveys has long drawn the attention of survey researchers. Yet, little is known about what drives a respondent's decision to answer the survey invitation early or late during the field period. Moreover, we still lack evidence on response timing in longitudinal surveys. That is, the questions on whether response timing is a rather stable respondent characteristic and what - if anything - affects change in response timing across different interviews remain open. We relied on data from a mixed-mode general population panel survey collected between 2014 and 2016 to study the stability of response timing across 18 panel waves and factors that influence the decision to participate early or late in the field period. Our results suggest that the factors which had effects on response timing are different in the mail and web modes. Moreover, we found that experience with prior panel waves affected the respondent's decision to participate early or late. Overall, the present study advocates understanding response timing as a metric variable and, consequently, the need to reflect this in modeling strategies

Good questions, bad questions? A Post-Survey Evaluation Strategy Based on Item Nonresponse

In this paper we discuss a three-step strategy to evaluate data quality in terms of item nonresponse and to identify potentially flawed questions. We provide an example with several data sets of a large-scale social scientific study to illustrate the application of the strategy and to highlight its benefits. In survey research it is common practice to test questions ex ante, for example by means of cognitive pretesting. Nevertheless, it is necessary to check the respondents’ response behavior throughout the questionnaire to evaluate the quality of the collected data. Articles addressing item nonresponse mostly focus on individuals or specific questions – adjusting the focus on the questionnaire as a whole seems to be a fruitful addition for survey methodology. Shifting the perspective enables us to identify problematic questions ex post and adjust the questionnaire or research design before re-applying it to further studies or to assess the data quality of a study. This need may arise from shortcomings or failures during the cognitive pretesting or as a result of unforeseen events during the data collection. Furthermore, result of this ex post analysis may be an integral part of data quality reports

A Note on How Prior Survey Experience With Self-Administered Panel Surveys Affects Attrition in Different Modes

Attrition poses an important challenge for panel surveys. With respect to these surveys, respondents’ decisions about whether to participate in reinterviews are affected by their participation in prior waves of the panel. However, in self-administered mixed-mode panels, the way of experiencing a survey differs between the mail mode and the web mode. Consequently, this study investigated how respondents' prior experience with the characteristics of a survey - such as length, difficulty, interestingness, sensitivity, and the diversity of the questionnaire - affects their informed decision about whether to participate again or not. We found that the length of a questionnaire seems to be of such importance to respondents that they base their participation on this characteristic, regardless of the mode. Our findings also suggest that the difficulty and diversity of questionnaires are readily accessible information that respondents use in the mail mode when making a decision about whether to participate again, whereas these characteristics have no effect in the web mode. In addition, privacy concerns have an impact in the web mode but not in the mail mode

Relying on External Information Sources When Answering Knowledge Questions in Web Surveys

Knowledge questions frequently are used in survey research to measure respondents’ topic-related cognitive ability and memory. However, in self-administered surveys, respondents can search external sources for additional information to answer a knowledge question correctly. In this case, the knowledge question measures accessible and procedural memory. Depending on what the knowledge question aims at, the validity of this measure is limited. Thus, in this study, we conducted three experiments using a web survey to investigate the effects of task difficulty, respondents’ ability, and respondents’ motivation on the likelihood of searching external sources for additional information as a form of over-optimizing response behavior when answering knowledge questions. We found that the respondents who are highly educated and more interested in a survey are more likely to invest additional efforts to answer knowledge questions correctly. Most importantly, our data showed that for these respondents, a more difficult question design further increases the likelihood of over-optimizing response behavior

A Reconnaissance Survey of the Environmental Chemistry in East-Central Ellesmere Island, N.W.T.

Snow-pack and surface water samples were collected from east-central Ellesmere Island near Cape Herschel between May and August in 1979-81 to ascertain whether anthropogenic pollution was detectable in a remote "pristine" arctic environment. Snow-pack samples were analyzed for organochlorine pesticide residues, polynuclear aromatic hydrocarbons, and chlorophenoxy acid herbicides. Precipitation and surface water samples were analyzed to determine whether the region has been subjected to "acid rain". In addition, the surface water samples were analyzed for as many as 35 inorganic parameters to provide background data on the water quality of the region. Measurable concentrations of Lindane (gamma BHC) and its isomer alpha BHC, HEOD (dieldrin), and DDT were detected at a number of sites, but no polynuclear aromatic hydrocarbons (PAHs) or chlorophenoxy acid herbicides were detected. The pattern of pesticide residues in this remote area of the Arctic is presumptive evidence that the residues are globally dispersed through the atmosphere. Only copper and the lithophilic metals aluminum and iron were consistently detectable in the snow-pack and surface water samples; all other metals were at or below their detection limits. Thus anthropogenic inputs of metal contaminants such as arsenic, cadmium, lead, mercury, selenium, and vanadium, via atmospheric deposition, were not detected in this region. Although "acid rain" was not in evidence in the study area, the surface waters of the local ponds and lakes, many of which are ombrogenic, are potentially susceptible to changes in the acidity of the atmospheric aerosol of the high Arctic.Key words: snow-pack, precipitation, pesticides, PAHs, acid rain, water chemistry, metals, Arctic, Ellesmere IslandMots clés: manteau nival, précipitation, pesticides, HAP, pluis acides, chimie aquatique, métaux, Arctique, l'île d'Ellesmer