Data quality in telephone surveys and the effect of questionnaire length: a cross-national experiment

Respondents in long telephone survey interviews may adopt satisficing strategies as they approach the end of the questionnaire (Holbrook, Green and Krosnick, 2003). However, there is inconsistency regarding the relationship between questionnaire length and different forms of satisficing. We investigate whether long questionnaires are associated with a reduction in response quality using data from a cross-national survey experiment. Sample members were randomly assigned to interviews of 60, 45 or 30 minutes. We compare responses to attitudinal measures from a module on happiness and well-being, which was asked at different points in the interview in each of the three groups

Item Wording and Internal Consistency of a Measure of Cohesion: The Group Environment Questionnaire

A common practice for counteracting response acquiescence in psychological measures has been to employ both negatively and positively worded items. However, previous research has highlighted that the reliability of measures can be affected by this practice (Spector, 1992). The purpose of the present study was to examine the effect that the presence of negatively worded items has on the internal reliability of the Group Environment Questionnaire (GEQ). Two samples (N = 276) were utilized, and participants were asked to complete the GEQ (original and revised) on separate occasions. Results demonstrated that the revised questionnaire (containing all positively worded items) had significantly higher Cronbach α values for three of the four dimensions of the GEQ. Implications, alternatives, and future directions are discussed

Models for Anchoring and Acquiescence Bias in Consumption Data

Item non-response in household survey data on economic variables such as income, assets or consumption is a well-known problem.Follow-up unfolding bracket questions have been used as a tool to collect partial information on respondents that do not answer an open-ended question.It is also known, however, that mistakes are made in answering such unfolding bracket questions.In this paper, we develop several limited dependent variable models to analyze two sources of mistakes, anchoring and acquiescence (or yeasaying), focusing on the first bracket question.We use the experimental module of the AHEAD 1995 data, where the sample is randomly split into respondents who get an open-ended question on the amount of total family consumption - with follow-up unfolding brackets (of the form: is consumption $X or more?) for those who answer don t know or refuse - and respondents who are immediately directed to unfolding brackets.In both cases, the entry point of the unfolding bracket sequence is randomized.We compare models in which the probability of a mistake depends on the deviation between the true consumption amount and the entry point amount$X and models in which it does not.We find that allowing for acquiescence bias substantially changes the conclusions on the selective nature of non-response to the open-ended question and on the distribution of consumption expenditures in the population.Once acquiescence bias is taken into account, anchoring in the first bracket question plays only a minor role.consumption;household economics;distribution;nonresponse

Causes of Mode Effects: Separating out Interviewer and Stimulus Effects in Comparisons of Face-to-Face and Telephone Surveys

We identify the causes of mode effects in comparisons of face-to-face and telephone surveys, by testing for differences in the extent of satisficing and social desirability bias due to differences in the stimulus (visual vs. aural presentation of response options) and the presence vs. absence of the interviewer. The stimulus did not lead to differential measurement error; the presence or absence of the interviewer however did. Telephone respondents were far more likely to give socially desirable responses than face-to-face respondents when the stimulus was the same for both modes

Identifying Unknown Response Styles: A Latent-Class Bilinear Multinomial Logit Model

Respondents can vary significantly in the way they use rating scales. Specifically, respondents can exhibit varying degrees of response style, which threatens the validity of the responses. The purpose of this article is to investigate to what extent rating scale responses show response style and substantive content of the item. The authors develop a novel model that accounts for possibly unknown kinds of response styles, content of the items, and background characteristics of respondents. By imposing a bilinear structure on the parameters of a multinomial logit model, the authors can visually distinguish the effects on the response behavior of both the characteristics of a respondent and the content of the item. This approach is combined with finite mixture modeling, so that two separate segmentations of the respondents are obtained: one for response style and one for item content. This latent-class bilinear multinomial logit (LC-BML) model is applied to a cross-national data set. The results show that item content is highly influential in explaining response behavior and reveal the presence of several response styles, including the prominent response styles acquiescence and extreme response style.multinomial logit model;visualization;segmentation;cross-cultural research;response style

Surveying Persons with Disabilities: A Source Guide (Version 1)

As a collaborator with the Cornell Rehabilitation Research and Training Center on Disability Demographics and Statistics, Mathematica Policy Research, Inc. has been working on a project that identifies the strengths and limitations in existing disability data collection in both content and data collection methodology. The intended outcomes of this project include expanding and synthesizing knowledge of best practices and the extent existing data use those practices, informing the development of data enhancement options, and contributing to a more informed use of existing data. In an effort to provide the public with an up-to-date and easily accessible source of research on the methodological issues associated with surveying persons with disabilities, MPR has prepared a Source Guide of material related to this topic. The Source Guide contains 150 abstracts, summaries, and references, followed by a Subject Index, which cross references the sources from the Reference List under various subjects. The Source Guide is viewed as a “living document,” and will be periodically updated

Verbal Response Modes in Action:Microrelationships as the Building Blocks of Relationship Role Dimensions

Dimensions of interpersonal relationships, such as attentiveness, directiveness, and presumptuousness, have typically been assessed through impressionistic ratings or by aggregate scores derived from coding of specific (e.g., verbal) behaviors. However, the meanings of these dimensions rest on the interpersonal microrelationships that are actually observed by the raters or coders. In this qualitative study, the way these global relationship qualities were built from microrelationships at the utterance level was examined in passages from one medical interaction. Applications of microrelationships to future communications research are suggested

Anchoring vignettes can they make adolescent self-reports of social-emotional skills more reliable, discriminant, and criterion-valid?

Individuals differ in the way they use rating scales to describe themselves, and these differences are particularly pronounced in children and early adolescents. One promising remedy is to correct (or "anchor'') an individual's responses according to the way they use the scale when they rate an anchoring vignette (a set of hypothetical targets differing on the attribute of interest). Studying adolescents' self-reports of their socio-emotional attributes, we compared traditional self-report scores with vignette-corrected scores in terms of reliability (internal consistency), discriminant validity (scale intercorrelations), and criterion validity (predicting achievement test scores in language and math). A large and representative sample of 12th grade Brazilian students (N = 8,582, 62% female, mean age 18.2) were administered a Portuguese-language self-report inventory assessing social-emotional skills related to the Big Five personality dimensions. Correcting scores according to vignette ratings led to increases in the reliability of scales measuring Conscientiousness and Openness, but discriminant validity and criterion validity increased only when each scale was corrected using its own corresponding vignette set. Moreover, accuracy in rating the vignettes was correlated with language achievement test scores, suggesting that verbal factors play a role in providing both normative vignette ratings of others and self-reports that are reliable and valid

An Adaptive Fault-Tolerant Communication Scheme for Body Sensor Networks

A high degree of reliability for critical data transmission is required in body sensor networks (BSNs). However, BSNs are usually vulnerable to channel impairments due to body fading effect and RF interference, which may potentially cause data transmission to be unreliable. In this paper, an adaptive and flexible fault-tolerant communication scheme for BSNs, namely AFTCS, is proposed. AFTCS adopts a channel bandwidth reservation strategy to provide reliable data transmission when channel impairments occur. In order to fulfill the reliability requirements of critical sensors, fault-tolerant priority and queue are employed to adaptively adjust the channel bandwidth allocation. Simulation results show that AFTCS can alleviate the effect of channel impairments, while yielding lower packet loss rate and latency for critical sensors at runtime.Comment: 10 figures, 19 page

Development of a Cohesion Questionnaire for Youth: The Youth Sport Environment Questionnaire

The purpose of the current study was to initiate the development of a psychometrically sound measure of cohesion for youth sport groups. A series of projects were undertaken in a four-phase research program. The initial phase was designed to garner an understanding of how youth sport group members perceived the concept of cohesion through focus groups (n = 56), open-ended questionnaires (n = 280), and a literature review. In Phase 2, information from the initial projects was used in the development of 142 potential items and content validity was assessed. In Phase 3, 227 participants completed a revised 87-item questionnaire. Principal components analyses further reduced the number of items to 17 and suggested a two-factor structure (i.e., task and social cohesion dimensions). Finally, support for the factorial validity of the resultant questionnaire was provided through confirmatory factor analyses with an independent sample (n = 352) in Phase 4. The final version of the questionnaire contains 16 items that assess task and social cohesion in addition to 2 negatively worded spurious items. Specific issues related to assessing youth perceptions of cohesion are discussed and future research directions are suggested