Estimating stochastic survey response errors using the multitrait‐multierror model

From Wiley via Jisc Publications RouterHistory: received 2018-09-17, rev-recd 2021-01-26, accepted 2021-05-30, pub-electronic 2021-10-12Article version: VoRPublication status: PublishedFunder: ESRC National Centre for Research Methods, University of Southampton; Id: http://dx.doi.org/10.13039/501100000613; Grant(s): R121711Abstract: Surveys are well known to contain response errors of different types, including acquiescence, social desirability, common method variance and random error simultaneously. Nevertheless, a single error source at a time is all that most methods developed to estimate and correct for such errors consider in practice. Consequently, estimation of response errors is inefficient, their relative importance is unknown and the optimal question format may not be discoverable. To remedy this situation, we demonstrate how multiple types of errors can be estimated concurrently with the recently introduced ‘multitrait‐multierror’ (MTME) approach. MTME combines the theory of design of experiments with latent variable modelling to estimate response error variances of different error types simultaneously. This allows researchers to evaluate which errors are most impactful, and aids in the discovery of optimal question formats. We apply this approach using representative data from the United Kingdom to six survey items measuring attitudes towards immigrants that are commonly used across public opinion studies

Estimating stochastic survey response errors using the multitrait‐multierror model

Surveys are well known to contain response errors of different types, including acquiescence, social desirability, common method variance and random error simultaneously. Nevertheless, a single error source at a time is all that most methods developed to estimate and correct for such errors consider in practice. Consequently, estimation of response errors is inefficient, their relative importance is unknown and the optimal question format may not be discoverable. To remedy this situation, we demonstrate how multiple types of errors can be estimated concurrently with the recently introduced ‘multitrait-multierror’ (MTME) approach. MTME combines the theory of design of experiments with latent variable modelling to estimate response error variances of different error types simultaneously. This allows researchers to evaluate which errors are most impactful, and aids in the discovery of optimal question formats. We apply this approach using representative data from the United Kingdom to six survey items measuring attitudes towards immigrants that are commonly used across public opinion studies

On the relation of classical and temporal planning

Abstract This paper explores a research strategy for a uniform and rational reconstruction of AI planning techniques. The strategy relies on two assumptions: (1) classical planners like STRIPS or SNLP are restricted variants of temporal planners like DEVISER, and (2) temporal planners may be best constructed atop a time map manager (TMM). The strategy aims at a reconstruction of timeless, classical as well as temporal systems in a TMM-based architectural framework. However, this paper shows that assumed restricted variants of DEVISER cannot be adequately recast in the TMM framework: this result is shown to hold for classical nonlinear planners like SNLP, and one reasonable extension by possibly simultaneous actions. Hence, in accordance with recent complexity results, this paper calls the intutively appealing research strategy into question. Motivation Classical planning in the tradition of STRIPS DEVISER The rational reconstruction of planners described in the literature is an important objective of current research. Chapman We are interested in planners which may be located in the middle-ground between classical nonlinear planners like NOAH When pursuing the research strategy assuming that any point in the spectrum is a restricted variant of DEViSER-style systems, and that general temporal planners may be adequately constructed on top of a TMM, two unexpected problems arise: equivalent recasting of neither (1) classical nonlinear planners nor (2) reasonable extension by simultaneous actions is possible within the TMM framework, when keeping the idea of clearly separating temporal reasoning and plan construction. These problems are caused by the facts that (1) conflict-free time map does not neccessarily represent a correct nonlinear plan, and (2) there is (at least) one sonable but not recastable alternative to TMM's built-in criterion for simultaneous exeeutability of actions. The paper has the following structure: After introducing basic definitions in section 2, two reasonable approaches to extending the planning framework by simultaneous actions are described in section 3. The TMMbased construction of the temporal, nonlinear planner 1 Recent complexity considerations 4

Response styles in consumer research

In researchers’ efforts to better understand consumers, questionnaires are an indispensable source of data. In consumer surveys the Likert item format, where respondents rate their agreement with specific statements, is very popular. However, responses to such items may be biased by response styles, defined as respondents’ tendencies to disproportionately select specific response options. A well-known example is the acquiescence response style, i.e. the tendency to disproportionately use the response options expressing agreement, but respondents may also make disproportionate use of the extreme options, the midpoint option, or the options expressing disagreement. Despite repeated warnings regarding the biasing effect of response styles, most survey research does not control or correct for their impact. A reason for this may be the incomplete understanding of response styles and their antecedents, as well as the difficulties encountered in measuring response styles. The research programme reported in this dissertation aimed to contribute to the understanding of response styles in consumer research by further crystalizing the conceptualization of response styles, by optimizing measurement of response styles, and by explaining the processes that underly response styles. To this end, five empirical studies were carried out. A first study investigated respondents’ understanding of reversed items in questionnaires. Reversed items relate to the same construct as their non-reversed counterparts, but in the opposite direction (e.g. ‘I love to buy new products’ is a reversal of ‘I dislike the purchase of innovations’). This study indicated that responses to items are influenced by the presence of other items that measure the same construct. The exact functional form of this influence is different for reversals and non-reversals, indicating a difference in the way respondents process both types of items. Since this study questioned the validity of reversals for measuring response styles, in the subsequent studies a measurement method for response styles was proposed that captures response tendencies across random samples of items. In a second study, it was shown that response styles are tendencies which are largely stable over the course of a single questionnaire administration. Study 3 established response styles as largely stable tendencies across different questionnaire administrations with a one year time gap in between and using different sets of questions. A fourth study compared response styles across different modes of data collection (self-administered paper and pencil questionnaires, telephone interviews and self-administered online questionnaires). This study showed that there may be differences in response styles across modes of data collection that cannot be detected by the traditional measurement invariance tests. A fifth and final study found two major segments of respondents that differ in the way they satisfice, i.e. economize on the time and effort invested in responding to questionnaire items. One group tends to disproportionately use the midpoint when satisficing. A second group, when satisficing, disproportionately uses the midpoint as well as the negative and positive extremes of the response scale. In sum, though many questions remain unresolved, this dissertation contributes to a better understanding of response styles. More specifically, theory is enhanced by (1) a further delineation of the concept of response styles, which is translated in a proposed operationalization of response styles, (2) evidence in support of the stabililty of response styles, (3) the establishment of response styles as a potential biasing factor in cross-mode comparisons, and (4) a model that captures the relation of response styles to satisficing

New Perspectives on the System Usage Construct

Information systems are designed to support human and organizational purposes. To achieve their ends, information systems must be used. Although this may seem to be self-evident, there are many aspects of systems usage that are not so, and yet, in spite of this, there has been little intense conceptual scrutiny of this construct in past research. The objective of this thesis, therefore, is to develop new in-depth perspectives for studying system usage. Drawing on critical realist assumptions and studies of research diversity, I explain how epistemological factors enable while ontological factors constrain the diversity of meanings of system usage, and I build on this reasoning to advance a systematic approach for conceptualizing and measuring system usage in an appropriate way for a given research context. To demonstrate the approach and judge its usefulness, I carry out three empirical studies to test whether measures of system usage that are selected according to the proposed approach provide more explanatory power and lead to more coherent results in specific research contexts than other measures of system usage. Exploring the relationship between system usage and user task performance among 804 users of spreadsheet software, the experiments reveal support for the usefulness of the approach and demonstrate how it can enable researchers to conceptualize and measure system usage in an appropriate manner for a given research context. Together, the conceptual approach and empirical studies contribute by: (1) providing a systematic way to conceptualize and measure system usage for a given study context, (2) revealing rich new directions for research on the nature of system usage, its antecedents, and its consequences, and (3) suggesting a new approach for construct development and investigation in IS research

Aerobic Exercise for the Promotion of Healthy Aging: Changes in Brain Structure Assessed with New Methods

As the proportion of older individuals in the population increases, so does the scientific concern surrounding age-related deterioration of brain tissue and related cognitive decline. One modifiable lifestyle factor of interest in the pursuit to slow or even reverse age-related brain atrophy is aerobic exercise. A number of studies have already demonstrated that aerobic exercise in older age can induce maintenance (i.e., reduction of loss) of both gray and white matter volume, particularly in the frontal regions of the brain, which are vulnerable to shrinkage in older age. Other magnetic resonance imaging (MRI)-based techniques, such as quantitative MRI and diffusion-weighted MRI, have been used to measure age-related deterioration of gray and white matter integrity in both voxel-wise analyses as well as on the latent level, but whether these negative changes can be ameliorated through exercise has yet to be shown. The current dissertation includes three papers which used a number of both established and novel MRI-based metrics to quantify changes in brain tissue integrity resulting from aging, as well as to investigate whether these changes can be ameliorated through aerobic exercise. In Paper I (Wenger et al., 2022), we tested the reliability of quantitative MRI measures, namely longitudinal relaxation rate, effective transverse relaxation rate, proton density, and magnetization transfer saturation, by measuring them in a two-day, four-session design with repositioning in the scanner. Using the intra-class effect decomposition model, we found that magnetization transfer saturation could reliably detect individual differences, validating its use to investigate changes in brain structure longitudinally, as well as correlations with other variables of interest, such as change in cardiovascular fitness. In Paper II (Polk et al., 2022), we tested the effects of aerobic exercise on a latent factor of gray-matter structural integrity, comprising observed measures of gray-matter volume, magnetization transfer saturation, and mean diffusivity, in regions of interest that have previously shown volumetric effects of aerobic exercise. We found that gray-matter structural integrity was maintained in frontal and midline regions, and that change in gray-matter structural integrity in the right anterior cingulate cortex was positively correlated with change in cardiovascular fitness within exercising participants. These results suggest a causal relationship between aerobic exercise, cardiovascular fitness, and gray-matter structural integrity in this region. In Paper III (Polk et al., 2022), we tested the effects of aerobic exercise on white matter integrity, measured with both established and recently developed metrics. We were able to replicate findings from a previous study on the effects of aerobic exercise on white matter volume, and we also found change-change correlations between white matter volume and cardiovascular fitness as well as between white matter volume and performance on a test of perceptual speed. We also found unexpected exercise-induced changes in the diffusion weighted imaging-derived metrics of fractional anisotropy, mean diffusivity, fiber density, and fiber density and cross-section. Specifically, we found increases (or decreases in the case of mean diffusivity) within control participants and decreases (or increases in mean diffusivity) in exercisers. Furthermore, we found that percent change in fiber density and fiber density and cross-section correlated negatively with percent change in both cardiovascular fitness and cognitive performance. This casts doubt on the functional interpretation of these measures and suggests that the “more is better” principle may not be universally applicable when investigating age-related and exercise-induced changes in white matter integrity. In sum, this dissertation showed that regular at-home aerobic exercise, which may be more accessible for older individuals than supervised exercise, can be an effective tool to ameliorate age-related decreases in a latent measure of gray-matter structural integrity as well as white matter volume. It also illuminated potential limitations of other measures of white matter integrity in the context of aging and aerobic exercise, and calls for further research into these novel measures, especially when considering functional outcomes such as cognitive performance

Alternative Perspectives on Selection: Social Impact and Validation of Graduate Selection within a Multinational Oil Company

This research investigates various theoretical perspectives on graduate selection. Four standpoints are examined: organisational justice in selection, the emergence of the psychological contract, the influence of social moderators on selection predictive validity, and assessment centre construct validity. Two studies were conducted into the graduate selection procedures of a multinational oil company (Shell International). In Study A, applicants were contacted in retrospect and asked about their experiences of the selection process. In Study B, a longitudinal design was employed whereby applicants responded to questionnaires at various time points within the selection process and following four months of employment, In Study B, data were also collected from the organisation, including ratings of applicants' potential at selection and subsequent potential four months post-entry. The results of both studies highlight the dynamic nature of the interaction between potential recruits and the organisation during the selection process. Differences between applicants' prior expectations of procedural justice and their perceptions of reality measured immediately after a selection method, are found to have an impact on several selection outcome variables. The selection decision is found to have a direct impact on applicants' reactions to procedural justice, with unsuccessful applicants having lower perceptions of procedural justice when measured post-, but not pre-communication of the outcome decision. The dynamic nature of the psychological contract is shown from selection to four months post entry, with recruits' perceptions generally becoming more congruent with the organisation's perspective. The results also highlight the potential influence of selection and socialisation moderators of predictive validity. Finally, poor assessment centre construct validity is demonstrated, despite the small number of criteria used and the reasonable predictive validity. Overall, this research illustrates the advantage of conducting integrated research which simultaneously examines multiple perspectives on selection. In conclusion, the original contributions of this research to selection theory, and a number of implications for practice are discussed

Cross-cultural survey equivalence

"This volume, the third in the ZUMA-Nachrichten-Spezial series on methodological issues in empirical social science research, is devoted to issues of cross-cultural methodology. The focus is on issues of equivalence, the key requirement in cross-national and cross-cultural comparative research. As the contributions indicate, equivalence is, however, better thought of in terms of equivalencies - in social science surveys and in other standardised instruments of measurement. Contributors come from different countries and continents and from widely differing research backgrounds, ranging from linguistics to survey research and its methodologies, to cultural anthropology and cross-cultural psychology." (author's abstract). Contents: Timothy P. Johnson: Approaches to equivalence in cross-cultural and cross-national survey research (1-40); Fons J.R. van de Vijver: Towards a theory of bias and equivalence (41-65); Willem E. Saris: The effects of measurement error in cross-cultural research (67-84); Janet A. Harkness, Alicia Schoua-Glusberg: Questionnaires in translation (87-126); Michael Braun, Jacqueline Scott: Multidimensional scaling and equivalence: is having a job the same as working? (129-144); Ingwer Borg: A facet-theoretical approach to item equivalency (145-158); Peter Ph. Mohler, Tom W. Smith, Janet A. Harkness: Respondents' ratings of expressions from response scales: a two-country, two-language investigation on equivalence and translation (159-184)

Measurement properties of respondent-defined rating-scales. An investigation of individual characteristics and respondent choices.

It is critical for researchers to be confident of the quality of survey data. Problems with data quality often relate to measurement method design, through choices made by researchers in their creation of standardised measurement instruments. This is known to affect the way respondents interpret and respond to these instruments, and can result in substantial measurement error. Current methods for removing measurement error are post-hoc and have been shown to be problematic. This research proposes that innovations can be made through the creation of measurement methods that take respondents¿ individual cognitions into consideration, to reduce measurement error in survey data. Specifically, the aim of the study was to develop and test a measurement instrument capable of having respondents individualise their own rating-scales. A mixed methodology was employed. The qualitative phase provided insights that led to the development of the Individualised Rating-Scale Procedure (IRSP). This electronic measurement method was then tested in a large multi-group experimental study, where its measurement properties were compared to those of Likert-Type Rating-Scales (LTRSs). The survey included pre-validated psychometric constructs which provided a baseline for comparing the methods, as well as to explore whether certain individual characteristics are linked to respondent choices. Structural equation modelling was used to analyse the survey data. Whilst no strong associations were found between individual characteristics and respondent choices, the results demonstrated that the IRSP is reliable and valid. This study has produced a dynamic measurement instrument that accommodates individual-level differences, not addressed by typical fixed rating-scales

The Valuation of Health Outcomes: A Contribution to the QALY Approach

The aim of any medical intervention is to improve or sustain health. Therefore, the assessment of health outcome is an important aspect in the scientific evaluation of medical effectiveness. Nowadays this is an important activity. Three major aspects of health outcome can be distinguished. The most prominent is survival, which defines the prevention of premature death as a major goal of care. Disease-specific clinical characteristics of the serio liS ness of the pathology are a second aspect of health outcome. More recently, a third aspect has been developed to express an entity that was previously more or less implicit: health status. Especially in the more prosperous part of the world where life expectancy has shown an impressive increase, a shifr of emphasis from the attention of quantity of life towards the quality of health, i.e., to health status can be noticed