Aligning physical elements with persons' attitude: an approach using Rasch measurement theory

Affective engineering uses mathematical models to convert the information obtained from persons' attitude to physical elements into an ergonomic design. However, applications in the domain have not in many cases met measurement assumptions. This paper proposes a novel approach based on Rasch measurement theory to overcome the problem. The research demonstrates that if data fit the model, further variables can be added to a scale. An empirical study was designed to determine the range of compliance where consumers could obtain an impression of a moisturizer cream when touching some product containers. Persons, variables and stimulus objects were parameterised independently on a linear continuum. The results showed that a calibrated scale preserves comparability although incorporating further variables

Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Reliability and responsiveness of measures of pain in people with osteoarthritis of the knee: a psychometric evaluation

PURPOSE: To examine the fit between data from the Short Form McGill Pain Questionnaire (SF-MPQ-2) and the Rasch model, and to explore the reliability and internal responsiveness of measures of pain in people with knee osteoarthritis. METHODS: Participants with knee osteoarthritis completed the SF-MPQ-2, Intermittent and Constant Osteoarthritis Pain questionnaire (ICOAP) and painDETECT. Participants were sent the same questionnaires 3 and 6 months later. RESULTS: Fit to the Rasch model was not achieved for the SF-MPQ-2 Total scale. The Continuous subscale yielded adequate fit statistics after splitting item 10 on uniform DIF for gender, and removing item 9. The Intermittent subscale fit the Rasch model after rescoring items. The Neuropathic subscale had relatively good fit to the model. Test-retest reliability was satisfactory for most scales using both original and Rasch scoring ranging from fair to substantial. Effect sizes ranged from 0.13 to 1.79 indicating good internal responsiveness for most scales. CONCLUSIONS: These findings support the use of ICOAP subscales as reliable and responsive measure of pain in people with knee osteoarthritis. The MPQ-SF-2 subscales found to be acceptable alternatives. Implications for Rehabilitation The McGill Pain Questionnaire short version 2 is not a unidimensional scale in people with knee osteoarthritis, whereas three of the subscales are unidimensional. The McGill Pain Questionnaire short version 2 Affective subscale does not have good measurement properties for people with knee osteoarthritis. The McGill Pain Questionnaire short version 2 and the Intermittent and Constant Osteoarthritis Pain scales can be used to assess change over time. The painDETECT performs better as a screening measure than as an outcome measure

Critical Values for Yen’s Q3: Identification of Local Dependence in the Rasch model using Residual Correlations

The assumption of local independence is central to all IRT models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3 there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence, and for this reason a variety of arbitrary rules of thumb are used. In this study, we used an empirical data example and Monte Carlo simulation to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about local dependence during scale development and validation. We propose that a parametric bootstrapping procedure should be implemented in each separate situation in order to obtain the critical value of local dependence applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories. Furthermore, our results show that local dependence should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic

Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey

Background: The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) was developed to meet demand for instruments to measure mental well-being. It comprises 14 positively phrased Likert-style items and fulfils classic criteria for scale development. We report here the internal construct validity of WEMWBS from the perspective of the Rasch measurement model. Methods: The model was applied to data collected from 779 respondents in Wave 12 (Autumn 2006) of the Scottish Health Education Population Survey. Respondents were aged 16–74 (average 41.9) yrs. Results: Initial fit to model expectations was poor. The items 'I've been feeling good about myself', 'I've been interested in new things' and 'I've been feeling cheerful' all showed significant misfit to model expectations, and were deleted. This led to a marginal improvement in fit to the model. After further analysis, more items were deleted and a strict unidimensional seven item scale (the Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS)) was resolved. Many items deleted because of misfit with model expectations showed considerable bias for gender. Two retained items also demonstrated bias for gender but, at the scale level, cancelled out. One further retained item 'I've been feeling optimistic about the future' showed bias for age. The correlation between the 14 item and 7 item versions was 0.954. Given fit to the Rasch model, and strict unidimensionality, SWEMWBS provides an interval scale estimate of mental well-being. Conclusion: A short 7 item version of WEMWBS was found to satisfy the strict unidimensionality expectations of the Rasch model, and be largely free of bias. This scale, SWEMWBS, provides a raw score-interval scale transformation for use in parametric procedures. In terms of face validity, SWEMWBS presents a more restricted view of mental well-being than the 14 item WEMWBS, with most items representing aspects of psychological and eudemonic well-being, and few covering hedonic well-being or affect. However, robust measurement properties combined with brevity make SWEMWBS preferable to WEMWBS at present for monitoring mental well-being in populations. Where face validity is an issue there remain arguments for continuing to collect data on the full 14 item WEMWBS

Author finding post print - The Multiple Sclerosis-Fatigue Self- Efficacy (MS-FSE) scale: initial validation.

To examine the validity and sensitivity to change of the Multiple Sclerosis-Fatigue Self-Efficacy scale

An evaluation of the structural validity of the Shoulder Pain and Disability Index (SPADI) using the Rasch model

Purpose: The Shoulder Pain and Disability Index (SPADI) has been extensively evaluated for its psychometric properties using classic test theory (CTT). The purpose of this study was to evaluate its structural validity using Rasch model analysis. Methods: Responses to the SPADI from 1030 patients referred for physiotherapy with shoulder pain and enrolled in a prospective cohort study were available for Rasch model analysis. Overall fit, individual person and item fit, response format, dependence, unidimensionality, targeting, reliability and differential item functioning (DIF) were examined. Results: The SPADI pain subscale initially demonstrated a misfit due to DIF by age and gender. After iterative analysis it showed good fit to the Rasch model with acceptable targeting and unidimensionality (overall fit (chi-square statistic 57.2, p=0.1); mean item fit residual 0.19 (1.5) and mean person fit residual 0.44 (1.1); person separation index (PSI) of 0.83). The disability subscale however shows significant misfit due to uniform DIF even after iterative analyses were used to explore different solutions to the sources of misfit (overall fit (chi-square statistic 57.2, p=0.1); mean item fit residual -0.54 (1.26) and mean person fit residual -0.38 (1.0); PSI 0.84). Conclusions: Rasch Model analysis of the SPADI has identified some strengths and limitations not previously observed using CTT methods. The SPADI should be treated as two separate subscales. The SPADI is a widely used outcome measure in clinical practice and research, however the scores derived from it must be interpreted with caution. The pain subscale fits the Rasch model expectations well. The disability subscale does not fit the Rasch model and its current format does not meet the criteria for true interval-level measurement required for use as a primary endpoint in clinical trials. Clinicians should therefore exercise caution when interpreting score changes on the disability subscale and attempt to compare their scores to age and sex stratified data

Conceptualising computerized adaptive testing for measurement of latent variables associated with physical objects

The notion of that more or less of a physical feature affects in different degrees the users' impression with regard to an underlying attribute of a product has frequently been applied in affective engineering. However, those attributes exist only as a premise that cannot directly be measured and, therefore, inferences based on their assessment are error-prone. To establish and improve measurement of latent attributes it is presented in this paper the concept of a stochastic framework using the Rasch model for a wide range of independent variables referred to as an item bank. Based on an item bank, computerized adaptive testing (CAT) can be developed. A CAT system can converge into a sequence of items bracketing to convey information at a user's particular endorsement level. It is through item banking and CAT that the financial benefits of using the Rasch model in affective engineering can be realised

Patient Uncertainty Questionnaire-Rheumatology (PUQ-R): development and validation of a new patient-reported outcome instrument for systemic lupus erythematosus (SLE) and rheumatoid arthritis (RA) in a mixed methods study

Background An in-depth qualitative exploration of uncertainty in systemic lupus erythematosus (SLE) and rheumatoid arthritis (RA) led to the development of a five-domain conceptual framework of patient uncertainty in these two conditions. The purpose of this study was to develop and evaluate a new patient-reported outcome (PRO) instrument for patient uncertainty in SLE and RA on the basis of this empirically developed conceptual framework. Methods Cognitive debriefing interviews were conducted to pre-test the initial items generated on the basis of the preliminary qualitative exploration of patient uncertainty in SLE and RA. Two separate field tests were conducted in five hospital sites to evaluate the measurement properties of the new instrument; the first to identify and form scales, and the second to assess measurement properties of the final version in an independent sample. Psychometric evaluation was conducted in line with the Rasch Measurement Theory (RMT), examining the extent to which sample to scale targeting was satisfactory, measurement scales were constructed effectively and the sample was measured successfully. Traditional psychometric techniques were also used to provide complementary analyses best understood by clinicians. Results Pre-testing supported the relevance, acceptability and comprehensibility of the initial items. Findings indicated that the Patient Uncertainty Questionnaire for Rheumatology PUQ-R instrument fulfilled the expectations of RMT to a large extent (including person separation index 0.73 – 0.91). The PUQ-R comprises 49 items across five scales; symptoms and flares (14 items), medication (11 items), trust in doctor (8 items), self-management (6 items) and impact (10 items) which further displayed excellent measurement properties as assessed against the traditional psychometric criteria (including Cronbach’s alpha 0.82 – 0.93). Conclusion The PUQ-R has been developed and evaluated specifically for patients with SLE and RA. By quantifying uncertainty, the PUQ-R has the potential to support evidence-based management programmes and research

Further investigation of confirmed urinary tract infection (UTI) in children under five years: a systematic review.

Background: Further investigation of confirmed UTI in children aims to prevent renal scarring and future complications. Methods: We conducted a systematic review to determine the most effective approach to the further investigation of confirmed urinary tract infection (UTI) in children under five years of age. Results: 73 studies were included. Many studies had methodological limitations or were poorly reported. Effectiveness of further investigations: One study found that routine imaging did not lead to a reduction in recurrent UTIs or renal scarring. Diagnostic accuracy: The studies do not support the use of less invasive tests such as ultrasound as an alternative to renal scintigraphy, either to rule out infection of the upper urinary tract (LR- = 0.57, 95%CI: 0.47, 0.68) and thus to exclude patients from further investigation or to detect renal scarring (LR+ = 3.5, 95% CI: 2.5, 4.8). None of the tests investigated can accurately predict the development of renal scarring. The available evidence supports the consideration of contrast-enhanced ultrasound techniques for detecting vesico-ureteric reflux (VUR), as an alternative to micturating cystourethrography (MCUG) (LR+ = 14.1, 95% CI: 9.5, 20.8; LR- = 0.20, 95%CI: 0.13, 0.29); these techniques have the advantage of not requiring exposure to ionising radiation. Conclusion: There is no evidence to support the clinical effectiveness of routine investigation of children with confirmed UTI. Primary research on the effectiveness, in terms of improved patient outcome, of testing at all stages in the investigation of confirmed urinary tract infection is urgently required