Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Internal construct validity of the Shirom-Melamed Burnout Questionnaire (SMBQ)

<p>Abstract</p> <p>Background</p> <p>Burnout is a mental condition defined as a result of continuous and long-term stress exposure, particularly related to psychosocial factors at work. This paper seeks to examine the psychometric properties of the Shirom-Melamed Burnout Questionnaire (SMBQ) for validation of use in a clinical setting.</p> <p>Methods</p> <p>Data from both a clinical (319) and general population (319) samples of health care and social insurance workers were included in the study. Data were analysed using both classical and modern test theory approaches, including Confirmatory Factor Analysis (CFA) and Rasch analysis.</p> <p>Results</p> <p>Of the 638 people recruited into the study 416 (65%) persons were working full or part time. Data from the SMBQ failed a CFA, and initially failed to satisfy Rasch model expectations. After the removal of 4 of the original items measuring tension, and accommodating local dependency in the data, model expectations were met. As such, the total score from the revised scale is a sufficient statistic for ascertaining burnout and an interval scale transformation is available. The scale as a whole was perfectly targeted to the joint sample. A cut point of 4.4 for severe burnout was chosen at the intersection of the distributions of the clinical and general population.</p> <p>Conclusion</p> <p>A revised 18 item version of the SMBQ satisfies modern measurement standards. Using its cut point it offers the opportunity to identify potential clinical cases of burnout.</p

Resource Modelling: The Missing Piece of the HTA Jigsaw?

Within health technology assessment (HTA), cost-effectiveness analysis and budget impact analyses have been broadly accepted as important components of decision making. However, whilst they address efficiency and affordability, the issue of implementation and feasibility has been largely ignored. HTA commonly takes place within a deliberative framework that captures issues of implementation and feasibility in a qualitative manner. We argue that only through a formal quantitative assessment of resource constraints can these issues be fully addressed. This paper argues the need for resource modelling to be considered explicitly in HTA. First, economic evaluation and budget impact models are described along with their limitations in evaluating feasibility. Next, resource modelling is defined and its usefulness is described along with examples of resource modelling from the literature. Then, the important issues that need to be considered when undertaking resource modelling are described before setting out recommendations for the use of resource modelling in HTA

Measure of Activity Performance in the Hand (MAP-Hand) questionnaire

Background: Developed in the Norway, the Measure of Activity Performance of the Hand (MAP-Hand) assesses 18 activities performed using the hands. It was developed for people with rheumatoid arthritis (RA) using patient generated items, which are scored on a 0-3 scale and summarised into a total score range (0 to 54). This study reports the development and psychometric testing of the British English MAP-Hand in a UK population of people with RA. Methods: Recruitment took place in the National Health Service (NHS) through 17 Rheumatology outpatient clinics. Phase 1 (cross-cultural adaptation) involved: forward translation to British English; synthesis; expert panel review and cognitive debriefing interviews with people with RA. Phase 2 (psychometric testing) involved postal completion of the MAP-Hand, Health Assessment Questionnaire (HAQ), Upper Limb HAQ (ULHAQ), Short-Form 36 (SF-36v2) and Disabilities of the Arm Shoulder Hand (DASH) to measure internal consistency (Cronbach’s alpha); concurrent validity (Spearman’s correlations) and Minimal Detectable Difference (MDC95). The MAP-Hand was repeated three-weeks later to assess test-retest reliability (linear weighted kappa and Intra-Class Correlations (ICC (2,1)). Unidimensionality (internal construct validity) was assessed using (i) Confirmatory Factor Analysis (CFA) (ii) Mokken scaling and (iii) Rasch model. The RUMM2030 software was used, applying the Rasch partial credit model. Results: In Phase 1, 31 participants considered all items relevant. In Phase 2, 340 people completed Test-1 and 273 (80%) completed Test-2 questionnaires. Internal consistency was excellent (α=0.96). Test-retest reliability was good (ICC (2,1) = 0.96 (95% CI 0.94, 0.97)). The MAP-Hand correlated strongly with HAQ20 (rs=.88), ULHAQ (rs=.91), SF-36v2 Physical Functioning (PF) Score (rs=-.80) and DASH (rs=.93), indicating strong concurrent validity. CFA failed to support unidimensionality (Chi-Square 236.0 (df 120; p &lt;0.001)). However, Mokken scaling suggested a probabilistic ordering. There was differential item functioning (DIF) for gender. Four testlets were formed, resulting in much improved fit and unidimensionality. Following this, testlets were further merged in pairs where opposite bias existed. This resulted in perfect fit to the model. Conclusions: The British English version of the MAP-Hand has good validity and reliability in people with RA and can be used in both research and clinical practice. Keywords: PROMS; Patient Reported Outcome Measures; hand activity performance; hand function; hand pain; psychometric testing; Rasch analysis; validity; reliabilit

Income Attainment among Victims of Violence: Results From a Preliminary Study

Violent victimisation may have many short-term psychological and physical outcomes. Occasionally, the negative aftermath of violence persists over time or induces other and more far-reaching consequences. Income attainment after victimisation is one of these outcomes. To date, previous studies have focussed on the income effects of violent victimisation during childhood and adolescence. Violence exposure during the early stages of the life course may frustrate processes of educational and occupational attainment and consequentially result in lower income levels. However, in addition or alternatively, many other and age-independent pathways between violent victimisation and income may be suggested. Prior studies appear to have paid little attention to this issue. Therefore, the purpose of the current study was to explore whether violent victimisation is associated with income levels several years after victimisation, irrespective of the age at which victimisation occurs. Victims of violence were recruited through the Dutch Victim Compensation Fund. To preliminary estimate the effect of violent victimisation on income, a comparable control group of non-victims was composed. The study sample contained 206 victims and 173 non-victims. Both bivariate correlational and multivariate statistical techniques suggested that violent victimisation is a significant predictor of income. Implications of the presented results were discussed with regard to future research and policy practice

Antifungal potential of extracts, fractions and compounds from Uvaria comperei (Annonaceae) and Oxyanthus unilocularis (Rubiaceae)

peer reviewe

"If only I had taken the other road...": Regret, risk and reinforced learning in informed route-choice

This paper presents a study of the effect of regret on route choice behavior when both descriptional information and experiential feedback on choice outcomes are provided. The relevance of Regret Theory in travel behavior has been well demonstrated in non-repeated choice environments involving decisions on the basis of descriptional information. The relation between regret and reinforced learning through experiential feedbacks is less understood. Using data obtained from a simple route-choice experiment involving different levels of travel time variability, discrete-choice models accounting for regret aversion effects are estimated. The results suggest that regret aversion is more evident when descriptional information is provided ex-ante compared to a pure learning from experience condition. Yet, the source of regret is related more strongly to experiential feedbacks rather than to the descriptional information itself. Payoff variability is negatively associated with regret. Regret aversion is more observable in choice situations that reveal risk-seeking, and less in the case of risk-aversion. These results are important for predicting the possible behavioral impacts of emerging information and communication technologies and intelligent transportation systems on travelers' behavior. © 2012 Springer Science+Business Media, LLC

Binge eating, purging and non-purging compensatory behaviours decrease from adolescence to adulthood: A population-based, longitudinal study

Background Subclinical forms of eating disorders (ED) are highly prevalent, but relatively little is known about age trends, gender differences and distinctions among symptoms. This study investigates age trends and gender difference in binge eating, purging and non-purging compensatory behaviours (CB) and the relationship of such behaviours to psychosocial problems. Methods Data from the national representative longitudinal study "Young in Norway" (ages 14-34 years) were analysed using χ 2 tests, logistic random intercept models and analyses of covariance. Results For both genders, a decrease was found in the prevalence of CB from age 14-16 years to 23 years and over. For binging, however, a significant decrease was found only for females, whose binge eating also declined more markedly over time than did males'. A significant gender difference was detected for purging, with females at higher risk. Purging was related to particularly serious symptoms of psychosocial problems: Those who purged had significantly higher levels of appearance dissatisfaction, anxiety and depressive symptoms, alcohol consumption, self-concept instability and loneliness than those with symptoms of other forms of disordered eating. Conclusions Individuals affected by purging need to be targeted as a high-risk group. The distinction in severity among the subclinical ED may indicate the need for the reformulation of the eating disorder not otherwise specified category in the Diagnostic and Statistical Manual of Mental Disorders-V

Subliminal Semantic Priming in Speech

Numerous studies have reported subliminal repetition and semantic priming in the visual modality. We transferred this paradigm to the auditory modality. Prime awareness was manipulated by a reduction of sound intensity level. Uncategorized prime words (according to a post-test) were followed by semantically related, unrelated, or repeated target words (presented without intensity reduction) and participants performed a lexical decision task (LDT). Participants with slower reaction times in the LDT showed semantic priming (faster reaction times for semantically related compared to unrelated targets) and negative repetition priming (slower reaction times for repeated compared to semantically related targets). This is the first report of semantic priming in the auditory modality without conscious categorization of the prime