Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey

Background: The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) was developed to meet demand for instruments to measure mental well-being. It comprises 14 positively phrased Likert-style items and fulfils classic criteria for scale development. We report here the internal construct validity of WEMWBS from the perspective of the Rasch measurement model. Methods: The model was applied to data collected from 779 respondents in Wave 12 (Autumn 2006) of the Scottish Health Education Population Survey. Respondents were aged 16–74 (average 41.9) yrs. Results: Initial fit to model expectations was poor. The items 'I've been feeling good about myself', 'I've been interested in new things' and 'I've been feeling cheerful' all showed significant misfit to model expectations, and were deleted. This led to a marginal improvement in fit to the model. After further analysis, more items were deleted and a strict unidimensional seven item scale (the Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS)) was resolved. Many items deleted because of misfit with model expectations showed considerable bias for gender. Two retained items also demonstrated bias for gender but, at the scale level, cancelled out. One further retained item 'I've been feeling optimistic about the future' showed bias for age. The correlation between the 14 item and 7 item versions was 0.954. Given fit to the Rasch model, and strict unidimensionality, SWEMWBS provides an interval scale estimate of mental well-being. Conclusion: A short 7 item version of WEMWBS was found to satisfy the strict unidimensionality expectations of the Rasch model, and be largely free of bias. This scale, SWEMWBS, provides a raw score-interval scale transformation for use in parametric procedures. In terms of face validity, SWEMWBS presents a more restricted view of mental well-being than the 14 item WEMWBS, with most items representing aspects of psychological and eudemonic well-being, and few covering hedonic well-being or affect. However, robust measurement properties combined with brevity make SWEMWBS preferable to WEMWBS at present for monitoring mental well-being in populations. Where face validity is an issue there remain arguments for continuing to collect data on the full 14 item WEMWBS

Internal construct validity of the Shirom-Melamed Burnout Questionnaire (SMBQ)

<p>Abstract</p> <p>Background</p> <p>Burnout is a mental condition defined as a result of continuous and long-term stress exposure, particularly related to psychosocial factors at work. This paper seeks to examine the psychometric properties of the Shirom-Melamed Burnout Questionnaire (SMBQ) for validation of use in a clinical setting.</p> <p>Methods</p> <p>Data from both a clinical (319) and general population (319) samples of health care and social insurance workers were included in the study. Data were analysed using both classical and modern test theory approaches, including Confirmatory Factor Analysis (CFA) and Rasch analysis.</p> <p>Results</p> <p>Of the 638 people recruited into the study 416 (65%) persons were working full or part time. Data from the SMBQ failed a CFA, and initially failed to satisfy Rasch model expectations. After the removal of 4 of the original items measuring tension, and accommodating local dependency in the data, model expectations were met. As such, the total score from the revised scale is a sufficient statistic for ascertaining burnout and an interval scale transformation is available. The scale as a whole was perfectly targeted to the joint sample. A cut point of 4.4 for severe burnout was chosen at the intersection of the distributions of the clinical and general population.</p> <p>Conclusion</p> <p>A revised 18 item version of the SMBQ satisfies modern measurement standards. Using its cut point it offers the opportunity to identify potential clinical cases of burnout.</p

Do not attempt resuscitation decisions in a cancer centre: addressing difficult ethical and communication issues

Talking to patients about ‘Do Not Attempt Resuscitation’ decisions is difficult for many doctors. Communication about ‘Do Not Attempt Resuscitation’ decisions should occur as part of a wider discussion of treatment goals at an earlier stage in the patient's illness. A doctor should not initiate any treatment, including cardio-pulmonary resuscitation if he/she does not believe it will benefit the patient. An ethical framework is offered which may be of practical help in clarifying decision-making

Smart Phone, Smart Science: How the Use of Smartphones Can Revolutionize Research in Cognitive Science

Investigating human cognitive faculties such as language, attention, and memory most often relies on testing small and homogeneous groups of volunteers coming to research facilities where they are asked to participate in behavioral experiments. We show that this limitation and sampling bias can be overcome by using smartphone technology to collect data in cognitive science experiments from thousands of subjects from all over the world. This mass coordinated use of smartphones creates a novel and powerful scientific “instrument” that yields the data necessary to test universal theories of cognition. This increase in power represents a potential revolution in cognitive science

Change in diet, physical activity, and body weight among young-adults during the transition from high school to college

<p>Abstract</p> <p>Background</p> <p>The freshmen year of college is likely a critical period for risk of weight gain among young-adults.</p> <p>Methods</p> <p>A longitudinal observational study was conducted to examine changes in weight, dietary intake, and other health-related behaviors among first-year college students (n = 186) attending a public University in the western United States. Weight was measured at the beginning and end of fall semester (August – December 2005). Participants completed surveys about dietary intake, physical activity and other health-related behaviors during the last six months of high school (January – June 2005) in August 2005 and during their first semester of college (August – December 2005) in December 2005.</p> <p>Results</p> <p>159 students (n = 102 women, 57 men) completed both assessments. The average BMI at the baseline assessment was 23.0 (standard deviation (SD) 3.8). Although the average amount of weight gained during the 15-week study was modest (1.5 kg), 23% of participants gained ≥ 5% of their baseline body weight. Average weight gain among those who gained ≥ 5% of baseline body weight was 4.5 kg. Those who gained ≥ 5% of body weight reported less physical activity during college than high school, were more likely to eat breakfast, and slept more than were those who did not gain ≥ 5% of body weight.</p> <p>Conclusion</p> <p>Almost one quarter of students gained a significant amount of weight during their first semester of college. This research provides further support for the implementation of education or other strategies aimed at helping young-adults entering college to achieve or maintain a healthy body weight.</p

Oral cancer treatment costs in Greece and the effect of advanced disease

BACKGROUND: The main purpose of the study was to quantify the direct costs of oral cancer treatment to the healthcare system of Greece. Another aim was to identify factors that affect costs and potential cost reduction items. More specifically, we examined the relationship between stage of disease, modality of treatment and total direct costs. METHODS: The medical records and clinic files of the Oral and Maxillofacial Clinic of the Athens General Hospital "Genimatas" were abstracted to investigate clinical treatment characteristics, including length of hospitalization, modes of treatment, stage of disease etc. Records of 95 patients with oral squamous cell carcinoma (OSSC), with at least six months of follow-up, were examined. The clinical data was then used to calculate actual direct costs, based on 2001 market values. RESULTS: The mean total direct costs for OSSC treatment estimated at euro 8,450 or approximately US$ 7,450. Costs depended on the stage of the disease, with significant increases in stages III and IV, as compared with stages I and II (p < 0.05). Multi-modality treatment applied mainly to patients in stages III and IV was the factor that affected the cost. Disease stage was also associated with the total duration of hospitalization (p < 0.05). CONCLUSIONS: The clinical management of advanced oral cancer is strongly associated with higher costs. Although the ideal would be to prevent cancer, the combination of high-risk screening, early diagnosis and early treatment seems the most efficient way to reduce costs, and most importantly, prolong life