Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

People of the British Isles: preliminary analysis of genotypes and surnames in a UK control population

There is a great deal of interest in fine scale population structure in the UK, both as a signature of historical immigration events and because of the effect population structure may have on disease association studies. Although population structure appears to have a minor impact on the current generation of genome-wide association studies, it is likely to play a significant part in the next generation of studies designed to search for rare variants. A powerful way of detecting such structure is to control and document carefully the provenance of the samples involved. Here we describe the collection of a cohort of rural UK samples (The People of the British Isles), aimed at providing a well-characterised UK control population that can be used as a resource by the research community as well as providing fine scale genetic information on the British population. So far, some 4,000 samples have been collected, the majority of which fit the criteria of coming from a rural area and having all four grandparents from approximately the same area. Analysis of the first 3,865 samples that have been geocoded indicates that 75% have a mean distance between grandparental places of birth of 37.3km, and that about 70% of grandparental places of birth can be classed as rural. Preliminary genotyping of 1,057 samples demonstrates the value of these samples for investigating fine scale population structure within the UK, and shows how this can be enhanced by the use of surnames

Energetic Selection of Topology in Ferredoxins

Models of early protein evolution posit the existence of short peptides that bound metals and ions and served as transporters, membranes or catalysts. The Cys-X-X-Cys-X-X-Cys heptapeptide located within bacterial ferredoxins, enclosing an Fe4S4 metal center, is an attractive candidate for such an early peptide. Ferredoxins are ancient proteins and the simple α+β fold is found alone or as a domain in larger proteins throughout all three kingdoms of life. Previous analyses of the heptapeptide conformation in experimentally determined ferredoxin structures revealed a pervasive right-handed topology, despite the fact that the Fe4S4 cluster is achiral. Conformational enumeration of a model CGGCGGC heptapeptide bound to a cubane iron-sulfur cluster indicates both left-handed and right-handed folds could exist and have comparable stabilities. However, only the natural ferredoxin topology provides a significant network of backbone-to-cluster hydrogen bonds that would stabilize the metal-peptide complex. The optimal peptide configuration (alternating αL,αR) is that of an α-sheet, providing an additional mechanism where oligomerization could stabilize the peptide and facilitate iron-sulfur cluster binding

Predicting residents' performance: A prospective study

BACKGROUND: Objective criteria for predicting residents' performance do not exist. The purpose of this study was to test the hypothesis that global assessment by an intern selection committee (ISC) would correlate with the future performance of residents. METHODS: A prospective study of 277 residents between 1992 and 1999. Global assessment at the time of interview was compared to subsequent clinical (assessed by chief residents) and cognitive performance (assessed by the American Board of Pediatrics in-service training examination). RESULTS: ISC ratings correlated significantly with clinical performance at 24 and 36 months of training (r = 0.58, P < .001; and r = 0.60, P < .001 respectively). ISC ratings also correlated significantly with in-service exam scores in the 1(st), 2(nd), and 3(rd) years of training (r = 0.35, P = .0016; r = 0.39, P = 0.0003; r = 0.50, P = 0.005 respectively). CONCLUSIONS: Global assessment by an ISC predicted residents' clinical and cognitive performances

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

Background: Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods: An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results: Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion: Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination.Edward J Palmer, Peter G Devit

Association between retinal vein occlusion, axial length and vitreous chamber depth measured by optical low coherence reflectometry.

BACKGROUND: Results of ocular biometric measurements in retinal vein occlusion (RVO) eyes are still inconclusive and controversial. The aim of this study was to evaluate the association between ocular axial length (AL), vitreous chamber depth (VCD) and both central (CRVO) and branch retinal vein occlusions (BRVO) using optical low coherence reflectometry (OLCR). METHODS: Both eyes of 37 patients with unilateral CRVO (mean age: 66 +/- 14 years, male:female - 21:16) and 46 patients with unilateral BRVO (mean age: 63 +/- 12 years, male:female - 24:22) were enrolled in this study. The control group consisted of randomly selected single eyes of 67 age and gender matched volunteers without the presence or history of RVO (mean age: 64 +/- 14 years, male:female - 34:33). Optical biometry was performed by OLCR biometer (LenStar LS 900). Average keratometry readings, central corneal thickness (CCT), anterior chamber depth (ACD), lens thickness (LT), AL and VCD of eyes with RVO were compared with those of fellow eyes using paired t-tests and with those of control eyes using independent t-tests. RESULTS: Mean CCT, ACD and LT, average keratometry readings of affected RVO eyes, unaffected fellow eyes and control eyes was not statistically different in either groups. In eyes with CRVO mean AL and VCD of affected eyes were significantly shorter than those of control eyes (p < 0.001, p < 0.05), mean difference in AL and VCD between the affected and control eyes was 0.56 +/- 0.15 mm and 0.45 +/- 0.19 mm, respectively. In eyes with BRVO, mean AL of the affected eyes was significantly shorter with a mean difference of 0.57 +/- 0.15 mm (p < 0.001) and the VCD was significantly shorter with a mean difference of 0.61 +/- 0.15 mm (p < 0.001) comparing with the control eyes. CONCLUSION: Shorter AL and VCD might be a potential anatomical predisposing factor for development either of CRVO or BRVO

Pathway analysis comparison using Crohn's disease genome wide association studies

<p>Abstract</p> <p>Background</p> <p>The use of biological annotation such as genes and pathways in the analysis of gene expression data has aided the identification of genes for follow-up studies and suggested functional information to uncharacterized genes. Several studies have applied similar methods to genome wide association studies and identified a number of disease related pathways. However, many questions remain on how to best approach this problem, such as whether there is a need to obtain a score to summarize association evidence at the gene level, and whether a pathway, dominated by just a few highly significant genes, is of interest.</p> <p>Methods</p> <p>We evaluated the performance of two pathway-based methods (Random Set, and Binomial approximation to the hypergeometric test) based on their applications to three data sets of Crohn's disease. We consider both the disease status as a phenotype as well as the residuals after conditioning on IL23R, a known Crohn's related gene, as a phenotype.</p> <p>Results</p> <p>Our results show that Random Set method has the most power to identify disease related pathways. We confirm previously reported disease related pathways and provide evidence for IL-2 Receptor Beta Chain in T cell Activation and IL-9 signaling as Crohn's disease associated pathways.</p> <p>Conclusions</p> <p>Our results highlight the need to apply powerful gene score methods prior to pathway enrichment tests, and that controlling for genes that attain genome wide significance enable further biological insight.</p