The standard error of measurement is a more appropriate measure of quality for postgraduate medical assessments than is reliability: an analysis of MRCP(UK) examinations

Background: Cronbach's alpha is widely used as the preferred index of reliability for medical postgraduate examinations. A value of 0.8-0.9 is seen by providers and regulators alike as an adequate demonstration of acceptable reliability for any assessment. Of the other statistical parameters, Standard Error of Measurement (SEM) is mainly seen as useful only in determining the accuracy of a pass mark. However the alpha coefficient depends both on SEM and on the ability range (standard deviation, SD) of candidates taking an exam. This study investigated the extent to which the necessarily narrower ability range in candidates taking the second of the three part MRCP(UK) diploma examinations, biases assessment of reliability and SEM.Methods: a) The interrelationships of standard deviation (SD), SEM and reliability were investigated in a Monte Carlo simulation of 10,000 candidates taking a postgraduate examination. b) Reliability and SEM were studied in the MRCP(UK) Part 1 and Part 2 Written Examinations from 2002 to 2008. c) Reliability and SEM were studied in eight Specialty Certificate Examinations introduced in 2008-9.Results: The Monte Carlo simulation showed, as expected, that restricting the range of an assessment only to those who had already passed it, dramatically reduced the reliability but did not affect the SEM of a simulated assessment. The analysis of the MRCP(UK) Part 1 and Part 2 written examinations showed that the MRCP(UK) Part 2 written examination had a lower reliability than the Part 1 examination, but, despite that lower reliability, the Part 2 examination also had a smaller SEM (indicating a more accurate assessment). The Specialty Certificate Examinations had small Ns, and as a result, wide variability in their reliabilities, but SEMs were comparable with MRCP(UK) Part 2.Conclusions: An emphasis upon assessing the quality of assessments primarily in terms of reliability alone can produce a paradoxical and distorted picture, particularly in the situation where a narrower range of candidate ability is an inevitable consequence of being able to take a second part examination only after passing the first part examination. Reliability also shows problems when numbers of candidates in examinations are low and sampling error affects the range of candidate ability. SEM is not subject to such problems; it is therefore a better measure of the quality of an assessment and is recommended for routine use

The educational impact of assessment: a comparison of DOPS and MCQs

Aim: To evaluate the impact of two different assessment formats on the approaches to learning of final year veterinary students. The relationship between approach to learning and examination performance was also investigated. Method: An 18-item version of the Study Process Questionnaire (SPQ) was sent to 87 final year students. Each student responded to the questionnaire with regards to DOPS (Direct Observation of Procedural Skills) and a Multiple Choice Examination (MCQ). Semi-structured interviews were conducted with 16 of the respondents to gain a deeper insight into the students’ perception of assessment. Results: Students’ adopted a deeper approach to learning for DOPS and a more surface approach with MCQs. There was a positive correlation between an achieving approach to learning and examination performance. Analysis of the qualitative data revealed that deep, surface and achieving approaches were reported by the students and seven major influences on their approaches to learning were identified: motivation, purpose, consequence, acceptability, feedback, time pressure and the individual difference of the students. Conclusions: The format of DOPS has a positive influence on approaches to learning. There is a conflict for students between preparing for final examinations and preparing for clinical practice

Modeling recursive RNA interference.

An important application of the RNA interference (RNAi) pathway is its use as a small RNA-based regulatory system commonly exploited to suppress expression of target genes to test their function in vivo. In several published experiments, RNAi has been used to inactivate components of the RNAi pathway itself, a procedure termed recursive RNAi in this report. The theoretical basis of recursive RNAi is unclear since the procedure could potentially be self-defeating, and in practice the effectiveness of recursive RNAi in published experiments is highly variable. A mathematical model for recursive RNAi was developed and used to investigate the range of conditions under which the procedure should be effective. The model predicts that the effectiveness of recursive RNAi is strongly dependent on the efficacy of RNAi at knocking down target gene expression. This efficacy is known to vary highly between different cell types, and comparison of the model predictions to published experimental data suggests that variation in RNAi efficacy may be the main cause of discrepancies between published recursive RNAi experiments in different organisms. The model suggests potential ways to optimize the effectiveness of recursive RNAi both for screening of RNAi components as well as for improved temporal control of gene expression in switch off-switch on experiments

Characterisation of the Immunophenotype of Dogs with Primary Immune-Mediated Haemolytic Anaemia

Immune-mediated haemolytic anaemia (IMHA) is reported to be the most common autoimmune disease of dogs, resulting in significant morbidity and mortality in affected animals. Haemolysis is caused by the action of autoantibodies, but the immunological changes that result in their production have not been elucidated.To investigate the frequency of regulatory T cells (Tregs) and other lymphocyte subsets and to measure serum concentrations of cytokines and peripheral blood mononuclear cell expression of cytokine genes in dogs with IMHA, healthy dogs and dogs with inflammatory diseases.19 dogs with primary IMHA, 22 dogs with inflammatory diseases and 32 healthy control dogs.Residual EDTA-anti-coagulated blood samples were stained with fluorophore-conjugated monoclonal antibodies and analysed by flow cytometry to identify Tregs and other lymphocyte subsets. Total RNA was also extracted from peripheral blood mononuclear cells to investigate cytokine gene expression, and concentrations of serum cytokines (interleukins 2, 6 10, CXCL-8 and tumour necrosis factor α) were measured using enhanced chemiluminescent assays. Principal component analysis was used to investigate latent variables that might explain variability in the entire dataset.There was no difference in the frequency or absolute numbers of Tregs among groups, nor in the proportions of other lymphocyte subsets. The concentrations of pro-inflammatory cytokines were greater in dogs with IMHA compared to healthy controls, but the concentration of IL-10 and the expression of cytokine genes did not differ between groups. Principal component analysis identified four components that explained the majority of the variability in the dataset, which seemed to correspond to different aspects of the immune response.The immunophenotype of dogs with IMHA differed from that of dogs with inflammatory diseases and from healthy control dogs; some of these changes could suggest abnormalities in peripheral tolerance that permit development of autoimmune disease. The frequency of Tregs did not differ between groups, suggesting that deficiency in the number of these cells is not responsible for development of IMHA

Determining which automatic digital blood pressure device performs adequately: a systematic review

The aim of this study is to systematically examine the proportion of accurate readings attained by automatic digital blood pressure (BP) devices in published validation studies. We included studies of automatic digital BP devices using recognized protocols. We summarized the data as mean and s.d. of differences between measured and observed BP, and proportion of measurements within 5 mm Hg. We included 79 articles (10 783 participants) reporting 113 studies from 22 different countries. Overall, 25/31 (81%), 37/41 (90%) and 34/35 (97%) devices passed the relevant protocols [BHS, AAMI and ESH international protocol (ESH-IP), respectively]. For devices that passed the BHS protocol, the proportion of measured values within 5 mm Hg of the observed value ranged from 60 to 86% (AAMI protocol 47–94% and ESH-IP 54–89%). The results for the same device varied significantly when a different protocol was used (Omron HEM-907 80% of readings were within 5 mm Hg using the AAMI protocol compared with 62% with the ESH-IP). Even devices with a mean difference of zero show high variation: a device with 74% of BP measurements within 5 mm Hg would require six further BP measurements to reduce variation to 95% of readings within 5 mm Hg. Current protocols for validating BP monitors give no guarantee of accuracy in clinical practice. Devices may pass even the most rigorous protocol with as few as 60% of readings within 5 mm Hg of the observed value. Multiple readings are essential to provide clinicians and patients with accurate information on which to base diagnostic and treatment decisions

Effect of human leukocyte antigen heterozygosity on infectious disease outcome: The need for allele-specific measures

BACKGROUND: Doherty and Zinkernagel, who discovered that antigen presentation is restricted by the major histocompatibility complex (MHC, called HLA in humans), hypothesized that individuals heterozygous at particular MHC loci might be more resistant to particular infectious diseases than the corresponding homozygotes because heterozygotes could present a wider repertoire of antigens. The superiority of heterozygotes over either corresponding homozygote, which we term allele-specific overdominance, is of direct biological interest for understanding the mechanisms of immune response; it is also a leading explanation for the observation that MHC loci are extremely polymorphic and that these polymorphisms have been maintained through extremely long evolutionary periods. Recent studies have shown that in particular viral infections, heterozygosity at HLA loci was associated with a favorable disease outcome, and such findings have been interpreted as supporting the allele-specific overdominance hypothesis in humans. METHODS: An algebraic model is used to define the expected population-wide findings of an epidemiologic study of HLA heterozygosity and disease outcome as a function of allele-specific effects and population genetic parameters of the study population. RESULTS: We show that overrepresentation of HLA heterozygotes among individuals with favorable disease outcomes (which we term population heterozygote advantage) need not indicate allele-specific overdominance. On the contrary, partly due to a form of confounding by allele frequencies, population heterozygote advantage can occur under a very wide range of assumptions about the relationship between homozygote risk and heterozygote risk. In certain extreme cases, population heterozygote advantage can occur even when every heterozygote is at greater risk of being a case than either corresponding homozygote. CONCLUSION: To demonstrate allele-specific overdominance for specific infections in human populations, improved analytic tools and/or larger studies (or studies in populations with limited HLA diversity) are necessary

Canadian Arctic sea ice reconstructed from bromine in the Greenland NEEM ice core

Reconstructing the past variability of Arctic sea ice provides an essential context for recent multi-year sea ice decline, although few quantitative reconstructions cover the Holocene period prior to the earliest historical records 1,200 years ago. Photochemical recycling of bromine is observed over first-year, or seasonal, sea ice in so-called "bromine explosions" and we employ a 1-D chemistry transport model to quantify processes of bromine enrichment over first-year sea ice and depositional transport over multi-year sea ice and land ice. We report bromine enrichment in the Northwest Greenland Eemian NEEM ice core since the end of the Eemian interglacial 120,000 years ago, finding the maximum extension of first-year sea ice occurred approximately 9,000 years ago during the Holocene climate optimum, when Greenland temperatures were 2 to 3 degrees C above present values. First-year sea ice extent was lowest during the glacial stadials suggesting complete coverage of the Arctic Ocean by multi-year sea ice. These findings demonstrate a clear relationship between temperature and first-year sea ice extent in the Arctic and suggest multi-year sea ice will continue to decline as polar amplification drives Arctic temperatures beyond the 2 degrees C global average warming target of the recent COP21 Paris climate agreement

Altered branching patterns of Purkinje cells in mouse model for cortical development disorder

Disrupted cortical cytoarchitecture in cerebellum is a typical pathology in reeler. Particularly interesting are structural problems at the cellular level: dendritic morphology has important functional implication in signal processing. Here we describe a combinatorial imaging method of synchrotron X-ray microtomography with Golgi staining, which can deliver 3-dimensional(3-D) micro-architectures of Purkinje cell(PC) dendrites, and give access to quantitative information in 3-D geometry. In reeler, we visualized in 3-D geometry the shape alterations of planar PC dendrites (i.e., abnormal 3-D arborization). Despite these alterations, the 3-D quantitative analysis of the branching patterns showed no significant changes of the 77 ± 8° branch angle, whereas the branch segment length strongly increased with large fluctuations, comparing to control. The 3-D fractal dimension of the PCs decreased from 1.723 to 1.254, indicating a significant reduction of dendritic complexity. This study provides insights into etiologies and further potential treatment options for lissencephaly and various neurodevelopmental disorders