Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA)

BACKGROUND: Ensuring equivalence of examiners' judgements across different groups of examiners is a priority for large scale performance assessments in clinical education, both to enhance fairness and reassure the public. This study extends insight into an innovation called Video-based Examiner Score Comparison and Adjustment (VESCA) which uses video scoring to link otherwise unlinked groups of examiners. This linkage enables comparison of the influence of different examiner-groups within a common frame of reference and provision of adjusted "fair" scores to students. Whilst this innovation promises substantial benefit to quality assurance of distributed Objective Structured Clinical Exams (OSCEs), questions remain about how the resulting score adjustments might be influenced by the specific parameters used to operationalise VESCA. Research questions, How similar are estimates of students' score adjustments when the model is run with either: fewer comparison videos per participating examiner?; reduced numbers of participating examiners? METHODS: Using secondary analysis of recent research which used VESCA to compare scoring tendencies of different examiner groups, we made numerous copies of the original data then selectively deleted video scores to reduce the number of 1/ linking videos per examiner (4 versus several permutations of 3,2,or 1 videos) or 2/examiner participation rates (all participating examiners (76%) versus several permutations of 70%, 60% or 50% participation). After analysing all resulting datasets with Many Facet Rasch Modelling (MFRM) we calculated students' score adjustments for each dataset and compared these with score adjustments in the original data using Spearman's correlations. RESULTS: Students' score adjustments derived form 3 videos per examiner correlated highly with score adjustments derived from 4 linking videos (median Rho = 0.93,IQR0.90-0.95,p < 0.001), with 2 (median Rho 0.85,IQR0.81-0.87,p < 0.001) and 1 linking videos (median Rho = 0.52(IQR0.46-0.64,p < 0.001) producing progressively smaller correlations. Score adjustments were similar for 76% participating examiners and 70% (median Rho = 0.97,IQR0.95-0.98,p < 0.001), and 60% (median Rho = 0.95,IQR0.94-0.98,p < 0.001) participation, but were lower and more variable for 50% examiner participation (median Rho = 0.78,IQR0.65-0.83, some ns). CONCLUSIONS: Whilst VESCA showed some sensitivity to the examined parameters, modest reductions in examiner participation rates or video numbers produced highly similar results. Employing VESCA in distributed or national exams could enhance quality assurance or exam fairness

HMMerThread: Detecting Remote, Functional Conserved Domains in Entire Genomes by Combining Relaxed Sequence-Database Searches with Fold Recognition

Conserved domains in proteins are one of the major sources of functional information for experimental design and genome-level annotation. Though search tools for conserved domain databases such as Hidden Markov Models (HMMs) are sensitive in detecting conserved domains in proteins when they share sufficient sequence similarity, they tend to miss more divergent family members, as they lack a reliable statistical framework for the detection of low sequence similarity. We have developed a greatly improved HMMerThread algorithm that can detect remotely conserved domains in highly divergent sequences. HMMerThread combines relaxed conserved domain searches with fold recognition to eliminate false positive, sequence-based identifications. With an accuracy of 90%, our software is able to automatically predict highly divergent members of conserved domain families with an associated 3-dimensional structure. We give additional confidence to our predictions by validation across species. We have run HMMerThread searches on eight proteomes including human and present a rich resource of remotely conserved domains, which adds significantly to the functional annotation of entire proteomes. We find ∼4500 cross-species validated, remotely conserved domain predictions in the human proteome alone. As an example, we find a DNA-binding domain in the C-terminal part of the A-kinase anchor protein 10 (AKAP10), a PKA adaptor that has been implicated in cardiac arrhythmias and premature cardiac death, which upon stress likely translocates from mitochondria to the nucleus/nucleolus. Based on our prediction, we propose that with this HLH-domain, AKAP10 is involved in the transcriptional control of stress response. Further remotely conserved domains we discuss are examples from areas such as sporulation, chromosome segregation and signalling during immune response. The HMMerThread algorithm is able to automatically detect the presence of remotely conserved domains in proteins based on weak sequence similarity. Our predictions open up new avenues for biological and medical studies. Genome-wide HMMerThread domains are available at http://vm1-hmmerthread.age.mpg.de

An investigation into the signal transduction pathways involved in mGluR dependent long-term depression

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Fast silencing reveals a lost role for reciprocal inhibition in locomotion

Summary Alternating contractions of antagonistic muscle groups during locomotion are generated by spinal “half-center” networks coupled in antiphase by reciprocal inhibition. It is widely thought that reciprocal inhibition only coordinates the activity of these muscles. We have devised two methods to rapidly and selectively silence neurons on just one side of Xenopus tadpole spinal cord and hindbrain, which generate swimming rhythms. Silencing activity on one side led to rapid cessation of activity on the other side. Analyses reveal that this resulted from the depression of reciprocal inhibition connecting the two sides. Although critical neurons in intact tadpoles are capable of pacemaker firing individually, an effect that could support motor rhythms without inhibition, the swimming network itself requires ∼23 min to regain rhythmic activity after blocking inhibition pharmacologically, implying some homeostatic changes. We conclude therefore that reciprocal inhibition is critical for the generation of normal locomotor rhythm.Publisher PDFPeer reviewe

The control of locomotor frequency by excitation and inhibition

Every type of neural rhythm has its own operational range of frequency. Neuronal mechanisms underlying rhythms at different frequencies, however, are poorly understood. We use a simple aquatic vertebrate, the two-day-old Xenopus tadpole, to investigate how the brainstem and spinal circuits generate swimming rhythms of different speeds. We first determined that the basic motor output pattern was not altered with varying swimming frequencies. The firing reliability of different types of rhythmic neuron involved in swimming was then analyzed. The results showed that there was a drop in the firing reliability in some inhibitory interneurons when fictive swimming slowed. We have recently established that premotor excitatory interneurons [descending interneurons (dINs)] are critical in rhythmically driving activity in the swimming circuit. Voltage-clamp recordings from dINs showed higher frequency swimming correlated with stronger background excitation and phasic inhibition, but did not correlate with phasic excitation. Two parallel mechanisms have been proposed for tadpole swimming maintenance: postinhibition rebound firing and NMDAR-dependent pacemaker firing in dINs. Rebound tests in dINs in this study showed that greater background depolarization and phasic inhibition led to faster rebound firing. Higher depolarization was previously shown to accelerate dIN pacemaker firing in the presence of NMDA. Here we show that enhancing dIN background excitation during swimming speeds up fictive swimming frequency while weakening phasic inhibition without changing background excitation slows down swimming rhythms. We conclude that both strong background excitation and phasic inhibition can promote faster tadpole swimming

Fast silencing reveals a lost role for reciprocal inhibition in locomotion

Summary Alternating contractions of antagonistic muscle groups during locomotion are generated by spinal “half-center” networks coupled in antiphase by reciprocal inhibition. It is widely thought that reciprocal inhibition only coordinates the activity of these muscles. We have devised two methods to rapidly and selectively silence neurons on just one side of Xenopus tadpole spinal cord and hindbrain, which generate swimming rhythms. Silencing activity on one side led to rapid cessation of activity on the other side. Analyses reveal that this resulted from the depression of reciprocal inhibition connecting the two sides. Although critical neurons in intact tadpoles are capable of pacemaker firing individually, an effect that could support motor rhythms without inhibition, the swimming network itself requires ∼23 min to regain rhythmic activity after blocking inhibition pharmacologically, implying some homeostatic changes. We conclude therefore that reciprocal inhibition is critical for the generation of normal locomotor rhythm

Determining influence, interaction and causality of contrast and sequence effects in objective structured clinical exams

INTRODUCTION: Differential rater function over time (DRIFT) and contrast effects (examiners' scores biased away from the standard of preceding performances) both challenge the fairness of scoring in Objective Structured Clinical Exams (OSCEs). This is important as, under some circumstances, these effects could alter whether some candidates pass or fail assessments. Benefitting from experimental control, this study investigated the causality, operation and interaction of both effects simultaneously for the first time in an OSCE setting. METHODS: We used secondary analysis of data from an OSCE in which examiners scored embedded videos of student performances interspersed between live students. Embedded video position varied between examiners (early vs late) whilst the standard of preceding performances naturally varied (previous high or low). We examined linear relationships suggestive of DRIFT and contrast effects in all within-OSCE data before comparing the influence and interaction of "Early" vs "Late" and "PreviousHigh" vs "PreviousLow" conditions on embedded video scores. RESULTS: Linear relationships data did not support the presence of DRIFT or contrast effects. Embedded videos were scored higher early (19.9(19.4-20.5)) vs late (18.6(18.1-19.1), p<0.001) but scores did not differ between PreviousHigh and PreviousLow conditions. The interaction term was non-significant. CONCLUSIONS: In this instance, the small DRIFT effect we observed on embedded video can be causally attributed to examiner behaviour. Contrast effects appear less ubiquitous than some prior research suggests. Possible mediators of these finding include: OSCE context, detail of task specification, examiners' cognitive load and the distribution of learners' ability. As the operation of these effects appears to vary across contexts, further research is needed to determine the prevalence and mechanisms of contrast and DRIFT effects, so that assessments may be designed in ways which are likely to avoid their occurrence. Quality assurance should monitor for these contextually variable effects in order to ensure OSCE equivalence

CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods

BackgroundThe Critical Assessment of Genome Interpretation (CAGI) aims to advance the state-of-the-art for computational prediction of genetic variant impact, particularly where relevant to disease. The five complete editions of the CAGI community experiment comprised 50 challenges, in which participants made blind predictions of phenotypes from genetic data, and these were evaluated by independent assessors.ResultsPerformance was particularly strong for clinical pathogenic variants, including some difficult-to-diagnose cases, and extends to interpretation of cancer-related variants. Missense variant interpretation methods were able to estimate biochemical effects with increasing accuracy. Assessment of methods for regulatory variants and complex trait disease risk was less definitive and indicates performance potentially suitable for auxiliary use in the clinic.ConclusionsResults show that while current methods are imperfect, they have major utility for research and clinical applications. Emerging methods and increasingly large, robust datasets for training and assessment promise further progress ahead