Exploring differences in individual and group judgements in standard setting

Context Standard setting is critically important to assessment decisions in medical education. Recent research has demonstrated variations between medical schools in the standards set for shared items. Despite the centrality of judgement to criterion-referenced standard setting methods, little is known about the individual or group processes that underpin them. This study aimed to explore the operation and interaction of these processes in order to illuminate potential sources of variability. Methods Using qualitative research, we purposively sampled across UK medical schools that set a low, medium or high standard on nationally shared items, collecting data by observation of graduation-level standard-setting meetings and semi-structured interviews with standard-setting judges. Data were analysed using thematic analysis based on the principles of grounded theory. Results Standard setting occurred through the complex interaction of institutional context, judges’ individual perspectives and group interactions. Schools’ procedures, panel members and atmosphere produced unique contexts. Individual judges formed varied understandings of the clinical and technical features of each question, relating these to their differing (sometimes contradictory) conceptions of minimally competent students, by balancing information and making suppositions. Conceptions of minimal competence variously comprised: limited attendance; limited knowledge; poor knowledge application; emotional responses to questions; ‘test-savviness’, or a strategic focus on safety. Judges experienced tensions trying to situate these abstract conceptions in reality, revealing uncertainty. Groups constructively revised scores through debate, sharing information and often constructing detailed clinical representations of cases. Groups frequently displayed conformity, illustrating a belief that outlying judges were likely to be incorrect. Less frequently, judges resisted change, using emphatic language, bargaining or, rarely, ‘polarisation’ to influence colleagues. Conclusions Despite careful conduct through well-established procedures, standard setting is judgementally complex and involves uncertainty. Understanding whether or how these varied processes produce the previously observed variations in outcomes may offer routes to enhance equivalence of criterion-referenced standards

Physical Origin of Differences among various Measures of Solar Meridional Circulation

We show that systematic differences between surface Doppler and magnetic element tracking measures of solar meridional flow can be explained by the effects of surface turbulent magnetic diffusion. Feature-tracking speeds are lower than plasma speeds in low and mid-latitudes, because magnetic diffusion opposes poleward plasma flow in low-latitudes whereas it adds to plasma flow at high latitudes. Flux transport dynamo models must input plasma flow; the model-outputs yield estimates of the surface magnetic feature tracking speed. We demonstrate that the differences between plasma speed and magnetic pattern speed in a flux-transport dynamo are consistent with the observed difference between these speeds.Comment: To appear in Ap

AlphaFold predicts the most complex protein knot and composite protein knots

The computer artificial intelligence system AlphaFold has recently predicted previously unknown three-dimensional structures of thousands of proteins. Focusing on the subset with high-confidence scores, we algorithmically analyze these predictions for cases where the protein backbone exhibits rare topological complexity, i.e. knotting. Amongst others, we discovered a $7_1$-knot, the most topologically complex knot ever found in a protein, as well several 6-crossing composite knots comprised of two methyltransferase or carbonic anhydrase domains, each containing a simple trefoil knot. These deeply embedded composite knots occur evidently by gene duplication and interconnection of knotted dimers. Finally, we report two new five-crossing knots including the first $5_1$-knot. Our list of analyzed structures forms the basis for future experimental studies to confirm these novel knotted topologies and to explore their complex folding mechanisms.Comment: This article appeared openly accessible in M. A. Brems et al., Protein Science. 2022; 31( 8):e4380 and may be found at https://doi.org/10.1002/pro.438

Understanding and developing procedures for video-based assessment in medical education

IntroductionNovel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We aimed to understand examiners’ and students’ interaction with video whilst developing procedures to promote its optimal use.MethodsUsing design-based research we developed theory and procedures for video use in assessment, iteratively adapting conditions across simulated OSCE stations. We explored examiners’ and students’ perceptions using think-aloud, interviews and focus group. Data were analysed using constructivist grounded-theory methods.ResultsVideo-based assessment produced detachment and reduced volitional control for examiners. Examiners ability to make valid video-based judgements was mediated by the interaction of station content and specifically selected filming parameters. Examiners displayed several judgemental tendencies which helped them manage videos’ limitations but could also bias judgements in some circumstances. Students rarely found carefully-placed cameras intrusive and considered filming acceptable if adequately justified.DiscussionSuccessful use of video-based assessment relies on balancing the need to ensure station-specific information adequacy; avoiding disruptive intrusion; and the degree of justification provided by video’s educational purpose. Video has the potential to enhance assessment validity and students’ learning when an appropriate balance is achieved

Technology enhanced assessment: Ottawa consensus statement and recommendations

INTRODUCTION In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how learners are assessed. In this updated consensus, we bring together the potential of technology and the ultimate goals of assessment on learner attainment, faculty development, and improved healthcare practices. METHODS As a material for the report, we used the scholarly publications on TEA in both HPE and general higher education, feedback from 2020 Ottawa Conference workshops, and scholarly publications on assessment technology practices during the Covid-19 pandemic. RESULTS AND CONCLUSION The group identified areas of consensus that remained to be resolved and issues that arose in the evolution of TEA. We adopted a three-stage approach (readiness to adopt technology, application of assessment technology, and evaluation/dissemination). The application stage adopted an assessment ‘lifecycle’ approach and targeted five key foci: (1) Advancing authenticity of assessment, (2) Engaging learners with assessment, (3) Enhancing design and scheduling, (4) Optimising assessment delivery and recording learner achievement, and (5) Tracking learner progress and faculty activity and thereby supporting longitudinal learning and continuous assessment.Peer reviewe

A Method for Data-Driven Simulations of Evolving Solar Active Regions

We present a method for performing data-driven simulations of solar active region formation and evolution. The approach is based on magnetofriction, which evolves the induction equation assuming the plasma velocity is proportional to the Lorentz force. The simulations of active region coronal field are driven by temporal sequences of photospheric magnetograms from the Helioseismic Magnetic Imager (HMI) instrument onboard the Solar Dynamics Observatory (SDO). Under certain conditions, the data-driven simulations produce flux ropes that are ejected from the modeled active region due to loss of equilibrium. Following the ejection of flux ropes, we find an enhancement of the photospheric horizontal field near the polarity inversion line. We also present a method for the synthesis of mock coronal images based on a proxy emissivity calculated from the current density distribution in the model. This method yields mock coronal images that are somewhat reminiscent of images of active regions taken by instruments such as SDO's Atmospheric Imaging Assembly (AIA) at extreme ultraviolet wavelengths.Comment: Accepted to ApJ; comments/questions related to this article are welcome via e-mail, even after publicatio