1,038 research outputs found
Developing a data repository of standard concussion assessment clinical data for research involving college athletes
LINKING ENGLISH-LANGUAGE TEST SCORES ONTO THE COMMON EUROPEAN FRAMEWORK OF REFERENCE: AN APPLICATION OF STANDARD-SETTING METHODOLOGY
The Role of Consequences in validity Theory
Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/72502/1/j.1745-3992.1998.tb00826.x.pd
Don't forget the group! The importance of social norms and empathy for shaping donation behaviour
Test development and use in five Iberian Latin American countries
The abundance of scholarship on test development and use generally is higher in English-speaking than in Iberian Latin
American countries. The purpose of this article is to help overcome this imbalance by describing and identifying
similarities and differences in test development and use in two Iberian (Portugal and Spain) and three of the largest
Latin American (Argentina, Brazil, and Venezuela) countries. The stages of test development in each country, roles of
professional associations, presence of standards for test use, professionalsâ educational training, commonly used tests,
together with prominent challenges to continued progress are discussed. Test development and use in these five countries
are transitioning from a dependence on the use of translated tests to greater reliance on adapted and finally nationally
constructed tests. Continued growth requires adherence to international standards guiding test development and use.
Stronger alliance among professional associations in the Iberian Latin American countries could serve as a catalyst to
promote test development in these regions.A abundĂąncia de estudos sobre o desenvolvimento do teste e usar geralmente Ă© maior em InglĂȘs de lĂngua do que nos paĂses latino-americanos ibĂ©ricos. O objetivo deste artigo Ă© ajudar a superar este desequilĂbrio, descrever e identificar semelhanças e diferenças no desenvolvimento de testes e uso em dois IbĂ©rica (Portugal e Espanha) e trĂȘs dos maiores paĂses da AmĂ©rica Latina (Argentina, Brasil e Venezuela). Os estĂĄgios de desenvolvimento do teste em cada paĂs, os papĂ©is das associaçÔes profissionais, presença de padrĂ”es para uso de teste, a formação dos profissionais da educação, os testes comumente utilizados, juntamente com desafios importantes ao progresso continuado sĂŁo discutidos. Desenvolvimento de testes e uso nestes cinco paĂses estĂŁo em transição de uma dependĂȘncia do uso de testes traduzidos para uma maior dependĂȘncia de testes adaptados e finalmente construĂdos nacionalmente. O crescimento contĂnuo exige a adesĂŁo a padrĂ”es internacionais orientadores desenvolvimento de testes e uso. Aliança mais forte entre as associaçÔes profissionais dos paĂses latino-americanos ibĂ©ricos poderia servir como um catalisador para promover o desenvolvimento do teste nessas regiĂ”es
Using Differential Item Functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment
BACKGROUND: Fairness is a critical component of defensible assessment. Candidates should perform according to ability without influence from background characteristics such as ethnicity or sex. However, performance differs by candidate background in many assessment environments. Many potential causes of such differences exist, and examinations must be routinely analysed to ensure they do not present inappropriate progression barriers for any candidate group. By analysing the individual questions of an examination through techniques such as Differential Item Functioning (DIF), we can test whether a subset of unfair questions explains group-level differences. Such items can then be revised or removed. METHODS: We used DIF to investigate fairness for 13,694 candidates sitting a major international summative postgraduate examination in internal medicine. We compared (a) ethnically white UK graduates against ethnically non-white UK graduates and (b) male UK graduates against female UK graduates. DIF was used to test 2773 questions across 14 sittings. RESULTS: Across 2773 questions eight (0.29%) showed notable DIF after correcting for multiple comparisons: seven medium effects and one large effect. Blinded analysis of these questions by a panel of clinician assessors identified no plausible explanations for the differences. These questions were removed from the question bank and we present them here to share knowledge of questions with DIF. These questions did not significantly impact the overall performance of the cohort. Group-level differences in performance between the groups we studied in this examination cannot be explained by a subset of unfair questions. CONCLUSIONS: DIF helps explore fairness in assessment at the question level. This is especially important in high-stakes assessment where a small number of unfair questions may adversely impact the passing rates of some groups. However, very few questions exhibited notable DIF so differences in passing rates for the groups we studied cannot be explained by unfairness at the question level
Recommended from our members
Practitioners' views and barriers to implementation of the Keeping Birth Normal tool: A pilot study
Background:
Poor implementation of evidence in practice has been reported as a reason behind the continued rise in unnecessary interventions in labour and birth. A validated tool can enable the systematic measurement of care to target interventions to support implementation of evidence. The Keeping Birth Normal tool has been developed to measure and support implementation of evidence to reduce unnecessary interventions in labour and birth.
Aims:
This pilot sought the views of midwives about the usefulness and relevance of the Keeping Birth Normal tool in measuring and supporting practice; it also identified barriers to implementation.
Methods:
Five midwives supported by five preceptors tested the tool on a delivery suite and birth centre in a local NHS Trust. Mixed methods were employed. Participants completed a questionnaire about the relevance and usefulness of the tool. Semi-structured interviews explored participants' experience of using the tool in practice.
Findings:
The domains and items in the tool were viewed as highly relevant to reducing unnecessary interventions. Not all midwives were open to their practice being observed, but those who were reported benefits from critical reflection and role-modelling to support implementation. An important barrier is a lack of expertise among preceptors to support the implementation of skills to reduce unnecessary interventions. This includes skills in the use of rating scales and critical reflection. Where expertise is available, there is a lack of protected time for such structured supportive activity. Norms in birth environments that do not promote normal birth are another important barrier.
Conclusions:
Midwives found the items in the tool relevant to evidence-informed skills to reduce unnecessary interventions and useful for measuring and supporting implementation. To validate and generalise these findings, further evidence about the quality of items needs to be gathered. Successful implementation of the tool requires preceptors skilled in care that reduces unnecessary interventions, using rating scales, role-modelling and critical reflection. Such structured preceptorship requires protected time and can only thrive in a culture that promotes normal birth
Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity
Linking tests of English for academic purposes to the CEFR: the score userâs perspective
The Common European Framework of Reference for Languages (CEFR) is widely used in setting language proficiency requirements, including for international students seeking access to university courses taught in English. When different language examinations have been related to the CEFR, the process is claimed to help score users, such as university admissions staff, to compare and evaluate these examinations as tools for selecting qualified applicants. This study analyses the linking claims made for four internationally recognised tests of English widely used in university admissions. It uses the Council of Europeâs (2009) suggested stages of specification, standard setting, and empirical validation to frame an evaluation of the extent to which, in this context, the CEFR has fulfilled its potential to âfacilitate comparisons between different systems of qualifications.â Findings show that testing agencies make little use of CEFR categories to explain test content; represent the relationships between their tests and the framework in different terms; and arrive at conflicting conclusions about the correspondences between test scores and CEFR levels. This raises questions about the capacity of the CEFR to communicate competing views of a test construct within a coherent overarching structure
- âŠ