From aggregation to interpretation:how assessors judge complex data in a competency-based portfolio

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen assessors appraised one of three competency-based mock portfolios while thinking aloud, before taking part in semi-structured interviews. A thematic analysis of the think-aloud protocols and interviews revealed that assessors reached judgments through a 3-phase cyclical cognitive process of acquiring, organizing, and integrating evidence. Upon conclusion of the first cycle, assessors reviewed the remaining portfolio evidence to look for confirming or disconfirming evidence. Assessors were inclined to stick to their initial judgments even when confronted with seemingly disconfirming evidence. Although assessors reached similar final (pass-fail) judgments of students' professional competence, they differed in their information-processing approaches and the reasoning behind their judgments. Differences sprung from assessors' divergent assessment beliefs, performance theories, and inferences about the student. Assessment beliefs refer to assessors' opinions about what kind of evidence gives the most valuable and trustworthy information about the student's competence, whereas assessors' performance theories concern their conceptualizations of what constitutes professional competence and competent performance. Even when using the same pieces of information, assessors furthermore differed with respect to inferences about the student as a person as well as a (future) professional. Our findings support the notion that assessors' reasoning in judgment and decision-making varies and is guided by their mental models of performance assessment, potentially impacting feedback and the credibility of decisions. Our findings also lend further credence to the assertion that portfolios should be judged by multiple assessors who should, moreover, thoroughly substantiate their judgments. Finally, it is suggested that portfolios be designed in such a way that they facilitate the selection of and navigation through the portfolio evidence

Student perspectives on competency-based portfolios:Does a portfolio reflect their competence development?

INTRODUCTION: Portfolio-based assessments require that learners' competence development is adequately reflected in portfolio documentation. This study explored how students select and document performance data in their portfolios and how they perceive these data to be representative for their competence development. METHODS: Students uploaded performance data in a competency-based portfolio. During one clerkship period, twelve students also recorded an audio diary in which they reflected on experiences and feedback that they perceived to be indicants of their competence development. Afterwards, these students were interviewed to explore the extent to which the performance documentation in the portfolio corresponded with what they considered illustrative evidence of their development. The interviews were analyzed using thematic analysis. RESULTS: Portfolios provide an accurate but fragmented picture of student development. Portfolio documentation was influenced by tensions between learning and assessment, student beliefs about the goal of portfolios, student performance evaluation strategies, the learning environment and portfolio structure. DISCUSSION: This study confirms the importance of taking student perceptions into account when implementing a competency-based portfolio. Students would benefit from coaching on how to select meaningful experiences and performance data for documentation in their portfolios. Flexibility in portfolio structure and requirements is essential to ensure optimal fit between students' experienced competence development and portfolio content

Women in sport leadership in Botswana

Enquête quantitative et qualitative sur le nombre de femmes occupant des positions exécutives au sein des fédérations sportives et organes gouvernementaux liés aux domaines du sport

Democratizing Flexible Endoscopy Training. Noninferiority Randomized Trial Comparing a Box-Trainer vs a Virtual Reality Simulator to Prepare for the Fundamental of Endoscopic Surgery Exam

Background: A considerable number of surgical residents fail the mandated endoscopy exam despite having completed the required clinical cases. Low-cost endoscopy box trainers (BTs) could democratize training; however, their effectiveness was never compared with higher-cost virtual reality simulators (VRSs). Study design: In this randomized noninferiority trial, endoscopy novices trained either on the VRS used in the Fundamental of Endoscopic Surgery manual skills (FESms) exam, or a validated BT, the Basic Endoscopic Skills Training (BEST) box. Trainees were tested at fixed timepoints on the FESms and on standardized ex-vivo models. Primary endpoint was FESms improvement at 1 week. Secondary endpoints were: FESms improvement at 2 weeks, FESms pass rates, ex-vivo tests performance and trainees' feedback. Results: 77 trainees completed the study. VRS and BT trainees showed comparable FESms improvements (25.16±14.29 vs 25.58±11.75 FESms points, respectively; p=0.89), FESms pass rates (76.32% vs 61.54%, respectively; p=0.16) and total ex-vivo tasks completion times (365.76±237.56 vs 322.68±186.04 seconds, respectively; p=0.55) after one week. Performance were comparable also after 2 weeks of training, but FESms pass rates increased significantly only in the first week. Trainees were significantly more satisfied with the BT platform (3.97±1.20 vs 4.81±0.40 points on a 5-point Likert scale for the VRS and the BT, respectively; p<0.001). Conclusions: Simulation-based training is an effective mean to develop competency in endoscopy, especially at beginning of the learning curve. Low-cost BTs like the BEST box compare well with high-tech VRSs and could help democratize endoscopy training