Scoring method of a Situational Judgment Test:influence on internal consistency reliability, adverse impact and correlation with personality?

textabstractSituational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used

Medical student changes in self-regulated learning during the transition to the clinical environment

BACKGROUND: Self-regulated learning (SRL), which is learners’ ability to proactively select and use different strategies to reach learning goals, is associated with academic and clinical success and life-long learning. SRL does not develop automatically in the clinical environment and its development during the preclinical to clinical learning transition has not been quantitatively studied. Our study aims to fill this gap by measuring SRL in medical students during the transitional period and examining its contributing factors. METHODS: Medical students were invited to complete a questionnaire at the commencement of their first clinical year (T0), and 10 weeks later (T1). The questionnaire included the Motivated Strategies for Learning Questionnaire (MSLQ) and asked about previous clinical experience. Information about the student’s background, demographic characteristics and first clinical rotation were also gathered. RESULTS: Of 118 students invited to participate, complete paired responses were obtained from 72 medical students (response rate 61%). At T1, extrinsic goal orientation increased and was associated with gender (males were more likely to increase extrinsic goal orientation) and type of first attachment (critical care and community based attachments, compared to hospital ward based attachments). Metacognitive self-regulation decreased at T1 and was negatively associated with previous clinical experience. CONCLUSIONS: Measurable changes in self-regulated learning occur during the transition from preclinical learning to clinical immersion, particularly in the domains of extrinsic goal orientation and metacognitive self–regulation. Self–determination theory offers possible explanations for this finding which have practical implications and point the way to future research. In addition, interventions to promote metacognition before the clinical immersion may assist in preserving SRL during the transition and thus promote life-long learning skills in preparation for real-world practice. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12909-017-0902-7) contains supplementary material, which is available to authorized users

The self-regulated learning of medical students in the clinical environment – a scoping review

Abstract Background Self-regulated learning is the individual’s ability to effectively use various strategies to reach their learning goals. We conducted this scoping review to explore what has been found regarding self-regulated learning in the clinical environment and how this was measured. Methods Using Arksey and O’Malley’s five-stage framework, we searched three medical and educational databases as well as Google Scholar for literature on the self-regulated learning of medical students in the clinical environment published between 1966 and February 2017. After results were screened and relevant studies were identified, the data was summarised and discursively reported. Results The search resulted in 911 articles, with 14 articles included in the scoping review after the inclusion criteria was applied. Self-regulated learning was explored in these studies in various ways including qualitative, quantitative and mixed methods. Three major findings were found: 1) levels of self-regulated learning change in the clinical environment, 2) self-regulated learning is associated with academic achievement, success in clinical skills and mental health and 3) various factors can support self-regulated learning levels in medical students. Conclusions Most of articles exploring the self-regulated learning of medical students during the clinical years have been published in the last 5 years, suggesting a growing interest in the area. Future research could explore the self-regulated learning levels of medical students during the clinical years using a longitudinal approach or through the use of novel qualitative approaches