Do personality traits assessed on medical school admission predict exit performance?:A UK-wide longitudinal cohort studyA2

We thank the UKCAT Research Group for funding this independent evaluation and thank Rachel Greatrix and Sandra Nicholson of the UKCAT Consortium for their support throughout this project, and their feedback on the draft paper. We also thank Professor Amanda Lee and Ms Katie Wilde for their input into the application for funding, and ongoing support.Peer reviewedPublisher PD

A comparison of course-related stressors in undergraduate problem-based learning (PBL) versus non-PBL medical programmes

Background: Medical students report high levels of stress related to their medical training as well as to other personal and financial factors. The aim of this study is to investigate whether there are differences in course-related stressors reported by medical students on undergraduate problem-based learning (PBL) and non-PBL programmes in the UK. Method: A cross-sectional study of second-year medical students in two UK medical schools (one PBL and one non-PBL programme) was conducted. A 16-question self-report questionnaire, derived from the Perceived Medical Student Stress Scale and the Higher Education Stress Inventory, was used to measure course-related stressors. Following univariate analysis of each stressor between groups, multivariate logistic regression was used to determine which stressors were the best predictors of each course type, while controlling for socio-demographic differences between the groups. Results: A total of 280 students responded. Compared to the non-PBL students (N = 197), the PBL students (N = 83) were significantly more likely to agree that: they did not know what the faculty expected of them (Odds Ratio (OR) = 0.38, p = 0.03); there were too many small group sessions facilitated only by students resulting in an unclear curriculum (OR = 0.04, p < 0.0001); and that there was a lack of opportunity to explore academic subjects of interest (OR = 0.40, p = 0.02). They were significantly more likely to disagree that: there was a lack of encouragement from teachers (OR = 3.11, p = 0.02); and that the medical course fostered a sense of anonymity and feelings of isolation amongst students (OR = 3.42, p = 0.008). Conclusion: There are significant differences in the perceived course-related stressors affecting medical students on PBL and non-PBL programmes. Course designers and student support services should therefore tailor their work to minimise, or help students cope with, the specific stressors on each course type to ensure optimum learning and wellbeing among our future doctors

Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100

Why use SJTs? Traditionally, selection into medical education professions has focused primarily upon academic ability alone. This approach has been questioned more recently, as although academic attainment predicts performance early in training, research shows it has less predictive power for demonstrating competence in postgraduate clinical practice. Such evidence, coupled with an increasing focus on individuals working in healthcare roles displaying the core values of compassionate care, benevolence and respect, illustrates that individuals should be selected on attributes other than academic ability alone. Moreover, there are mounting calls to widen access to medicine, to ensure that selection methods do not unfairly disadvantage individuals from specific groups (e.g. regarding ethnicity or socio-economic status), so that the future workforce adequately represents society as a whole. These drivers necessitate a method of assessment that allows individuals to be selected on important non-academic attributes that are desirable in healthcare professionals, in a fair, reliable and valid way. [Table: see text] What are SJTs? Situational judgement tests (SJTs) are tests used to assess individuals' reactions to a number of hypothetical role-relevant scenarios, which reflect situations candidates are likely to encounter in the target role. These scenarios are based on a detailed analysis of the role and should be developed in collaboration with subject matter experts, in order to accurately assess the key attributes that are associated with competent performance. From a theoretical perspective, SJTs are believed to measure prosocial Implicit Trait Policies (ITPs), which are shaped by socialisation processes that teach the utility of expressing certain traits in different settings such as agreeable expressions (e.g. helping others in need), or disagreeable actions (e.g. advancing ones own interest at others, expense). Are SJTs reliable, valid and fair? Several studies, including good quality meta-analytic and longitudinal research, consistently show that SJTs used in many different occupational groups are reliable and valid. Although there is over 40 years of research evidence available on SJTs, it is only within the past 10 years that SJTs have been used for recruitment into medicine. Specifically, evidence consistently shows that SJTs used in medical selection have good reliability, and predict performance across a range of medical professions, including performance in general practice, in early years (foundation training as a junior doctor) and for medical school admissions. In addition, SJTs have been found to have significant added value (incremental validity) over and above other selection methods such as knowledge tests, measures of cognitive ability, personality tests and application forms. Regarding differential attainment, generally SJTs have been found to have lower adverse impact compared to other selection methods, such as cognitive ability tests. SJTs have the benefit of being appropriate both for use in selection where candidates are novices (i.e. have no prior role experience or knowledge such as in medical school admissions) as well as settings where candidates have substantial job knowledge and specific experience (as in postgraduate recruitment for more senior roles). An SJT specification (e.g. scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT. Once designed however, SJTs are cost efficient to administer to large numbers of candidates compared to other tests of non-academic attributes (e.g. personal statements, structured interviews), as they are standardised and can be computer-delivered and machine-marked

Scoring method of a Situational Judgment Test:influence on internal consistency reliability, adverse impact and correlation with personality?

textabstractSituational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used

Impaired work functioning due to common mental disorders in nurses and allied health professionals: the Nurses Work Functioning Questionnaire

Common mental disorders (CMD) negatively affect work functioning. In the health service sector not only the prevalence of CMDs is high, but work functioning problems are associated with a risk of serious consequences for patients and healthcare providers. If work functioning problems due to CMDs are detected early, timely help can be provided. Therefore, the aim of this study is to develop a detection questionnaire for impaired work functioning due to CMDs in nurses and allied health professionals working in hospitals. First, an item pool was developed by a systematic literature study and five focus group interviews with employees and experts. To evaluate the content validity, additional interviews were held. Second, a cross-sectional assessment of the item pool in 314 nurses and allied health professionals was used for item selection and for identification and corroboration of subscales by explorative and confirmatory factor analysis. The study results in the Nurses Work Functioning Questionnaire (NWFQ), a 50-item self-report questionnaire consisting of seven subscales: cognitive aspects of task execution, impaired decision making, causing incidents at work, avoidance behavior, conflicts and irritations with colleagues, impaired contact with patients and their family, and lack of energy and motivation. The questionnaire has a proven high content validity. All subscales have good or acceptable internal consistency. The Nurses Work Functioning Questionnaire gives insight into precise and concrete aspects of impaired work functioning of nurses and allied health professionals. The scores can be used as a starting point for purposeful intervention

Why Do Situational Interviews Predict Performance? Is it Saying How You Would Behave or Knowing How You Should Behave?

Purpose: The present study examined two theoretical explanations for why situational interviews predict work-related performance, namely (a) that they are measures of interviewees’ behavioral intentions or (b) that they are measures of interviewees’ ability to correctly decipher situational demands. Design/Methodology/Approach: We tested these explanations with 101 students, who participated in a 2-day selection simulation. Findings: In line with the first explanation, there was considerable similarity between what participants said they would do and their actual behavior in corresponding work-related situations. However, the underlying postulated mechanism was not supported by the data. In line with the second explanation, participants’ ability to correctly decipher situational demands was related to performance in both the interview and work-related situations. Furthermore, the relationship between the interview and performance in the work-related situations was partially explained by this ability to decipher situational demands. Implications: Assessing interviewees’ ability to identify criteria might be of additional value for making selection decisions, particularly for jobs where it is essential to assess situational demands. Originality/Value: The present study made an effort to open the ‘black box’ of situational interview validity by examining two explanations for their validity. The results provided only moderate support for the first explanation. However, the second explanation was fully supported by these results

Leading Mindfully: Two Studies of the Influence of Supervisor Trait Mindfulness on Employee Well-Being and Performance

10.1007/s12671-012-0144-zMindfulness5136-4