Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100

Why use SJTs? Traditionally, selection into medical education professions has focused primarily upon academic ability alone. This approach has been questioned more recently, as although academic attainment predicts performance early in training, research shows it has less predictive power for demonstrating competence in postgraduate clinical practice. Such evidence, coupled with an increasing focus on individuals working in healthcare roles displaying the core values of compassionate care, benevolence and respect, illustrates that individuals should be selected on attributes other than academic ability alone. Moreover, there are mounting calls to widen access to medicine, to ensure that selection methods do not unfairly disadvantage individuals from specific groups (e.g. regarding ethnicity or socio-economic status), so that the future workforce adequately represents society as a whole. These drivers necessitate a method of assessment that allows individuals to be selected on important non-academic attributes that are desirable in healthcare professionals, in a fair, reliable and valid way. [Table: see text] What are SJTs? Situational judgement tests (SJTs) are tests used to assess individuals' reactions to a number of hypothetical role-relevant scenarios, which reflect situations candidates are likely to encounter in the target role. These scenarios are based on a detailed analysis of the role and should be developed in collaboration with subject matter experts, in order to accurately assess the key attributes that are associated with competent performance. From a theoretical perspective, SJTs are believed to measure prosocial Implicit Trait Policies (ITPs), which are shaped by socialisation processes that teach the utility of expressing certain traits in different settings such as agreeable expressions (e.g. helping others in need), or disagreeable actions (e.g. advancing ones own interest at others, expense). Are SJTs reliable, valid and fair? Several studies, including good quality meta-analytic and longitudinal research, consistently show that SJTs used in many different occupational groups are reliable and valid. Although there is over 40 years of research evidence available on SJTs, it is only within the past 10 years that SJTs have been used for recruitment into medicine. Specifically, evidence consistently shows that SJTs used in medical selection have good reliability, and predict performance across a range of medical professions, including performance in general practice, in early years (foundation training as a junior doctor) and for medical school admissions. In addition, SJTs have been found to have significant added value (incremental validity) over and above other selection methods such as knowledge tests, measures of cognitive ability, personality tests and application forms. Regarding differential attainment, generally SJTs have been found to have lower adverse impact compared to other selection methods, such as cognitive ability tests. SJTs have the benefit of being appropriate both for use in selection where candidates are novices (i.e. have no prior role experience or knowledge such as in medical school admissions) as well as settings where candidates have substantial job knowledge and specific experience (as in postgraduate recruitment for more senior roles). An SJT specification (e.g. scenario content, response instructions and format) may differ depending on the level of job knowledge required. Research consistently shows that SJTs are usually found to be positively received by candidates compared to other selection tests such as cognitive ability and personality tests. Practically, SJTs are difficult to design effectively, and significant expertise is required to build a reliable and valid SJT. Once designed however, SJTs are cost efficient to administer to large numbers of candidates compared to other tests of non-academic attributes (e.g. personal statements, structured interviews), as they are standardised and can be computer-delivered and machine-marked

Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression

Phage integrase genes often play a role in the establishment of lysogeny in temperate phage by catalyzing the integration of the phage into one of the host's replicons. To investigate temperate phage gene expression, an induced viral metagenome from Tampa Bay was sequenced by 454/Pyrosequencing. The sequencing yielded 294,068 reads with 6.6% identifiable. One hundred-three sequences had significant similarity to integrases by BLASTX analysis (e≤0.001). Four sequences with strongest amino-acid level similarity to integrases were selected and real-time PCR primers and probes were designed. Initial testing with microbial fraction DNA from Tampa Bay revealed 1.9×107, and 1300 gene copies of Vibrio-like integrase and Oceanicola-like integrase L−1 respectively. The other two integrases were not detected. The integrase assay was then tested on microbial fraction RNA extracted from 200 ml of Tampa Bay water sampled biweekly over a 12 month time series. Vibrio-like integrase gene expression was detected in three samples, with estimated copy numbers of 2.4-1280 L−1. Clostridium-like integrase gene expression was detected in 6 samples, with estimated copy numbers of 37 to 265 L−1. In all cases, detection of integrase gene expression corresponded to the occurrence of lysogeny as detected by prophage induction. Investigation of the environmental distribution of the two expressed integrases in the Global Ocean Survey Database found the Vibrio-like integrase was present in genome equivalents of 3.14% of microbial libraries and all four viral metagenomes. There were two similar genes in the library from British Columbia and one similar gene was detected in both the Gulf of Mexico and Sargasso Sea libraries. In contrast, in the Arctic library eleven similar genes were observed. The Clostridium-like integrase was less prevalent, being found in 0.58% of the microbial and none of the viral libraries. These results underscore the value of metagenomic data in discovering signature genes that play important roles in the environment through their expression, as demonstrated by integrases in lysogeny

Do personality traits assessed on medical school admission predict exit performance?:A UK-wide longitudinal cohort studyA2

We thank the UKCAT Research Group for funding this independent evaluation and thank Rachel Greatrix and Sandra Nicholson of the UKCAT Consortium for their support throughout this project, and their feedback on the draft paper. We also thank Professor Amanda Lee and Ms Katie Wilde for their input into the application for funding, and ongoing support.Peer reviewedPublisher PD

Assessing the psychometric and ecometric properties of neighborhood scales using adolescent survey data from urban and rural Scotland

This work was supported by NHS Health Scotland and the University of St Andrews.Background:  Despite the well-established need for specific measurement instruments to examine the relationship between neighborhood conditions and adolescent well-being outcomes, few studies have developed scales to measure features of the neighborhoods in which adolescents reside. Moreover, measures of neighborhood features may be operationalised differently by adolescents living in different levels of urban/rurality. This has not been addressed in previous studies. The objectives of this study were to: 1) establish instruments to measure adolescent neighborhood features at both the individual and neighborhood level, 2) assess their psychometric and ecometric properties, 3) test for invariance by urban/rurality, and 4) generate neighborhood level scores for use in further analysis. Methods:  Data were from the Scottish 2010 Health Behaviour in School-aged Children Survey, which included an over-sample of rural adolescents. The survey responses of interest came from questions designed to capture different facets of the local area in which each respondent resided. Intermediate data zones were used as proxies for neighborhoods. Internal consistency was evaluated by Cronbach’s alpha. Invariance was examined using confirmatory factor analysis. Multilevel models were used to estimate ecometric properties and generate neighborhood scores. Results:  Two constructs labeled neighborhood social cohesion and neighborhood disorder were identified. Adjustment was made to the originally specified model to improve model fit and measures of invariance. At the individual level, reliability was .760 for social cohesion and .765 for disorder, and between .524 and .571 for both constructs at the neighborhood level. Individuals in rural areas experienced greater neighborhood social cohesion and lower levels of neighborhood disorder compared with those in urban areas. Conclusions:  The scales are appropriate for measuring neighborhood characteristics experienced by adolescents across urban and rural Scotland, and can be used in future studies of neighborhoods and health. However, trade-offs between neighborhood sample size and reliability must be considered.Publisher PDFPeer reviewe

Prospective memory functioning among ecstasy/polydrug users: evidence from the Cambridge Prospective Memory Test (CAMPROMPT)

Rationale: Prospective memory (PM) deficits in recreational drug users have been documented in recent years. However, the assessment of PM has largely been restricted to self-reported measures that fail to capture the distinction between event-based and time-based PM. The aim of the present study is to address this limitation. Objectives: Extending our previous research, we augmented the range laboratory measures of PM by employing the CAMPROMPT test battery to investigate the impact of illicit drug use on prospective remembering in a sample of cannabis only, ecstasy/polydrug and non-users of illicit drugs, separating event and time-based PM performance. We also administered measures of executive function and retrospective memory in order to establish whether ecstasy/polydrug deficits in PM were mediated by group differences in these processes. Results: Ecstasy/polydrug users performed significantly worse on both event and time-based prospective memory tasks in comparison to both cannabis only and non-user groups. Furthermore, it was found that across the whole sample, better retrospective memory and executive functioning was associated with superior PM performance. Nevertheless, this association did not mediate the drug-related effects that were observed. Consistent with our previous study, recreational use of cocaine was linked to PM deficits. Conclusions: PM deficits have again been found among ecstasy/polydrug users, which appear to be unrelated to group differences in executive function and retrospective memory. However, the possibility that these are attributable to cocaine use cannot be excluded

Scoring method of a Situational Judgment Test:influence on internal consistency reliability, adverse impact and correlation with personality?

textabstractSituational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used

The host metabolite D-serine contributes to bacterial niche specificity through gene selection

Escherichia coli comprise a diverse array of both commensals and niche-specific pathotypes. The ability to cause disease results from both carriage of specific virulence factors and regulatory control of these via environmental stimuli. Moreover, host metabolites further refine the response of bacteria to their environment and can dramatically affect the outcome of the host–pathogen interaction. Here, we demonstrate that the host metabolite, D-serine, selectively affects gene expression in E. coli O157:H7. Transcriptomic profiling showed exposure to D-serine results in activation of the SOS response and suppresses expression of the Type 3 Secretion System (T3SS) used to attach to host cells. We also show that concurrent carriage of both the D-serine tolerance locus (dsdCXA) and the locus of enterocyte effacement pathogenicity island encoding a T3SS is extremely rare, a genotype that we attribute to an ‘evolutionary incompatibility’ between the two loci. This study demonstrates the importance of co-operation between both core and pathogenic genetic elements in defining niche specificity