Predicting fitness to practise events in international medical graduates who registered as UK doctors via the Professional and Linguistic Assessments Board (PLAB) system: a national cohort study

Background International medical graduates working in the UK are more likely to be censured in relation to fitness to practise compared to home graduates. Performance on the General Medical Council’s (GMC’s) Professional and Linguistic Assessments Board (PLAB) tests and English fluency have previously been shown to predict later educational performance in this group of doctors. It is unknown whether the PLAB system is also a valid predictor of unprofessional behaviour and malpractice. The findings would have implications for regulatory policy. Methods This was an observational study linking data relating to fitness to practise events (referral or censure), PLAB performance, demographic variables and English language competence, as evaluated via the International English Language Test System (IELTS). Data from 27,330 international medical graduates registered with the GMC were analysed, including 210 doctors who had been sanctioned in relation to at least one fitness to practise issue. The main outcome was risk of eventual censure (including a warning). Results The significant univariable educational predictors of eventual censure (versus no censures or referrals) were lower PLAB part 1 (hazard ratio [HR], 0.99; 95% confidence interval, 0.98 to 1.00) and part 2 scores (HR, 0.94; 0.91 to 0.97) at first sitting, multiple attempts at both parts of the PLAB, lower IELTS reading (HR, 0.79; 0.65 to 0.94) and listening scores (HR, 0.76; 0.62 to 0.93) and higher IELTS speaking scores (HR, 1.28; 1.04 to 1.57). Multiple resits at either part of the PLAB and higher IELTS speaking score (HR, 1.49; 1.20 to 1.84) were also independent predictors of censure. We estimated that the proposed limit of four attempts at both parts of the PLAB would reduce the risk in this entire group by only approximately two censures per 5 years in this group of doctors. Conclusions Making the PLAB, or any replacement assessment, more stringent and raising the required standards of English reading and listening may result in fewer fitness to practice events in international medical graduates. However, the number of PLAB resits permitted would have to be further capped to meaningfully impact the risk of sanctions in this group of doctor

The ability of 'non-cognitive' traits to predict undergraduate performance in medical schools : a national linkage study

Abstract Background In addition to the evaluation of educational attainment and intellectual ability there has been interest in the potential to select medical school applicants on non-academic qualities. Consequently, a battery of self-report measures concerned with assessing ‘non-cognitive’ traits was piloted as part of the UK Clinical Aptitude Test (UKCAT) administration to evaluate their potential to be used in selection. Methods The four non-cognitive instruments piloted were: 1) the Libertarian-communitarian scale, (2) The NACE (narcissism, aloofness, confidence and empathy, (3) the MEARS (Managing emotions and resilience scale; self-esteem, optimism, control, self-discipline, emotional-nondefensiveness and faking, and (4) an abridged version of instruments (1) and (2) combined. Non-cognitive scores and sociodemographic characteristics were available for 14,387 applicants. A series of univariable and multivariable analyses were conducted in order to assess the ability of the non-cognitive scores to predict knowledge and skills-based performance, as well as the odds of passing each academic year at first attempt. Non-cognitive scores and medical performance were standardised within cohorts. Results The scores on the non-cognitive scales showed only very small (magnitude of standardised betas< 0.2), though sometimes statistically significant (p < 0.01) univariable associations with subsequent performance on knowledge or skills-based assessments. The only statistically significant association between the non-cognitive scores and the probability of passing an academic year at first attempt was the narcissism score from one the abridged tests (OR 0.84,95% confidence intervals 0.71 to 0.97, p = 0.02). Conclusions Our findings are consistent with previously published research. The tests had a very limited ability to predict undergraduate academic performance, though further research on identifying narcissism in medical students may be warranted. However, the validity of such self-report tools in high-stakes settings may be affected, making such instruments unlikely to add value within the selection process

Comparison of the sensitivity of the UKCAT and A levels to sociodemographic characteristics: a national study

Background: The UK Clinical Aptitude Test (UKCAT) was introduced to facilitate widening participation in medical and dental education in the UK by providing universities with a continuous variable to aid selection; one that might be less sensitive to the sociodemographic background of candidates compared to traditional measures of educational attainment. Initial research suggested that males, candidates from more advantaged socioeconomic backgrounds and those who attended independent or grammar schools performed better on the test. The introduction of the A* grade at A level permits more detailed analysis of the relationship between UKCAT scores, secondary educational attainment and sociodemographic variables. Thus, our aim was to further assess whether the UKCAT is likely to add incremental value over A level (predicted or actual) attainment in the selection process. Methods: Data relating to UKCAT and A level performance from 8,180 candidates applying to medicine in 2009 who had complete information relating to six key sociodemographic variables were analysed. A series of regression analyses were conducted in order to evaluate the ability of sociodemographic status to predict performance on two outcome measures: A level ‘best of three’ tariff score; and the UKCAT scores. Results: In this sample A level attainment was independently and positively predicted by four sociodemographic variables (independent/grammar schooling, White ethnicity, age and professional social class background). These variables also independently and positively predicted UKCAT scores. There was a suggestion that UKCAT scores were less sensitive to educational background compared to A level attainment. In contrast to A level attainment, UKCAT score was independently and positively predicted by having English as a first language and male sex. Conclusions: Our findings are consistent with a previous report; most of the sociodemographic factors that predict A level attainment also predict UKCAT performance. However, compared to A levels, males and those speaking English as a first language perform better on UKCAT. Our findings suggest that UKCAT scores may be more influenced by sex and less sensitive to school type compared to A levels. These factors must be considered by institutions utilising the UKCAT as a component of the medical and dental school selection process

The UK medical education database (UKMED) what is it?:Why and how might you use it?

The development of UKMED has not been directly funded by but has received support from the General Medical Council and Medical Schools Council as well as staff time from all participating organisations.Peer reviewedPublisher PD

Free text adversity statements as part of a contextualised admissions process:a qualitative analysis

Abstract Background Medical schools globally are encouraged to widen access and participation for students from less privileged backgrounds. Many strategies have been implemented to address this inequality, but much still needs to be done to ensure fair access for all. In the literature, adverse circumstances include financial issues, poor educational experience and lack of professional-status parents. In order to take account of adverse circumstances faced by applicants, The University of Dundee School of Medicine offers applicants the opportunity to report circumstances which may have resulted in disadvantage. Applicants do this by completing a free text statement, known as an ‘adversity statement’, in addition to the other application information. This study analysed adversity statements submitted by applicants during two admissions cycles. Analysis of content and theme was done to identify the information applicants wished to be taken into consideration, and what range of adverse circumstances individuals reported. Methods This study used a qualitative approach with thematic analysis to categorise the adversity statements. The data was initially analysed to create a coding framework which was then applied to the whole data set. Each coded segment was then analysed for heterogeneity and homogeneity, segments merged into generated themes, or to create sub-themes. Results The data set comprised a total of 384 adversity statements. These showed a wide range of detail involving family, personal health, education and living circumstances. Some circumstances, such as geographical location, have been identified and explored in previous research, while others, such as long term health conditions, have had less attention in the literature. The degree of impact, the length of statement and degree of detail, demonstrated wide variation between submissions. Conclusions This study adds to the debate on best practice in contextual admissions and raises awareness of the range of circumstances and impact applicants wish to be considered. The themes which emerged from the data included family, school, personal health, and geographical location issues. Descriptions of the degree of impact that an adverse circumstance had on educational or other attainment was found to vary substantially from statements indicating minor, impact through to circumstances stated as causing major impact

Feasibility of an automated interview grounded in multiple mini interview (MMI) methodology for selection into the health professions: an international multimethod evaluation.

OBJECTIVES: Global, COVID-driven restrictions around face-to-face interviews for healthcare student selection have forced admission staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in multiple mini-interview (MMI) methodology. This study aimed to explore test-retest reliability, acceptability and usability of the system. DESIGN, SETTING AND PARTICIPANTS: Multimethod feasibility study in Physician Associate programmes from two UK and one US university during 2019-2020. PRIMARY, SECONDARY OUTCOMES: Feasibility measures (test-retest reliability, acceptability and usability) were assessed using intraclass correlation (ICC), descriptive statistics, thematic and content analysis. METHODS: Volunteers took (T1), then repeated (T2), the automated MMI, with a 7-day interval (±2) then completed an evaluation questionnaire. Admission staff participated in focus group discussions. RESULTS: Sixty-two students and seven admission staff participated; 34 students and 4 staff from UK and 28 students and 3 staff from US universities. Good-excellent test-retest reliability was observed at two sites (US and UK2) with T1 and T2 ICC between 0.65 and 0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005 and individual site (≥0.79 p<0.001). Mean test re-test ICC across all three sites was 0.82 p<0.001 (95% CI 0.7 to 0.9). Admission staff reported potential to reduce resource costs and bias through a more objective screening tool for preselection or to replace some MMI stations in a 'hybrid model'. Maintaining human interaction through 'touch points' was considered essential. Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored. CONCLUSION: These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection

Clinical characteristics of cyclodeviation

PURPOSE: To retrospectively evaluate the incidence of cyclodeviation among patients with diplopia and analyse the causative diseases and clinical manifestations of cyclodeviation. METHODS: The medical records of 266 consecutive patients of 15 years of age or older presenting with diplopia, who had undergone the Lancaster red-green test (LRGT) from January 2001 to December 2002, were retrospectively reviewed. The presence of cyclodeviation on LRGT, predisposing conditions, causative diseases, and clinical manifestations of cyclotropia were analysed. Cyclodeviation on the LRGT were compared with those from the Maddox double-rod test (MDRT) and fundus photography. RESULTS: A total of 63 (24%) out of 266 patients exhibited cyclodeviation on LRGT. Eight out of 63 patients with cyclodeviation on the LRGT complained of torsional diplopia. Superior oblique palsy (SOP) was the most common causative disease (42 patients), followed by skew deviation (six) and thyroid orbitopathy (three). Excyclodeviation was found in 57 patients and incyclodeviation in four patients on the LRGT. The spontaneous recovery rate was 83% in patients of vascular origin and 17% of traumatic origin. Cyclodeviation with the MDRT and fundus photography showed good correlation with those obtained from the LRGT. There was no association of the amount of cyclotropia with the presence of torsional diplopia as well as with its recovery. CONCLUSION: In spite of the rare complaint of torsional diplopia, 24% of the patients with diplopia showed cyclodeviation on the LRGT. SOP was the most common causative disease. Most of the patients with cyclodeviation of a vascular origin showed spontaneous improvement

The impact of large scale licensing examinations in highly developed countries: a systematic review

BACKGROUND: To investigate the existing evidence base for the validity of large-scale licensing examinations including their impact. METHODS: Systematic review against a validity framework exploring: Embase (Ovid Medline); Medline (EBSCO); PubMed; Wiley Online; ScienceDirect; and PsychINFO from 2005 to April 2015. All papers were included when they discussed national or large regional (State level) examinations for clinical professionals, linked to examinations in early careers or near the point of graduation, and where success was required to subsequently be able to practice. Using a standardized data extraction form, two independent reviewers extracted study characteristics, with the rest of the team resolving any disagreement. A validity framework was used as developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education to evaluate each paper’s evidence to support or refute the validity of national licensing examinations. RESULTS: 24 published articles provided evidence of validity across the five domains of the validity framework. Most papers (n = 22) provided evidence of national licensing examinations relationships to other variables and their consequential validity. Overall there was evidence that those who do well on earlier or on subsequent examinations also do well on national testing. There is a correlation between NLE performance and some patient outcomes and rates of complaints, but no causal evidence has been established. CONCLUSIONS: The debate around licensure examinations is strong on opinion but weak on validity evidence. This is especially true of the wider claims that licensure examinations improve patient safety and practitioner competence

Is social stress in the first half of life detrimental to later physical and mental health in both men and women?

This study examined gender differences in the associations between affection- and status-related stressors encountered in the first half of life and physical and mental health problems later on. Based on the theory of Social Production Functions (SPF) two hypotheses have been formulated, which were tested in a representative sample of 446 men and 514 women (aged 40–79). Main outcome measures were number of chronic somatic diseases and level of psychological distress. As expected, regression analyses showed no gender differences in the associations between affection-related stressors and physical and mental health problems later on. In contrast, but as also expected, status-related stressors encountered in the first half of life were associated with later physical and mental health for men only. It is concluded that the gender differences in the associations between earlier social stressors and later health problems may be more complex than the common assumption that men are only affected by status stress and women only by affection stress. This study contributes to the knowledge on gender differences concerning the link between social stress and health, and it indicates that social experiences encountered earlier in life are of importance for being healthy and happy in later life