How is performance at selection to general practice related to performance at the endpoint of GP training?

Abstract

Background The selection process for entry to speciality training for general practice (GP) in the UK was changed in 2016. Doctors scoring above an agreed threshold in the computer-marked Multi-Specialty Recruitment Assessment (MSRA) were deemed appointable on that score alone and were offered a direct pathway (DP) to training, exempting them from further assessment at the final Selection Centre (SC). The SC was subsequently suspended in response to the COVID-19 pandemic and has yet to be reinstated. We aimed to evaluate the relationship between performance at selection and outcomes of GP training at licensing, to reassess the threshold score in MSRA used to bypass the SC, and to estimate the incremental predictive value of the SC after MSRA. Methods We used a longitudinal design linking selection, licensing and demographic data from doctors applying to enter GP specialty training in 2016. MSRA scores were divided into 12 score bands and SC scores into seven score bands to better identify MSRA or SC scores that corresponded to dffering GP performance on licensing assessments. Multivariable logistic regression models were used to establish the predictive validity of the MSRA scores and score bands for passing or failing the Membership of the Royal College of General Practitioners (MRCGP) licensing assessments including the Applied Knowledge Test (AKT), Clinical Skills Assessment (CSA) or Recorded Consultation Assessment (RCA), Workplace Based Assessment - Annual Review of Competence Progression (WPBA-ARCP), and performance overall. The model adjusted for sex, ethnicity, country of qualification, and declared disability. Receiver Operating Characteristic (ROC) curves of MSRA scores against performance outcomes were constructed to determine the optimal MSRA threshold scores for achieving licensing. Results We included 3338 doctors who entered specialty training for general practice in 2016 of different sex (female 63.81% vs male 36.19%), ethnicity (White British 53.95%, minority ethnic 43.04% or mixed 3.01%), country of qualification (UK 76.76%, non-UK 23.24%), and declared disability (disability declared 11.98%, no disability declared 88.02%). MSRA scores or score bands were highly predictive for all assessments of GP training outcome (AKT, CSA, RCA, and WPBA-ARCP). Lower SC score bands were predictive of lower pass rates on summative assessments and /or ARCP outcomes 2, 3, or 4. Adding SC scores did not change the predictive validity of the MSRA, and therefore the SC did not add further information to MSRA scores. An MSRA threshold of 500 (or, more precisely, 497) was optimal for correctly identifying pass/fail rates on the AKT, RCA, and CSA within the study period, and only standard outcomes on WPBA-ARCP. Thirty-five percent of candidates in the lowest two MSRA Bands (i.e., scores below 420) had at least one developmental outcome (2, 3) or outcome 4. Ethnicity did not reduce the chance of passing GP licensing tests once sex, place of primary medical qualification, declared disability and MSRA scores were taken into account. Conclusion MSRA scores predict licensing outcomes for AKT, CSA, RCA, and WPBA-ARCP within five years of starting training. The optimal MSRA threshold score for predicting an uncomplicated training pathway to licensing was around 500 in this large cohort. The SC added little to the predictive validity of the MSRA. Doctors scoring below this threshold may need additional support during training to maximise their chances of achieving licensing