Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

Background: Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Methods: Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation. Results: Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels. Conclusions: Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills

The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools

Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training

Narrating unfinished business: adult learners using credit transfer to re-engage with higher education

This paper seeks to advance our understanding of the credit transfer phenomenon in the UK, specifically how students draw on a credit as a form of institutional cultural capital. Drawing on interviews with twenty-six part-time mature learners this paper examines the progressive and retrospective orientations to study that surfaced in students’ accounts of credit transfer and their life-long learning journeys. A common theme of “unfinished business” appeared to dominate these accounts and a narrative-oriented analysis of the findings revealed the role of credit transfer in enabling students to complete varied personal projects or forms of “unfinished business”. The findings of this work suggest that in an increasingly complex higher education market there is a need to understand students’ strategic use of the credit they accumulate. In particular, this paper explores how credit transfer features in the narratives of students’ successful learning journey

Some observations on the factors that influence strategies for educational improvement in post-1992 universities

Along with the rest of the education sector universities have been encouraged to develop strategies for educational improvement. The strategies that have emerged have been heavily influenced by the policies and priorities of the government and government agencies, such as the Quality Assurance Agency and the Higher Education Funding Council, and the targets set as a consequence of these policies and priorities. The government's emphasis has been on such things as fitness for purpose and value for money, whilst at the same time demanding high standards of quality assurance and widening participation. To what extent are the demands for educational improvement and the approach to this taken by the post-1992 university sector justified or supported by the current research in this area? It is the conclusion of this article that there is a paucity of directly relevant research, but that the research that does apply indicates that the sector needs to rethink its approach to educational improvement if it wishes to avoid the impoverishment of the education it provides