Perceived causes of differential attainment in UK postgraduate medical training: a national qualitative study

Objectives: Explore trainee doctors’ experiences of postgraduate training and perceptions of fairness in relation to ethnicity and country of primary medical qualification. Design: Qualitative semistructured focus group and interview study. Setting: Postgraduate training in England (London, Yorkshire and Humber, Kent Surrey and Sussex) and Wales. Participants: 137 participants (96 trainees, 41 trainers) were purposively sampled from a framework comprising: doctors from all stages of training in general practice, medicine, obstetrics and gynaecology, psychiatry, radiology, surgery or foundation, in 4 geographical areas, from white and black and minority ethnic (BME) backgrounds, who qualified in the UK and abroad. Results: Most trainees described difficult experiences, but BME UK graduates (UKGs) and international medical graduates (IMGs) could face additional difficulties that affected their learning and performance. Relationships with senior doctors were crucial to learning but bias was perceived to make these relationships more problematic for BME UKGs and IMGs. IMGs also had to deal with cultural differences and lack of trust from seniors, often looking to IMG peers for support instead. Workplace-based assessment and recruitment were considered vulnerable to bias whereas examinations were typically considered more rigorous. In a system where success in recruitment and assessments determines where in the country you can get a job, and where work–life balance is often poor, UK BME and international graduates in our sample were more likely to face separation from family and support outside of work, and reported more stress, anxiety or burnout that hindered their learning and performance. A culture in which difficulties are a sign of weakness made seeking support and additional training stigmatising. Conclusions: BME UKGs and IMGs can face additional difficulties in training which may impede learning and performance. Non-stigmatising interventions should focus on trainee–trainer relationships at work and organisational changes to improve trainees’ ability to seek social support outside work

“You can’t be a person and a doctor”. The work-life balance of doctors in training: a qualitative study

Objectives Investigate the work–life balance of doctors in training in the UK from the perspectives of trainers and trainees. Design Qualitative semistructured focus groups and interviews with trainees and trainers. Setting Postgraduate medical training in London, Yorkshire and Humber, Kent, Surrey and Sussex, and Wales during the junior doctor contract dispute at the end of 2015. Part of a larger General Medical Council study about the fairness of postgraduate medical training. Participants 96 trainees and 41 trainers. Trainees comprised UK graduates and International Medical Graduates, across all stages of training in 6 specialties (General Practice, Medicine, Obstetrics and Gynaecology, Psychiatry, Radiology, Surgery) and Foundation. Results Postgraduate training was characterised by work–life imbalance. Long hours at work were typically supplemented with revision and completion of the e-portfolio. Trainees regularly moved workplaces which could disrupt their personal lives and sometimes led to separation from friends and family. This made it challenging to cope with personal pressures, the stresses of which could then impinge on learning and training, while also leaving trainees with a lack of social support outside work to buffer against the considerable stresses of training. Low morale and harm to well-being resulted in some trainees feeling dehumanised. Work–life imbalance was particularly severe for those with children and especially women who faced a lack of less-than-full-time positions and discriminatory attitudes. Female trainees frequently talked about having to choose a specialty they felt was more conducive to a work–life balance such as General Practice. The proposed junior doctor contract was felt to exacerbate existing problems. Conclusions A lack of work–life balance in postgraduate medical training negatively impacted on trainees' learning and well-being. Women with children were particularly affected, suggesting this group would benefit the greatest from changes to improve the work–life balance of trainees

Academic support for the Assessment and Appraisal Workstream of Health Education England’s review of the ARCP

1. Executive summary 1.1 Background This report was prepared for Health Education England (HEE) by the Research Department of Medical Education at UCL Medical School. Its purpose is to inform the recommendations of HEE’s Annual Review of Competency Progression (ARCP) Review Assessment and Appraisal Workstream in relation to a number of aspects of the ARCP process. This report aims to answer the following questions set by the Assessment & Appraisal Workstream: How can the summative components of the ARCP process be standardised, and what measures can be taken to ensure that the process is robust, reliable and valid? How can the appraisal elements of the ARCP process be standardised, ensuring that appropriate formative feedback is provided to all trainees? How can ARCP processes be aligned to GMC revalidation requirements? What understanding do trainees and supervisors have of the educational principles that underpin the summative and formative elements of ARCPs? In terms of the summative elements of the ARCP we considered: the criteria against which panels make decisions, which evidence panels use to make decisions, the ability of the ARCP to distinguish between different levels of trainee performance and to identify patient safety concerns, the reliability and fairness of the ARCP, and the impact of the ARCP on trainee learning. In terms of the appraisal elements of the ARCP we focused on the feedback given to trainees by the panel, and how trainees are prepared for the ARCP panel. We also considered the relationship between the ARCP and medical revalidation, and trainee and supervisor perceptions of the ARCP. 1.2 Methods We started with a scoping review of the peer-reviewed and grey literature on the ARCP to identify evidence regarding the validity of the summative and appraisal aspects of ARCP, the ARCP and revalidation, and trainee and supervisor perspectives on the ARCP. We then reviewed the official national guidance relating to ARCPs, and some other local or specialty-specific guidance that we were aware of or had found via non-systematic Google searches. We then drew together the literature findings and guidance with evidence from the wider medical education and mainstream education literature, to provide suggestions for increasing the validity, reliability, standardisation, and robustness of the ARCP as a tool to assess and drive trainee progress, to revalidate trainees, and to improve trainee and supervisor understanding of the ARCP. 1.3 Results The overarching suggestion from this report is that the ARCP panel is recognised as a high-stakes assessment that is likely to have a significant impact on patient care. The ARCP panel should therefore be subject to the same scrutiny and psychometric considerations as other high-stakes assessments, such as medical school finals and postgraduate examinations. 5 Report findings and suggestions are summarised under headings relating to the research questions. Major findings and suggestions are presented first, followed by a summary of the more detailed findings and suggestions, as appropriate. 1.3.1 How can the summative components of the ARCP process be standardised, and what measures can be taken to ensure that the process is robust, reliable and valid? 1.3.1.1 Major findings and suggestions Major finding: The guidance lacks consistency and detail about how panels should make decisions. This is a threat to the standardisation, reliability, and validity of the ARCP. Major suggestion: The guidance to provide more detailed and practicable information about the information panels should use to make decisions, and how panels should go about making decisions in practice (including that panel decisions should be made without the trainee present). Decision-aids to be developed that are applicable nationally, and consistent in quality across specialties. Major finding: There are concerns that ARCPs only identify very poorly performing trainees and fail to identify other trainees with specific or less serious performance issues. This may result partly from the ‘failure to fail’ phenomenon. Major suggestion: Minimise failure to fail by having panels provide constructive feedback for all trainees post-decision; by providing time for panels to discuss trainee performance; by having panels consider multiple sources of evidence not just the Educational Supervisor’s Report; by improving support for trainees who are failed; and by introducing lower-stakes formative pre-ARCP reviews. Major finding: There is a lack of empirical research and published evaluation of the ARCP, and ARCP data provided for research has sometimes been of poor quality. This makes it difficult to assess the quality of the ARCP as an assessment. Major suggestion: Collect and provide good quality data for rigorous research and evaluation. 1.3.1.2 Summary of more detailed findings and suggestions Finding: The guidance is unclear about the basis upon which panels should make decisions, leading to inconsistency and threatening validity. Suggestion: In order to develop decision-aids that are applicable nationally, comparable across specialties, and that help panels make valid and robust decisions, the following needs to be clarified in the guidance: Expectations around trainee progression and performance. How panels should weight different pieces of information (research is needed to discover what panels are doing currently, and the impact that weighting can have on outcomes). How panels should take into account any contextual and environmental factors that affected a trainee’s performance, whether positively or negatively. The purpose of the additional information sought and the processes to be followed when a trainee is anticipated to receive an unsatisfactory outcome; and how the additional information should be used by panels to improve the validity of their decision-making. Expectation that assessments submitted in the portfolio are sampled across the curriculum and assessors. 6 Requirement that Educational Supervisors be absent when panels are making decisions. If a trainee is present at panel, this will not influence panel decision-making (i.e. decisions to be made in absentia). Consider having all trainees submit written evidence stating whether they support or disagree with the Educational Supervisor’s report to lessen the conflict between the Educational Supervisor’s dual roles as mentor and assessor. Suggestion: To aid the development of decision-aids, it may be helpful to map all UK local and specialty-specific ARCP guidance to draw out the similarities and differences in the competencies required and the information provided about how panels should make judgements. Following this, a Delphi process (or similar) could be performed to achieve consensus about how panels should make decisions in practice. This could be done by specialty or by families of specialities. Suggestion: Newly-developed decision aids to be compared and reviewed to ensure consistency across specialties and locations, and then piloted to ensure they are practicable. Finding: There is a lack of evidence about whether the ARCP is able to reliably distinguish between satisfactory and unsatisfactory performance, or between different levels of unsatisfactory performance. Trainees and trainers are sceptical about the ARCP’s ability to identify anything other than extreme poor performance, feeling it is not able to identify patient safety concerns but may be overly harsh to trainees with protected characteristics. Medical education research from outside the UK demonstrates reliable judgements about a trainee’s competence can be made on a basis of a several workplace based assessments sampled across curriculum areas and assessors. Suggestion: Collect and provide good quality data for rigorous research and evaluation regarding the validity and reliability of the ARCP. Ensure research is published. Suggestion: Put in place training and support for supervisors and panels to fail trainees if necessary. This includes: normalising ‘remedial’ training by providing constructive feedback for all trainees (including stretching feedback for trainees performing well); better information for supervisors, panels, and trainees about support provided to trainees receiving unsatisfactory outcomes, especially for those receiving an Outcome 4; providing sufficient time for panels to discuss each trainee’s performance; ensuring panels consider multiple sources of evidence not just the Educational Supervisor’s Report; having formative relatively low-stakes pre-ARCPs for all trainees and/or using other tools to help the early identification of problems. Finding: Panels make more reliable judgements than individuals, but steps need to be taken to guard against problems that can arise from group decision-making. Suggestion: Panels to agree on their roles and their expectations of trainees at the start (potentially including clearer role expectations and Terms of Reference for panel members); proformas to aid structured decision-making; sufficient time for panel members to discuss each trainee, with panel chairs regularly summarising and ensuring panel members contribute and share as much information as possible. More panel members are recommended (with the proviso that all should be fully engaged) with three being a minimum. Finding: On average IMGs, male doctors, older doctors, and doctors from black and minority ethnic (BME) backgrounds are more likely to have an unsatisfactory ARCP outcome. It is unclear whether 7 this reflects panel decision-making or other factors. Unconscious bias training may not be sufficient to combat the potential for unfair bias. Suggestion: Panels to explicitly state their commitment to ensuring decision-making is fair during the ARCP. Panel membership to be monitored, efforts made to ensure diversity, and monitoring undertaken to check for any unfair bias. Panels to have greater consideration of the impact of training environments or other external factors on trainee progression and performance. Finding: Many trainees feel demotivated by the fact that it is impossible to achieve an ARCP outcome higher than ‘Satisfactory’ and see it as discouraging excellence. Suggestion: Consideration of how the ARCP process can recognise excellence. Constructive feedback for all trainees, including ‘stretching’ feedback for those performing well. 1.3.2 How can the appraisal elements of the ARCP process be standardised, ensuring that appropriate formative feedback is provided to all trainees? 1.3.2.1 Major findings and suggestions Major finding: At present, only trainees expected to receive an unsatisfactory outcome are required to talk to the panel to discuss their performance, although the limited literature suggests in practice most trainees receive some kind of feedback from the panel, and many receive this in person. There is little guidance about the format in which feedback should be provided or what feedback should be about (e.g. past performance or future performance). There appears to be considerable variation between specialties and geographic locations, which is likely to hinder the validity of the ARCP. Major suggestion: Ensure all trainees receive constructive feedback to improve their learning and performance, including trainees who are progressing satisfactorily. Provide guidance to standardise the way in which trainees receive feedback across specialties and locations. Ensure trainees and panels know panel decision-making will not be influenced by a trainee’s attendance. Major finding: There is no national guidance relating to preparing trainees for ARCPs. In practice, it seems different LETB’s/Deaneries provide different types of pre-ARCP meetings for different groups of trainees. This lack of standardisation is likely to hinder the validity of the ARCP. Major suggestion: Ensure all trainees have a pre-ARCP meeting with their Educational Supervisor and another person, possibly an ARCP panel member, to check progress and provide feedback. Provide guidance to standardise the pre-ARCP meeting process. 1.3.2.2 Summary of detailed findings and suggestions Finding: All trainees are likely to benefit from feedback. Research on appraisal suggests the benefits depend on appraiser and appraisee factors, but it is uncertain how relevant this research is to panels providing feedback. ARCP prioritises summative assessment in the form of the Educational Supervisor Report and assessments of trainees by seniors, but peer- and self-assessment is important for learning. Suggestion: All trainees to receive constructive feedback post-ARCP panel, including stretching feedback for those performing well. Standardised guidance on feedback to be developed and evaluated. Panel members to be trained to provide feedback. Panels to consider peer and self-assessment as well as assessor by seniors. 8 Finding: Neither the Gold Guide nor the Foundation Programme Guidance describe in any detail how the trainee should be prepared for the ARCP. The literature provides various ways to prepare trainees for the panel, such as ensuring trainee portfolios meet up-to-date curriculum requirements and having interim/pre-ARCP panels. Suggestion: Formative interim/pre-ARCPs for all trainees, which focus on providing constructive feedback and feedforward, and on identifying any problems a trainee is having, including any contextual or environmental factors affecting their progression. A degree of externality, possibly from a panel representative, is likely to be beneficial. Suggestion: Explore the potential for developing a standardised tool (adaptable to local contexts), to help trainees and supervisors track achievement over the course of the year and map these to curricular requirements. Finding: Lack of standardised support for trainees receiving an Outcome 4, despite many of these doctors continuing to practice medicine. Potentially risky for patient safety. Also likely to increase failure to fail. Suggestion: Greater educational and career support for trainees receiving an Outcome 4. Trainees, panels, and supervisors to have more information about the support for trainees who are failed in order to combat failure to fail. 1.3.3 How can ARCP processes be aligned to GMC revalidation requirements? 1.3.3.1 Major findings and suggestions Major finding: Lack of clarity about how revalidation decisions and progression decisions relate to one another. Lack of consistency between revalidation for trainees and non-trainees (e.g. consultants, staff grade doctors). The former is based on a summative ARCP and progression against a curriculum, the latter is based on multiple formative appraisals and considered against Good Medical Practice. Major suggestion: Clarification in the guidance as to how the processes for determining progression and revalidation relate to one another. Trainees could receive feedback from the ARCP panel that is aligned with the major domains of Good Medical Practice, to help align revalidation during and after training. 1.3.4 What understanding do trainees and supervisors have of the educational principles that underpin the summative and formative elements of ARCPs? 1.3.4.1 Major findings and suggestions Major finding: The limited literature suggests many trainees feel the an annual review is valuable in principle, but have serious criticisms of the ARCP, perceiving that it does not provide meaningful feedback, that it can discourage excellence, and that it is not sensitive enough to pick up anything but very poor performance. Major suggestion: Communicate to trainees how the ARCP review aims to: ensure all trainees receive constructive feedback in preparation for the ARCP and after the ARCP, including feedback to stretch trainees who are already progressing satisfactorily to encourage excellence, and by finding ways for process to recognise excellence; and by providing 9 interim/pre-ARCPs and developing tools to help trainees and supervisors track a trainee’s progression and identify problems early. improve the reliability and validity of the ARCP (and thereby contribute to patient safety) by standardising panel decision-making across specialties, grades, and locations; by increasing the panel’s consideration of environmental and contextual impacts on a trainee’s progression; by reducing the impact of a single Educational Supervisor’s report on outcomes; by ensuring panels make decisions without the Educational Supervisor or the trainee present; by ensuring rigorous and transparent evaluation of the ARCP. 1.4 Conclusions We found relatively little published research on the ARCP, and much of the evidence was small-scale. This reflects the fact that, despite its high-stakes nature, the ARCP panel is not officially an assessment and does not receive the same scrutiny as other high-stakes assessments in medical education. Our primary suggestion therefore that is the ARCP panel is officially recognised as a high-stakes assessment likely to have a significant impact on patient care and subject to the same scrutiny as other high-stakes assessments. The lack of standardisation in ARCP processes is concerning, and we have suggested much greater clarity in the guidance in a number of areas to address this. To prevent the proliferation of local guidance and the concomitant threat to validity, we have suggested guidance is standardised and then piloted to determine feasibility in practice. We have also suggested that the relationship between the ARCP and revalidation, and the appraisal elements of the ARCP, are both more clearly defined and communicated. The ARCP is generally perceived negatively by trainees, who feel it does not provide meaningful feedback, that excellence is not rewarded, and only the poorest performance is identified. We have suggested a number of changes designed to improve the validity of the ARCP by making it more meaningful. Communicating with trainees about the changes being made to the ARCP and the rationale for those changes is likely to be crucial to restore confidence in the assessment. Finally, we firmly believe in the principle that ‘assessment is not easy to develop and is only as good as the time and energy put into it’ (1) (p.707). We recognise that investment in the ARCP is particularly challenging in the current circumstances, but we believe it is worthwhile. As Eva et al (2) point out: ‘It seems antithetical to the very reasoning behind assessment (the protection of patients) to suggest that we should not think about how to improve current assessment practices, not only in terms of their role in gatekeeping but also in terms of their opportunities for shaping further learning’ (p.907). We suggest investment in undertaking proper and continual evaluation of the process and outcomes of the ARCP, including any changes made, is essential to ensure the validity, reliability, robustness, and defensibility of the ARCP and its role in postgraduate training

Organisational perspectives on addressing differential attainment in postgraduate medical education: a qualitative study in the United Kingdom

OBJECTIVES: To explore how representatives from organisations with responsibility for doctors in training perceive risks to the educational progression of UK medical graduates from black and minority ethnic groups (BME UKGs), and graduates of non-UK medical schools (international medical graduates, IMGs). To identify the barriers to and facilitators of change. DESIGN: Qualitative semi-structured individual and group interview study. SETTING: Postgraduate medical education in the United Kingdom. Participants Individuals with roles in examinations and/or curriculum design from UK Medical Royal Colleges. Employees of NHS Employers. RESULTS: Representatives from 11 medical Royal Colleges (n=29) and NHS Employers (n=2) took part (55% medically qualified, 61% male, 71% white British/Irish, 23% Asian/Asian British, 6% missing ethnicity). Risks were perceived as significant, although more so for IMGs than BME UKGs. Participants based significance ratings on evidence obtained largely through personal experience. A lack of evidence lead to downgrading of significance. Participants were pessimistic about effecting change, two main barriers being sensitivities around race, and the isolation of interventions. Participants felt organisations should acknowledge problems, but felt concerned about being transparent without a solution; and talking about race with trainees was felt to be difficult. Participants mentioned 63 schemes aiming to address differential attainment, but these were typically local or specialty-specific, were not aimed at BME UKGs, and were largely unevaluated. Participants felt national change was needed, but only felt empowered to effect change locally or within their specialty. CONCLUSIONS: Representatives from organisations responsible for training doctors perceived the risks faced by BME UKGs and IMGs as significant but difficult to change. Strategies to help organisations address these risks include: increased openness to discussing race (including ethnic differences in attainment among UKGs); better sharing of information and resources nationally to empower organisations to effect change locally and within specialties; and evaluation of evidence-based interventions

Target profiling of an antimetastatic RAPTA agent by chemical proteomics: relevance to the mode of action.

The clinical development of anticancer metallodrugs is often hindered by the elusive nature of their molecular targets. To identify the molecular targets of an antimetastatic ruthenium organometallic complex based on 1,3,5-triaza-7-phosphaadamantane (RAPTA), we employed a chemical proteomic approach. The approach combines the design of an affinity probe featuring the pharmacophore with mass-spectrometry-based analysis of interacting proteins found in cancer cell lysates. The comparison of data sets obtained for cell lysates from cancer cells before and after treatment with a competitive binder suggests that RAPTA interacts with a number of cancer-related proteins, which may be responsible for the antiangiogenic and antimetastatic activity of RAPTA complexes. Notably, the proteins identified include the cytokines midkine, pleiotrophin and fibroblast growth factor-binding protein 3. We also detected guanine nucleotide-binding protein-like 3 and FAM32A, which is in line with the hypothesis that the antiproliferative activity of RAPTA compounds is due to induction of a G2/M arrest and histone proteins identified earlier as potential targets

Thiazolidine derivatives as potent and selective inhibitors of the PIM kinase family

The PIM family of serine/threonine kinases have become an attractive target for anti-cancer drug development, particularly for certain hematological malignancies. Here, we describe the discovery of a series of inhibitors of the PIM kinase family using a high throughput screening strategy. Through a combination of molecular modeling and optimization studies, the intrinsic potencies and molecular properties of this series of compounds was significantly improved. An excellent pan-PIM isoform inhibition profile was observed across the series, while optimized examples show good selectivity overother kinases. Two PIM-expressing leukemic cancer cell lines, MV4-11 and K562, were employed to evaluate the in vitro anti-proliferative effects of selected inhibitors. Encouraging activities were observed for many examples, with the best example (44) giving an IC50 of 0.75 μM against the K562 cell line. These data provide a promising starting point for further development of this series as a new cancer therapy through PIM kinase inhibitio

A chemical biology toolbox to study protein methyltransferases and epigenetic signaling

Protein methyltransferases (PMTs) comprise a major class of epigenetic regulatory enzymes with therapeutic relevance. Here we present a collection of chemical probes and associated reagents and data to elucidate the function of human and murine PMTs in cellular studies. Our collection provides inhibitors and antagonists that together modulate most of the key regulatory methylation marks on histones H3 and H4, providing an important resource for modulating cellular epigenomes. We describe a comprehensive and comparative characterization of the probe collection with respect to their potency, selectivity, and mode of inhibition. We demonstrate the utility of this collection in CD4+ T cell differentiation assays revealing the potential of individual probes to alter multiple T cell subpopulations which may have implications for T cell-mediated processes such as inflammation and immuno-oncology. In particular, we demonstrate a role for DOT1L in limiting Th1 cell differentiation and maintaining lineage integrity. This chemical probe collection and associated data form a resource for the study of methylation-mediated signaling in epigenetics, inflammation and beyond

Rapid Covalent-Probe Discovery by Electrophile-Fragment Screening

Covalent probes can display unmatched potency, selectivity, and duration of action; however, their discovery is challenging. In principle, fragments that can irreversibly bind their target can overcome the low affinity that limits reversible fragment screening, but such electrophilic fragments were considered nonselective and were rarely screened. We hypothesized that mild electrophiles might overcome the selectivity challenge and constructed a library of 993 mildly electrophilic fragments. We characterized this library by a new high-throughput thiol-reactivity assay and screened them against 10 cysteine-containing proteins. Highly reactive and promiscuous fragments were rare and could be easily eliminated. In contrast, we found hits for most targets. Combining our approach with high-throughput crystallography allowed rapid progression to potent and selective probes for two enzymes, the deubiquitinase OTUB2 and the pyrophosphatase NUDT7. No inhibitors were previously known for either. This study highlights the potential of electrophile-fragment screening as a practical and efficient tool for covalent-ligand discovery