Assessment, surgeon, and society

An increasing public demand to monitor and assure the quality of care provided by physicians and surgeons has been accompanied by a deepening appreciation within the profession of the demands of self-regulation and the need for accountability. To respond to these developments, the public and the profession have turned increasingly to assessment, both to establish initial competence and to ensure that it is maintained throughout a career. Fortunately, this comes at a time when there have been significant advances in the breadth and quality of the assessment tools available. This article provides an overview of the drivers of change in assessment which includes the educational outcomes movement, the development of technology, and advances in assessment. It then outlines the factors that are important in selecting assessment devices as well as a system for classifying the methods that are available. Finally, the drivers of change have spawned a number of trends in the assessment of competence as a surgeon. Three of them are of particular note, simulation, workplace-based assessment, and the assessment of new competences, and each is reviewed with a focus on its potential

Peer assessment of competence

Objective This instalment in the series on professional assessment summarises how peers are used in the evaluation process and whether their judgements are reliable and valid. Method The nature of the judgements peers can make, the aspects of competence they can assess and the factors limiting the quality of the results are described with reference to the literature. The steps in implementation are also provided. Results Peers are asked to make judgements about structured tasks or to provide their global impressions of colleagues. Judgements are gathered on whether certain actions were performed, the quality of those actions and ⁄ or their suitability for a particular purpose. Peers are used to assess virtually all aspects of professional competence, including technical and nontechnical aspects of proficiency. Factors influencing the quality of those assessments are reliability, relationships, stakes and equivalence. Conclusion Given the broad range of ways peer evaluators can be used and the sizeable number of competencies they can be asked to judge, generalisations are difficult to derive and this form of assessment can be good or bad depending on how it is carried out

Virtual patients in health professions education

La práctica clínica de hoy con hospitalizaciones cortas, riesgos de responsabilidad legal, exigentes normas de acreditación y la menor disponibilidad de docentes, han llevado a la búsqueda de nuevas opciones para la formación de los estudiantes en las profesiones de la salud. Los pacientes virtuales (PV) son programas de computador que simulan a un paciente real y están diseñados para la formación y evaluación del razonamiento clínico. Ofrecen un medio seguro para el aprendizaje de nuestros estudiantes. Se utilizan en evaluación sumativa y se requiere de 6-9 casos para que la evaluación tenga validez. El diseño del PV determina el tipo de evaluación que estamos realizando. Los pacientes electrónicos virtuales son complejos de diseñar lo que conlleva a un alto costo para su desarrollo. A pesar de lo anterior es una herramienta ideal para utilizarse en países en desarrollo y su diseño se ajusta a las necesidades particulares de cada país o institución. Presentamos dos experiencias exitosas en la aplicación de esta tecnología. Otra opción es repotenciar un PV que ya está terminado, haciendo los ajustes necesarios para la nueva situación donde se va a aplicar. Describimos el proceso que es más rápido y menos dispendioso que construir un PV de novo. Existen un banco de la Comunidad Europea, eVIP, donde se encuentran aproximadamente 340 casos que se pueden repotenciar de acuerdo a las necesidades de la institución o país. En resumen, se trata de una excelente herramienta para promover razonamiento clínico accesible a todos.201-209Hospital practice today is characterized by hospitalizations, legal liabilities, strict norms of accreditation, and faculty at short hand. This situation has led to the search of new options for the education of our students in health professions. Electronic Virtual Patients (VP) are computer programs that simulate a real patient, which are designed for the formation and assessment of clinical reasoning and knowledge, depending on their design. They offer a secure scenario for the education of our students. They can be used for assessment but they require 6-9 cases to have content validity. The VP design determines the kind of evaluation to be applied. VPs are very complex to design and they entail a high development cost. Regardless of this situation, they are an ideal tool to be used in developing countries, responding to the socioeconomic situation of the institution or country. We present two successful experiences in two different continents. Another option is to refurbish the existing VP to local needs. We describe the process, which is faster and more efficient than building the case again. In the European Union we find the eVIP program, which has a repertory on its website with 340 cases that can be refurbished by anyone according to the institution or local requirements. In summary, VPs are an excellent resource for clinical reasoning, available to everybody

Assessing clinical reasoning skills using Script Concordance Test (SCT) and extended matching questions (EMQs): A pilot for urology trainees

Introduction: Clinical reasoning skill is the core of medical competence. Commonly used assessment methods for medical competence have limited ability to evaluate critical thinking and reasoning skills. Script Concordance Test (SCT) and Extended Matching Questions(EMQs) are the evolving tests which are considered to be valid and reliable tools for assessing clinical reasoning and judgment. We performed this pilot study to determine whether SCT and EMQs can differentiate clinical reasoning ability among urology residents, interns and medical students.Methods: This was a cross-sectional study in which an examination with 48 SCT-based items on eleven clinical scenarios and four themed EMQs with 21 items were administered to a total of 27 learners at three differing levels of experience i.e. 9 urology residents, 6 interns and 12 fifth year medical students. A non-probability convenience sampling was done. The SCTs and EMQs were developed from clinical situations representative of urological practice by 5 content experts (urologists) and assessed by a medical education expert. Learners\u27 responses were scored using the standard and the graduated key. A one way analysis of variance (ANOVA) was conducted to compare the mean scores across the level of experience. A p-value of \u3c 0.05 was considered statistically significant. Test reliability was estimated by Cronbach α. A focused group discussion with candidates was done to assess their perception of test.Results: Both SCT and EMQs successfully differentiated residents from interns and students. Statistically significant difference in mean score was found for both SCT and EMQs among the 3 groups using both the standard and the graduated key. The mean scores were higher for all groups as measured by the graduated key compared to the standard key. The internal consistency (Cronbach\u27s α) was 0.53 and 0.6 for EMQs and SCT, respectively. Majority of the participants were satisfied with regard to time, environment, instructions provided and the content covered and nearly all felt that the test helped them in thinking process particularly clinical reasoning.Conclusions: Our data suggest that both SCT and EMQs are capable of discriminating between learners according to their clinicalexperience in urology. As there is a wide acceptability by all candidates, these tests could be used to assess and enhance clinical reasoningskills. More research is needed to prove validity of these tests

Interprofessional communication in a sociohierarchical culture: development of the TRI-O guide

Objectives: Interprofessional education (IPE) and collaborative practice are essential for patient safety. Effective teamwork starting with partnership-based communications should be introduced early in the educational process. Many societies in the world hold socio-hierarchical culture with a wide power distance, which makes collaboration among health professionals challenging. Since an appropriate communication framework for this context is not yet available, this study filled that gap by developing a guide for interprofessional communication, which is best suited to the socio-hierarchical and socio-cultural contexts. Materials and methods: The draft of the guide was constructed based on previous studies of communication in health care in a socio-hierarchical context, referred to international IPE literature, and refined by focus group discussions among various health professionals. Nominal group technique, also comments from national and international experts of communication skills in health care, was used to validate the guide. A pilot study with a pre–posttest design was conducted with 53 first- and 107 fourth-year undergraduate medical, nursing, and health nutrition students. Results: We developed the “TRI-O” guide of interprofessional communication skills, emphasizing “open for collaboration, open for information, open for discussion”, and found that the application of the guide during training was feasible and positively influenced students’ perceptions. Conclusion: The findings suggest that the TRI-O guide is beneficial to help students initiate partnership-based communication and mutual collaboration among health professionals in the socio-hierarchical and socio-cultural context

Learning to mark: a qualitative study of the experiences and concerns of medical markers

BACKGROUND: Although there is published research on the methods markers use in marking various types of assessment, there is relatively little information on the processes markers use in approaching a marking exercise. This qualitative paper describes the preparation and experiences of general practice (GP) teachers who undertake marking a written assessment in an undergraduate medical course. METHODS: Semi-structured interviews were conducted with seven of the 16 GP tutors on an undergraduate course. The purposive sample comprised two new markers, two who had marked for a couple of years and three experienced markers. Each respondent was interviewed twice, once following a formative assessment of a written case study, and again after a summative assessment. All interviews were audio-taped and analysed for emerging themes. A respondent validation exercise was conducted with all 16 GP tutors. RESULTS: Markers had internal concerns about their ability to mark fairly and made considerable efforts to calibrate their marking. They needed guidance and coaching when marking for the first time and adopted a variety of marking styles, reaching a decision through a number of routes. Dealing with pass/fail borderline scripts and the consequences of the mark on the student were particular concerns. Even experienced markers felt the need to calibrate their marks both internally and externally CONCLUSION: Previous experience of marking appears to improve markers' confidence and is a factor in determining the role which markers adopt. Confidence can be improved by giving clear instructions, along with examples of marking. The authors propose that one method of providing this support and coaching could be by a process of peer review of a selection of papers prior to the main marking. New markers in particular would benefit from further guidance, however they are influenced by others early on in their marking career and course organisers should be mindful of this when arranging double marking

Elective Cancer Surgery in COVID-19-Free Surgical Pathways During the SARS-CoV-2 Pandemic: An International, Multicenter, Comparative Cohort Study.

PURPOSE: As cancer surgery restarts after the first COVID-19 wave, health care providers urgently require data to determine where elective surgery is best performed. This study aimed to determine whether COVID-19-free surgical pathways were associated with lower postoperative pulmonary complication rates compared with hospitals with no defined pathway. PATIENTS AND METHODS: This international, multicenter cohort study included patients who underwent elective surgery for 10 solid cancer types without preoperative suspicion of SARS-CoV-2. Participating hospitals included patients from local emergence of SARS-CoV-2 until April 19, 2020. At the time of surgery, hospitals were defined as having a COVID-19-free surgical pathway (complete segregation of the operating theater, critical care, and inpatient ward areas) or no defined pathway (incomplete or no segregation, areas shared with patients with COVID-19). The primary outcome was 30-day postoperative pulmonary complications (pneumonia, acute respiratory distress syndrome, unexpected ventilation). RESULTS: Of 9,171 patients from 447 hospitals in 55 countries, 2,481 were operated on in COVID-19-free surgical pathways. Patients who underwent surgery within COVID-19-free surgical pathways were younger with fewer comorbidities than those in hospitals with no defined pathway but with similar proportions of major surgery. After adjustment, pulmonary complication rates were lower with COVID-19-free surgical pathways (2.2% v 4.9%; adjusted odds ratio [aOR], 0.62; 95% CI, 0.44 to 0.86). This was consistent in sensitivity analyses for low-risk patients (American Society of Anesthesiologists grade 1/2), propensity score-matched models, and patients with negative SARS-CoV-2 preoperative tests. The postoperative SARS-CoV-2 infection rate was also lower in COVID-19-free surgical pathways (2.1% v 3.6%; aOR, 0.53; 95% CI, 0.36 to 0.76). CONCLUSION: Within available resources, dedicated COVID-19-free surgical pathways should be established to provide safe elective cancer surgery during current and before future SARS-CoV-2 outbreaks

Elective cancer surgery in COVID-19-free surgical pathways during the SARS-CoV-2 pandemic: An international, multicenter, comparative cohort study

PURPOSE As cancer surgery restarts after the first COVID-19 wave, health care providers urgently require data to determine where elective surgery is best performed. This study aimed to determine whether COVID-19–free surgical pathways were associated with lower postoperative pulmonary complication rates compared with hospitals with no defined pathway. PATIENTS AND METHODS This international, multicenter cohort study included patients who underwent elective surgery for 10 solid cancer types without preoperative suspicion of SARS-CoV-2. Participating hospitals included patients from local emergence of SARS-CoV-2 until April 19, 2020. At the time of surgery, hospitals were defined as having a COVID-19–free surgical pathway (complete segregation of the operating theater, critical care, and inpatient ward areas) or no defined pathway (incomplete or no segregation, areas shared with patients with COVID-19). The primary outcome was 30-day postoperative pulmonary complications (pneumonia, acute respiratory distress syndrome, unexpected ventilation). RESULTS Of 9,171 patients from 447 hospitals in 55 countries, 2,481 were operated on in COVID-19–free surgical pathways. Patients who underwent surgery within COVID-19–free surgical pathways were younger with fewer comorbidities than those in hospitals with no defined pathway but with similar proportions of major surgery. After adjustment, pulmonary complication rates were lower with COVID-19–free surgical pathways (2.2% v 4.9%; adjusted odds ratio [aOR], 0.62; 95% CI, 0.44 to 0.86). This was consistent in sensitivity analyses for low-risk patients (American Society of Anesthesiologists grade 1/2), propensity score–matched models, and patients with negative SARS-CoV-2 preoperative tests. The postoperative SARS-CoV-2 infection rate was also lower in COVID-19–free surgical pathways (2.1% v 3.6%; aOR, 0.53; 95% CI, 0.36 to 0.76). CONCLUSION Within available resources, dedicated COVID-19–free surgical pathways should be established to provide safe elective cancer surgery during current and before future SARS-CoV-2 outbreaks

Outcomes from elective colorectal cancer surgery during the SARS-CoV-2 pandemic

This study aimed to describe the change in surgical practice and the impact of SARS-CoV-2 on mortality after surgical resection of colorectal cancer during the initial phases of the SARS-CoV-2 pandemic