Exploring objective measures for assessing team performance in healthcare: an interview study.

INTRODUCTION Effective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach. METHODS A total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question. RESULTS The findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures. DISCUSSION The study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research

Effects of interacting with a large language model compared with a human coach on the clinical diagnostic process and outcomes among fourth-year medical students: study protocol for a prospective, randomised experiment using patient vignettes.

INTRODUCTION Versatile large language models (LLMs) have the potential to augment diagnostic decision-making by assisting diagnosticians, thanks to their ability to engage in open-ended, natural conversations and their comprehensive knowledge access. Yet the novelty of LLMs in diagnostic decision-making introduces uncertainties regarding their impact. Clinicians unfamiliar with the use of LLMs in their professional context may rely on general attitudes towards LLMs more broadly, potentially hindering thoughtful use and critical evaluation of their input, leading to either over-reliance and lack of critical thinking or an unwillingness to use LLMs as diagnostic aids. To address these concerns, this study examines the influence on the diagnostic process and outcomes of interacting with an LLM compared with a human coach, and of prior training vs no training for interacting with either of these 'coaches'. Our findings aim to illuminate the potential benefits and risks of employing artificial intelligence (AI) in diagnostic decision-making. METHODS AND ANALYSIS We are conducting a prospective, randomised experiment with N=158 fourth-year medical students from Charité Medical School, Berlin, Germany. Participants are asked to diagnose patient vignettes after being assigned to either a human coach or ChatGPT and after either training or no training (both between-subject factors). We are specifically collecting data on the effects of using either of these 'coaches' and of additional training on information search, number of hypotheses entertained, diagnostic accuracy and confidence. Statistical methods will include linear mixed effects models. Exploratory analyses of the interaction patterns and attitudes towards AI will also generate more generalisable knowledge about the role of AI in medicine. ETHICS AND DISSEMINATION The Bern Cantonal Ethics Committee considered the study exempt from full ethical review (BASEC No: Req-2023-01396). All methods will be conducted in accordance with relevant guidelines and regulations. Participation is voluntary and informed consent will be obtained. Results will be published in peer-reviewed scientific medical journals. Authorship will be determined according to the International Committee of Medical Journal Editors guidelines

Physicians’ attitudes toward, use of, and perceived barriers to clinical guidelines: a survey among Swiss physicians

BACKGROUND: Little is known about the attitudes toward, use of, and perceived barriers to clinical guidelines in Switzerland, a country with no national guideline agency. Moreover, there is no available data on the objective assessment of guideline knowledge in Switzerland. Therefore, we conducted a study at a large university's Department of General Internal Medicine in Switzerland to assess physicians' attitudes toward, use of, perceived barriers to, and knowledge of clinical guidelines. PARTICIPANTS AND METHODS: Ninety-six physicians (residents, n=78, and attendings, n=18) were invited to take part in a survey. Attitudes toward, self-reported use of, and barriers hindering adherence to the clinical guidelines were assessed using established scales and frameworks. Knowledge of the guidelines was objectively tested in a written assessment comprising of 14 multiple-choice and 3 short answer case-based questions. RESULTS: Fifty-five participants completed the survey (residents, n=42, and attendings, n=13; overall response rate 57%). Of these, 50 took part in the knowledge assessment (residents, n=37, and attendings, n=13; overall response rate 52%). Attitudes toward guidelines were favorable. They were considered to be a convenient source of advice (94% agreement), good educational tools (89% agreement), and likely to improve patient quality of care (91% agreement). Self-reported use of guidelines was limited, with only one-third reporting using guidelines often or very often. The main barriers to guideline adherence were identified as lack of guideline awareness and familiarity, applicability of existing guidelines to multimorbid patients, unfavorable guideline factors, and lack of time as well as inertia toward changing previous practice. In the assessment of guideline knowledge, the scores were rather modest (mean ± standard deviation: 60.5%±12.7% correct answers). CONCLUSION: In general, this study found favorable physician attitudes toward clinical guidelines. However, several barriers hindering guideline implementation were identified. The importance of improving guideline implementation was supported by modest results in a guideline knowledge test

Frequency and predictors of unspecific medical diagnoses in the emergency department: a prospective observational study.

BACKGROUND Misdiagnosis is a major public health problem, causing increased morbidity and mortality. In the busy setting of an emergency department (ED) patients are diagnosed under difficult circumstances. As a consequence, the ED diagnosis at hospital admittance may often be a descriptive diagnosis, such as "decreased general condition". Our objective was to determine in how far patients with such an unspecific ED diagnosis differ from patients with a specific ED diagnosis and whether they experience a worse outcome. METHODS We conducted a prospective observational study in Bern university hospital in Switzerland for all adult non-trauma patients admitted to any internal medicine ward from August 15th 2015 to December 7th 2015. Unspecific ED diagnoses were defined through the clinical classification software for ICD-10 by two outcome assessors. As outcome parameters, we assessed in-hospital mortality and length of hospital stay. RESULTS Six hundred eighty six consecutive patients were included. Unspecific diagnoses were identified in 100 (14.6%) of all consultations. Patients receiving an unspecific diagnosis at ED discharge were significantly more often women (56.0% vs. 43.9%, p = 0.024), presented more often with a non-specific complaint (34% vs. 21%, p = 0.004), were less often demonstrating an abnormal heart rate (5.0% vs. 12.5%, p = 0.03), and less often on antibiotics (32.0% vs. 49.0%, p = 0.002). Apart from these, no studied drug intake, laboratory or clinical data including change in diagnosis was associated significantly with an unspecific diagnosis. Unspecific diagnoses were neither associated with in-hospital mortality in multivariable analysis (OR = 1.74, 95% CI: 0.60-5.04; p = 0.305) adjusted for relevant confounders nor with length of hospital stay (GMR = 0.87, 95% CI: 0.23-3.32; p = 0.840). CONCLUSIONS Women and patients with non-specific presenting complaints and no abnormal heart rate are at risk of receiving unspecific ED diagnoses that do not allow for targeted treatment, discharge and prognosis. This study did not find an effect of such diagnoses on length of hospital stay nor in-hospital mortality

Automated identification of diagnostic labelling errors in medicine

Objectives: Identification of diagnostic error is complex and mostly relies on expert ratings, a severely limited procedure. We developed a system that allows to automatically identify diagnostic labelling error from diagnoses coded according to the international classification of diseases (ICD), often available as routine health care data. Methods: The system developed (index test) was validated against rater based classifications taken from three previous studies of diagnostic labeling error (reference standard). The system compares pairs of diagnoses through calculation of their distance within the ICD taxonomy. Calculation is based on four different algorithms. To assess the concordance between index test and reference standard, we calculated the area under the receiver operating characteristics curve (AUROC) and corresponding confidence intervals. Analysis were conducted overall and separately per algorithm and type of available dataset. Results: Diagnoses of 1,127 cases were analyzed. Raters previously classified 24.58% of cases as diagnostic labelling errors (ranging from 12.3 to 87.2% in the three datasets). AUROC ranged between 0.821 and 0.837 overall, depending on the algorithm used to calculate the index test (95% CIs ranging from 0.8 to 0.86). Analyzed per type of dataset separately, the highest AUROC was 0.924 (95% CI 0.887-0.962). Conclusions: The trigger system to automatically identify diagnostic labeling error from routine health care data performs excellent, and is unaffected by the reference standards' limitations. It is however only applicable to cases with pairs of diagnoses, of which one must be more accurate or otherwise superior than the other, reflecting a prevalent definition of a diagnostic labeling error

What factors affect team members' evaluation of collaboration in medical teams?

INTRODUCTION Perceived teamwork quality is associated with numerous work-related outcomes, ranging from team effectiveness to job satisfaction. This study explored what situational and stable factors affect the perceived quality of teamwork during a specific team task: when a medical team comprising a senior (supervisor) and a junior (trainee) physician diagnoses a patient. METHODS During a field study in an emergency department, multisource data describing the patients, the diagnosing physicians, and the context were collected, including physicians' ratings of their teamwork. The relationships between perceived teamwork quality and situational (e.g., workload) and stable (e.g., seniority) factors were estimated in a latent regression model using the structural equation modeling (SEM) approach. RESULTS Across the N = 495 patients included, SEM analyses revealed that the patient-specific case clarity and urgency influenced the perceived teamwork quality positively, whereas the work experience of the supervisor influenced the perceived teamwork quality of both supervisor and trainee negatively, albeit to different degrees. DISCUSSION Our findings shed light on the complex underpinnings of perceived teamwork quality, a performance-relevant factor that may influence work and organizational effectiveness in healthcare settings

Virtual reality simulation to enhance advanced trauma life support trainings - a randomized controlled trial.

BACKGROUND Advanced Trauma Life Support (ATLS) is the gold standard of initial assessment of trauma patients and therefore a widely used training program for medical professionals. Practical application of the knowledge taught can be challenging for medical students and inexperienced clinicians. Simulation-based training, including virtual reality (VR), has proven to be a valuable adjunct to real-world experiences in trauma education. Previous studies have demonstrated the effectiveness of VR simulations for surgical and technical skills training. However, there is limited evidence on VR simulation training specifically for trauma education, particularly within the ATLS curriculum. The purpose of this pilot study is to evaluate the feasibility, effectiveness, and acceptance of using a fully immersive VR trauma simulation to prepare medical students for the ATLS course. METHODS This was a prospective randomised controlled pilot study on a convenience sample of advanced medical students (n = 56; intervention group with adjunct training using a commercially available semi-automated trauma VR simulation, n = 28, vs control group, n = 28) taking part in the ATLS course of the Military Physician Officer School. Feasibility was assessed by evaluating factors related to technical factors of the VR training (e.g. rate of interruptions and premature termination). Objective and subjective effectiveness was assessed using confidence ratings at four pre-specified points in the curriculum, validated surveys, clinical scenario scores, multiple choice knowledge tests, and ATLS final clinical scenario and course pass rates. Acceptance was measured using validated instruments to assess variables of media use (Technology acceptance, usability, presence and immersion, workload, and user satisfaction). RESULTS The feasibility assessment demonstrated that only one premature termination occurred and that all remaining participants in the intervention group correctly stabilised the patient. No significant differences between the two groups in terms of objective effectiveness were observed (p = 0.832 and p = 0.237 for the pretest and final knowledge test, respectively; p = 0.485 for the pass rates for the final clinical scenario on the first attempt; all participants passed the ATLS course). In terms of subjective effectiveness, the authors found significantly improved confidence post-VR intervention (p < .001) in providing emergency care using the ATLS principles. Perceived usefulness in the TEI was stated with a mean of 4 (SD 0.8; range 0-5). Overall acceptance and usability of the VR simulation was rated as positive (System Usability Scale total score mean 79.4 (SD 11.3, range 0-100). CONCLUSIONS The findings of this prospective pilot study indicate the potential of using VR trauma simulations as a feasible and acceptable supplementary tool for the ATLS training course. Where objective effectiveness regarding test and scenario scores remained unchanged, subjective effectiveness demonstrated improvement. Future research should focus on identifying specific scenarios and domains where VR can outperform or enhance traditional learning methods in trauma simulation

a prospective observational study

Introduction Emergency rooms (ERs) generally assign a preliminary diagnosis to patients, who are then hospitalised and may subsequently experience a change in their lead diagnosis (cDx). In ERs, the cDx rate varies from around 15% to more than 50%. Among the most frequent reasons for diagnostic errors are cognitive slips, which mostly result from faulty data synthesis. Furthermore, physicians have been repeatedly found to be poor self-assessors and to be overconfident in the quality of their diagnosis, which limits their ability to improve. Therefore, some of the clinically most relevant research questions concern how diagnostic decisions are made, what determines their quality and what can be done to improve them. Research that addresses these questions is, however, still rare. In particular, field studies that allow for generalising findings from controlled experimental settings are lacking. The ER, with its high throughput and its many simultaneous visits, is perfectly suited for the study of factors contributing to diagnostic error. With this study, we aim to identify factors that allow prediction of an ER's diagnostic performance. Knowledge of these factors as well as of their relative importance allows for the development of organisational, medical and educational strategies to improve the diagnostic performance of ERs. Methods and analysis We will conduct a field study by collecting diagnostic decision data, physician confidence and a number of influencing factors in a real-world setting to model real-world diagnostic decisions and investigate the adequacy, validity and informativeness of physician confidence in these decisions. We will specifically collect data on patient, physician and encounter factors as predictors of the dependent variables. Statistical methods will include analysis of variance and a linear mixed-effects model. Ethics and dissemination The Bern ethics committee approved the study under KEK Number 197/15. Results will be published in peer-reviewed scientific medical journals. Authorship will be determined according to ICMJE guidelines. Trial registration number The study protocol Version 1.0 from 17 May 2015 is registered in the Inselspital Research Database Information System (IRDIS) and with the IRB (‘Kantonale Ethikkomission’) Bern under KEK Number 197/15

High-dose intravenous immunoglobulins: A promising therapeutic approach for idiopathic systemic capillary leak syndrome

The systemic capillary leak syndrome (SCLS), also known as Clarkson’s disease, is a life-threatening disorder of unknown cause. It is characterised by recurrent acute episodes of hypotension, weight gain and generalised oedema with haemoconcentration and hypoproteinaemia caused by paroxysmal capillary hyperpermeability with a shift of plasma fluid from the intravascular to the interstitial space. We report the case of a 40-year-old woman with chronic SCLS treated with high-dose intravenous immunoglobulins, after a prophylactic therapy with theophylline and terbutaline was poorly tolerated and failed to decrease the frequency and severity of the attacks sufficiently