Advantages and pitfalls in utilizing artificial intelligence for crafting medical examinations: a medical education pilot study with GPT-4

Abstract Background The task of writing multiple choice question examinations for medical students is complex, timely and requires significant efforts from clinical staff and faculty. Applying artificial intelligence algorithms in this field of medical education may be advisable. Methods During March to April 2023, we utilized GPT-4, an OpenAI application, to write a 210 multi choice questions-MCQs examination based on an existing exam template and thoroughly investigated the output by specialist physicians who were blinded to the source of the questions. Algorithm mistakes and inaccuracies, as identified by specialists were classified as stemming from age, gender or geographical insensitivities. Results After inputting a detailed prompt, GPT-4 produced the test rapidly and effectively. Only 1 question (0.5%) was defined as false; 15% of questions necessitated revisions. Errors in the AI-generated questions included: the use of outdated or inaccurate terminology, age-sensitive inaccuracies, gender-sensitive inaccuracies, and geographically sensitive inaccuracies. Questions that were disqualified due to flawed methodology basis included elimination-based questions and questions that did not include elements of integrating knowledge with clinical reasoning. Conclusion GPT-4 can be used as an adjunctive tool in creating multi-choice question medical examinations yet rigorous inspection by specialist physicians remains pivotal

In-hospital and 6-month outcomes in patients with COVID-19 supported with extracorporeal membrane oxygenation (EuroECMO-COVID): a multicentre, prospective observational study

Background: Extracorporeal membrane oxygenation (ECMO) has been widely used in patients with COVID-19, but uncertainty remains about the determinants of in-hospital mortality and data on post-discharge outcomes are scarce. The aims of this study were to investigate the variables associated with in-hospital outcomes in patients who received ECMO during the first wave of COVID-19 and to describe the status of patients 6 months after ECMO initiation. Methods: EuroECMO-COVID is a prospective, multicentre, observational study developed by the European Extracorporeal Life Support Organization. This study was based on data from patients aged 16 years or older who received ECMO support for refractory COVID-19 during the first wave of the pandemic—from March 1 to Sept 13, 2020—at 133 centres in 21 countries. In-hospital mortality and mortality 6 months after ECMO initiation were the primary outcomes. Mixed-Cox proportional hazards models were used to investigate associations between patient and management-related variables (eg, patient demographics, comorbidities, pre-ECMO status, and ECMO characteristics and complications) and in-hospital deaths. Survival status at 6 months was established through patient contact or institutional charts review. This study is registered with ClinicalTrials.gov, NCT04366921, and is ongoing. Findings: Between March 1 and Sept 13, 2020, 1215 patients (942 [78%] men and 267 [22%] women; median age 53 years [IQR 46–60]) were included in the study. Median ECMO duration was 15 days (IQR 8–27). 602 (50%) of 1215 patients died in hospital, and 852 (74%) patients had at least one complication. Multiorgan failure was the leading cause of death (192 [36%] of 528 patients who died with available data). In mixed-Cox analyses, age of 60 years or older, use of inotropes and vasopressors before ECMO initiation, chronic renal failure, and time from intubation to ECMO initiation of 4 days or more were associated with higher in-hospital mortality. 613 patients did not die in hospital, and 547 (95%) of 577 patients for whom data were available were alive at 6 months. 102 (24%) of 431 patients had returned to full-time work at 6 months, and 57 (13%) of 428 patients had returned to part-time work. At 6 months, respiratory rehabilitation was required in 88 (17%) of 522 patients with available data, and the most common residual symptoms included dyspnoea (185 [35%] of 523 patients) and cardiac (52 [10%] of 514 patients) or neurocognitive (66 [13%] of 512 patients) symptoms. Interpretation: Patient's age, timing of cannulation (<4 days vs ≥4 days from intubation), and use of inotropes and vasopressors are essential factors to consider when analysing the outcomes of patients receiving ECMO for COVID-19. Despite post-discharge survival being favourable, persisting long-term symptoms suggest that dedicated post-ECMO follow-up programmes are required. Funding: None