AI chatbots not yet ready for clinical use

As large language models (LLMs) expand and become more advanced, so do the natural language processing capabilities of conversational AI, or “chatbots”. OpenAI's recent release, ChatGPT, uses a transformer-based model to enable human-like text generation and question-answering on general domain knowledge, while a healthcare-specific Large Language Model (LLM) such as GatorTron has focused on the real-world healthcare domain knowledge. As LLMs advance to achieve near human-level performances on medical question and answering benchmarks, it is probable that Conversational AI will soon be developed for use in healthcare. In this article we discuss the potential and compare the performance of two different approaches to generative pretrained transformers—ChatGPT, the most widely used general conversational LLM, and Foresight, a GPT (generative pretrained transformer) based model focused on modelling patients and disorders. The comparison is conducted on the task of forecasting relevant diagnoses based on clinical vignettes. We also discuss important considerations and limitations of transformer-based chatbots for clinical use

Dietary nitrate prevents progression of carotid subclinical atherosclerosis through blood pressure-independent mechanisms in patients with or at risk of type 2 diabetes mellitus

Aims To test if 6 months' intervention with dietary nitrate and spironolactone could affect carotid subclinical atherosclerosis and stiffness, respectively, vs. placebo/doxazosin, to control for blood pressure (BP). Methods A subgroup of participants in our double-blind, randomized-controlled, factorial VaSera trial had carotid imaging. Patients with hypertension and with/at risk of type 2 diabetes were randomized to active nitrate-containing beetroot juice or placebo nitrate-depleted juice, and spironolactone or doxazosin. Vascular ultrasound for carotid diameter (CD, mm) and intima–media thickness (CIMT, mm) was performed at baseline, 3- and 6-months. Carotid local stiffness (CS, m/s) was estimated from aortic pulse pressure (Arteriograph) and carotid lumen area. Data were analysed by modified intention to treat and using mixed-model effect, adjusted for confounders. Results In total, 93 subjects had a baseline evaluation and 86% had follow-up data. No statistical interactions occurred between the juice and drug arms and BP was similar between the juices and between the drugs. Nitrate-containing vs. placebo juice significantly lowered CIMT (−0.06 [95% confidence interval −0.12, −0.01], P = .034), an overall difference of ~8% relative to baseline; but had no effect on CD or CS. Doxazosin appeared to reduce CS from baseline (−0.34 [−0.62, −0.06]) however, no difference was detected vs. spironolactone (−0.15 [−0.46, 0.16]). No differences were detected between spironolactone or doxazosin on CIMT and CD. Conclusions Our results show that 6 months' intervention with dietary nitrate influences vascular remodelling, but not carotid stiffness or diameter. Neither spironolactone nor doxazosin had a BP-independent effect on carotid structure and function

Artificial intelligence (AI) for neurologists:do digital neurones dream of electric sheep?

Artificial intelligence (AI) is routinely mentioned in journals and newspapers, and non-technical outsiders may have difficulty in distinguishing hyperbole from reality. We present a practical guide to help non-technical neurologists to understand healthcare AI. AI is being used to support clinical decisions in treating neurological disorders. We introduce basic concepts of AI, such as machine learning and natural language processing, and explain how AI is being used in healthcare, giving examples its benefits and challenges. We also cover how AI performance is measured, and its regulatory aspects in healthcare. An important theme is that AI is a general-purpose technology like medical statistics, with broad utility applicable in various scenarios, such that niche approaches are outpaced by approaches that are broadly applicable in many disease areas and specialties. By understanding AI basics and its potential applications, neurologists can make informed decisions when evaluating AI used in their clinical practice. This article was written by four humans, with generative AI helping with formatting and image generation.</p

Machine learning enabled multi-Trust audit of stroke co-morbidities using Natural language Processing:Machine learning enabled audit of stroke

Background and purpose: With the increasing adoption of electronic records in the health system, machine learning-enabled techniques offer the opportunity for greater computer-assisted curation of these data for audit and research purposes. In this project, we evaluate the consistency of traditional curation methods used in routine clinical practice against a new machine learning-enabled tool, MedCAT, for the extraction of the stroke comorbidities recorded within the UK's Sentinel Stroke National Audit Programme (SSNAP) initiative. Methods: A total of 2327 stroke admission episodes from three different National Health Service (NHS) hospitals, between January 2019 and April 2020, were included in this evaluation. In addition, current clinical curation methods (SSNAP) and the machine learning-enabled method (MedCAT) were compared against a subsample of 200 admission episodes manually reviewed by our study team. Performance metrics of sensitivity, specificity, precision, negative predictive value, and F1 scores are reported. Results: The reporting of stroke comorbidities with current clinical curation methods is good for atrial fibrillation, hypertension, and diabetes mellitus, but poor for congestive cardiac failure. The machine learning-enabled method, MedCAT, achieved better performances across all four assessed comorbidities compared with current clinical methods, predominantly driven by higher sensitivity and F1 scores. Conclusions: We have shown machine learning-enabled data collection can support existing clinical and service initiatives, with the potential to improve the quality and speed of data extraction from existing clinical repositories. The scalability and flexibility of these new machine-learning tools, therefore, present an opportunity to revolutionize audit and research methods.</p

Table1_AI chatbots not yet ready for clinical use.docx

As large language models (LLMs) expand and become more advanced, so do the natural language processing capabilities of conversational AI, or “chatbots”. OpenAI's recent release, ChatGPT, uses a transformer-based model to enable human-like text generation and question-answering on general domain knowledge, while a healthcare-specific Large Language Model (LLM) such as GatorTron has focused on the real-world healthcare domain knowledge. As LLMs advance to achieve near human-level performances on medical question and answering benchmarks, it is probable that Conversational AI will soon be developed for use in healthcare. In this article we discuss the potential and compare the performance of two different approaches to generative pretrained transformers—ChatGPT, the most widely used general conversational LLM, and Foresight, a GPT (generative pretrained transformer) based model focused on modelling patients and disorders. The comparison is conducted on the task of forecasting relevant diagnoses based on clinical vignettes. We also discuss important considerations and limitations of transformer-based chatbots for clinical use.</p

Correction: Hu, W., et al. Targeting Dopamine Receptor D2 by Imipridone Suppresses Uterine Serous Cancer Malignant Phenotype. Cancers 2020, 12, 2436

In the original article, there was a mistake in Figure 2B as published [...