Assessing accuracy of ChatGPT in response to questions from day to day pharmaceutical care in hospitals

Background: The advent of Large Language Models (LLMs) such as ChatGPT introduces opportunities within the medical field. Nonetheless, use of LLM poses a risk when healthcare practitioners and patients present clinical questions to these programs without a comprehensive understanding of its suitability for clinical contexts. Objective: The objective of this study was to assess ChatGPT's ability to generate appropriate responses to clinical questions that hospital pharmacists could encounter during routine patient care. Methods: Thirty questions from 10 different domains within clinical pharmacy were collected during routine care. Questions were presented to ChatGPT in a standardized format, including patients' age, sex, drug name, dose, and indication. Subsequently, relevant information regarding specific cases were provided, and the prompt was concluded with the query “what would a hospital pharmacist do?”. The impact on accuracy was assessed for each domain by modifying personification to “what would you do?”, presenting the question in Dutch, and regenerating the primary question. All responses were independently evaluated by two senior hospital pharmacists, focusing on the availability of an advice, accuracy and concordance. Results: In 77% of questions, ChatGPT provided an advice in response to the question. For these responses, accuracy and concordance were determined. Accuracy was correct and complete for 26% of responses, correct but incomplete for 22% of responses, partially correct and partially incorrect for 30% of responses and completely incorrect for 22% of responses. The reproducibility was poor, with merely 10% of responses remaining consistent upon regeneration of the primary question. Conclusions: While concordance of responses was excellent, the accuracy and reproducibility were poor. With the described method, ChatGPT should not be used to address questions encountered by hospital pharmacists during their shifts. However, it is important to acknowledge the limitations of our methodology, including potential biases, which may have influenced the findings

Patient Preparation with Esomeprazole Is Comparable to Ranitidine in Meckel Diverticulum Scintigraphy

To localize ectopic gastric mucosa in patients with unexplained gastrointestinal bleeding and diagnose a Meckel diverticulum, 99mTc-pertechnetate imaging is the standard procedure. H2 inhibitor pretreatment enhances the sensitivity of the scan by reducing washout of 99mTc activity from the intestinal lumen. We aim to provide evidence of the effectiveness of the proton pump inhibitor esomeprazole as an ideal substitute for ranitidine. Methods: The scan quality for 142 patients who underwent a Meckel scan during a period of 10 y was evaluated. The patients were pretreated with ranitidine orally or intravenously before a switch to a proton pump inhibitor after ranitidine was no longer available. Good scan quality was characterized by the absence of 99mTc-pertechnetate activity in the gastrointestinal lumen. The effectiveness of esomeprazole to diminish 99mTc-pertechnetate release was compared with the standard treatment using ranitidine. Results: Pretreatment with intravenous esomeprazole resulted in 48% of scans with no 99mTc-pertechnetate release, 17% with release either in the intestine or in the duodenum, and 35% with 99mTc-pertechnetate activity both in the intestine and in the duodenum. Evaluation of scans obtained after oral ranitidine and intravenous ranitidine showed absence of activity in both intestine and duodenum in 16% and 23% of the cases, respectively. The indicated time to administer esomeprazole before starting the scan procedure was 30 min, but a delay of 15 min did not negatively influence the scan quality. Conclusion: This study confirms that esomeprazole, 40 mg, when administered intravenously 30 min before a Meckel scan, enhances the scan quality comparably to ranitidine. This procedure can be incorporated into protocols