This letter critically examines the recent article by Infante et al. assessing the utility of large language models (LLMs) like GPT-4, Perplexity, and Bard in identifying urgent findings in emergency radiology reports. While acknowledging the potential of LLMs in generating labels for computer vision, concerns are raised about the ethical implications of using patient data without explicit approval, highlighting the necessity of stringent data protection measures under GDPR

Agarwal, Siddharth

Booth, Thomas C

Carpenter, Robin

Modat, Marc

Wei, Yiran

Wood, David

English

King's Research Portal

                       King’s Research Portal  Document VersionPeer reviewed versionLink to publication record in King's Research PortalCitation for published version (APA):Agarwal, S., Wood, D., Carpenter, R., Wei, Y., Modat, M., & Booth, T. C. (in press). Letter to the Editor: Whatare the legal and ethical considerations of submitting radiology reports to ChatGPT? Clinical Radiology.Citing this paperPlease note that where the full-text provided on King's Research Portal is the Author Accepted Manuscript or Post-Print version this maydiffer from the final Published version. If citing, it is advised that you check and use the publisher's definitive version for pagination,volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you areagain advised to check the publisher's website for any subsequent corrections.General rightsCopyright and moral rights for the publications made accessible in the Research Portal are retained by the authors and/or other copyrightowners and it is a condition of accessing publications that users recognize and abide by the legal requirements associated with these rights.•Users may download and print one copy of any publication from the Research Portal for the purpose of private study or research.•You may not further distribute the material or use it for any profit-making activity or commercial gain•You may freely distribute the URL identifying the publication in the Research PortalTake down policyIf you believe that this document breaches copyright please contact librarypure@kcl.ac.uk providing details, and we will remove access tothe work immediately and investigate your claim.Download date: 04. Aug. 2024Clinical Radiology Siddharth Agarwal, David Wood, Robin Carpenter, Yiran Wei, Marc Modat, Thomas CBooth--Manuscript Draft-- Manuscript Number:Full Title: Siddharth Agarwal, David Wood, Robin Carpenter, Yiran Wei, Marc Modat, Thomas CBoothArticle Type: Letter to the EditorCorresponding Author: Thomas C Booth, MA MBBS MRCP FRCR PhDKing's College LondonLondon, UNITED KINGDOMCorresponding Author SecondaryInformation:Corresponding Author's Institution: King's College LondonCorresponding Author's SecondaryInstitution:First Author: Siddharth Agarwal, MBBS, MResFirst Author Secondary Information:Order of Authors: Siddharth Agarwal, MBBS, MResDavid Wood, PhDRobin CarpenterYiran Wei, PhDMarc Modat, PhDThomas C BoothOrder of Authors Secondary Information:Powered by Editorial Manager® and ProduXion Manager® from Aries Systems CorporationLetter to the Editor: What are the legal and ethical considerations of submitting radiology reports to ChatGPT?  Siddharth Agarwal1, David Wood1, Robin Carpenter1, Yiran Wei1, Marc Modat1, Thomas C Booth1,2  1. School of Biomedical Engineering & Imaging Sciences, King’s College London, 9th Floor, Becket House, 1 Lambeth Palace Rd, London, SE1 7EU, UK 2. Department of Neuroradiology, Ruskin Wing, King’s College Hospital NHS Foundation Trust, London SE5 9RS, UK  Correspondence to: Thomas C Booth, thomas.booth@kcl.ac.uk, +44 (0) 203 299 4828  Competing interests All authors declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.  Role of the funding source SA is supported by an Engineering and Physical Sciences Research Council (EPSRC) funded PhD studentship (EP/R513064/1). TB and MM are supported by the Wellcome/EPSRC Centre for Medical Engineering (WT 203148/Z/16/Z), The Royal College of Radiologists (TB) and King’s College Hospital Research and Innovation (TB).  Author contributions SA drafted the letter. All authors contributed to the response and approved the final version.  1 guarantor of integrity of the entire study  TCB 2 study concepts and design    N/A 3 literature research      SA, DW, RC 4 clinical studies     N/A 5 experimental studies / data analysis  N/A 6 statistical analysis     N/A 7 manuscript preparation    SA 8 manuscript editing     SA, DW, RC, YW, MM, TCB Title PageLetter to the Editor: What are the legal and ethical considerations of submitting 1 radiology reports to ChatGPT? 2  3 To the Editor, 4 We read with great interest the article titled ‘Large language models (LLMs) in the evaluation 5 of emergency radiology reports: performance of ChatGPT-4, Perplexity, and Bard’.1 The 6 authors compare three LLMs’ abilities to extract the presence of an urgent finding in 7 emergency radiology reports. The study adds to a growing body of literature regarding how 8 LLMs can perform classification tasks rather than natural language generation, the latter task 9 being the one that they are best suited for. We would question the utility of LLMs to ‘establish 10 a reading priority based on the likely presence of emergency findings’, given that a human 11 radiologist is first required to generate the report from which the finding is extracted, however 12 the task itself can be instrumental in generating large quantities of labels for downstream 13 computer vision research tasks.2–4 14  15 We read with some concern that the authors felt that ethical approval was not required 16 because ‘patient-related information was not collected [and there was a] total absence of 17 patient-identifiable data.’ One should analogously consider if the use of patient imaging for 18 research would require no ethical approval, even when the DICOM headers have been 19 scrubbed and faces blocked out. In England, all research studies involving NHS patient data 20 require Health Research Authority (HRA) approval, even if the data was fully anonymised. A 21 Research Ethics Committee (REC) may share our concerns regarding the submission of 22 patient reports to OpenAI and Google. Once submitted, these reports will be stored on 23 proprietary servers located in the United States. Should a patient wish for data about them to 24 be deleted, neither company would be able to identify that specific patient’s data, nor would 25 they be under any obligation to do so. 26  27 In addition, true anonymisation of patient reports, as opposed to pseudonymisation, may be 28 difficult to achieve for some patients. Reidentification of anonymised patient data can 29 Manuscript (without Author details) Click here to view linked Referencessometimes be achieved by cross referencing the data with other datasets the patient also 30 appears in.5 Controlling the context the data sits in is therefore often important for 31 maintaining privacy. The right to privacy is part of the 1950 European Convention on Human 32 Rights and is a foundation for the General Data Protection Regulation (GDPR).6 GDPR 33 would apply if reports were pseudonymised rather than truly anonymised. 34  35 Data submitted to the non-enterprise version of ChatGPT is then used to train subsequent 36 models and is also shared with third parties.7  Healthcare providers, who are often the data 37 controllers and therefore have responsibility for the data they hold, can come into 38 agreements with artificial intelligence (AI) companies to share data for the purposes of 39 developing AI models. Firstly, this can be legally complex and requires contractual controls 40 in place to control the use of data – but it can be achieved alongside the correct ethical and 41 governance framework. Secondly, at the very least, there would be an expectation for 42 financial compensation - in the UK, the second principle of the “Value Sharing Framework for 43 National Health Service (NHS) data partnerships” states that “the NHS should seek to 44 recover the costs of providing access to data. Failing to recover these costs, including a 45 proportionate share of overheads, takes money away from frontline services”.8 Patients and 46 members of the public in the UK would also likely want the NHS to share in the value 47 created by its data.9 Individual radiologists or researchers are employed by the data 48 controllers and are not the data controllers themselves, therefore have no right to unilaterally 49 share data with commercial organisations. 50  51 The reasons mentioned above may explain why the authors found almost no published 52 studies using these three models in similar radiology research. We would therefore 53 encourage any reader of this study to not share even anonymised patient data to ChatGPT 54 or any online service without ethical approval and explicit permission of the data controller 55 (often the hospital or healthcare provider). A practical alternative involves utilising open-56 source large language models (LLMs) like Llama or its derivatives, which can be set up and 57 run directly on a local computer within a hospital or research organisation's secure 58 network.10 Here, the research community is active. 59  60 From a technical perspective, it seems surprising that the prompt yielded a single-word 61 answer (‘yes’ or ‘no’) from all three models, with no other words in the response. In our own 62 experience of LLMs (locally set up on a secure network), extracting the label from a 63 paragraph of the model’s reasoning or explanation is an extra cleaning step. We would 64 encourage the authors to share if this was done by human interpreters, or automatically with 65 a script. The poor performances of all models could be explained by the quality of the 66 prompt, which itself should be considered a hyperparameter that can be optimised. A large 67 amount of the LLM literature is dedicated to optimal prompt engineering.11 Providing the LLM 68 with positive and negative examples, allowing the model to perform ‘few-shot’ rather than 69 ‘zero-shot’ inference is an example, and techniques such as chain-of-thought prompting and 70 self-consistency are straightforward methods of increasing performance. It would have also 71 been interesting to have seen a comparison with the auto-encoding type transformer 72 architectures, such as Bidirectional Encoder Representations from Transformers (BERT) and 73 its derivative models.12,13 These models are naturally suited for classification tasks and can 74 be fine-tuned easily on a subset of the labels to achieve over 95% balanced accuracy for 75 binary abnormality detection in radiology reports.14,15 76   77 1. Infante A, Gaudino S, Orsini F, et al. Large language models (LLMs) in the evaluation of 78 emergency radiology reports: performance of ChatGPT-4, Perplexity, and Bard. Clinical 79 Radiology 2024;79(2):102–6. https://doi.org/10.1016/J.CRAD.2023.11.011. 80 2. Wood DA, Kafiabadi S, Busaidi AA, et al. Deep learning models for triaging hospital head 81 MRI examinations. Medical Image Analysis 2022;78:102391. 82 https://doi.org/10.1016/J.MEDIA.2022.102391. 83 3. Wood DA, Kafiabadi S, Al Busaidi A, et al. Automated triaging of head MRI examinations 84 using convolutional neural networks. Medical Imaging with Deep Learning. PMLR; 85 2021;813–41. 86 4. Agarwal S, Wood D, Grzeda M, et al. Systematic Review of Artificial Intelligence for 87 Abnormality Detection in High-volume Neuroimaging and Subgroup Meta-analysis for 88 Intracranial Hemorrhage Detection. Clin Neuroradiol 2023;33(4):943–56. 89 https://doi.org/10.1007/s00062-023-01291-1. 90 5. Ghazarian A, Zheng J, Struppa D, Rakovski C. Assessing the Reidentification Risks 91 Posed by Deep Learning Algorithms Applied to ECG Data. IEEE Access 2022;10:68711–92 23. https://doi.org/10.1109/ACCESS.2022.3185615. 93 6. What is GDPR, the EU’s new data protection law? GDPREu 2018. https://gdpr.eu/what-is-94 gdpr/ (accessed March 9, 2024). 95 7. Data usage for consumer services FAQ | OpenAI Help Center n.d. 96 https://help.openai.com/en/articles/7039943-data-usage-for-consumer-services-faq 97 (accessed March 9, 2024). 98 8. Value Sharing Framework for NHS data partnerships. NHS Transformation Directorate 99 n.d. https://transform.england.nhs.uk/key-tools-and-info/centre-improving-data-100 collaboration/value-sharing-framework-for-nhs-data-partnerships/ (accessed March 9, 101 2024). 102 9. Putting Good into Practice: A public dialogue on making public benefit assessments when 103 using health and care data. GOVUK n.d. 104 https://www.gov.uk/government/publications/putting-good-into-practice-a-public-dialogue-105 on-making-public-benefit-assessments-when-using-health-and-care-data (accessed 106 March 9, 2024). 107 10. Touvron H, Martin L, Stone K, et al. Llama 2: Open Foundation and Fine-Tuned Chat 108 Models 2023. https://doi.org/10.48550/arXiv.2307.09288. 109 11. Chen B, Zhang Z, Langrené N, Zhu S. Unleashing the potential of prompt 110 engineering in Large Language Models: a comprehensive review 2023. 111 https://doi.org/10.48550/arXiv.2310.14735. 112 12. Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional 113 Transformers for Language Understanding. In: Burstein J, Doran C, Solorio T, eds. 114 Proceedings of the 2019 Conference of the North American Chapter of the Association for 115 Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short 116 Papers). Minneapolis, Minnesota: Association for Computational Linguistics; 2019;4171–117 86. https://doi.org/10.18653/v1/N19-1423. 118 13. Liu Y, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining 119 Approach 2019. https://doi.org/10.48550/arXiv.1907.11692. 120 14. Wood DA, Kafiabadi S, Al Busaidi A, et al. Deep learning to automate the labelling of 121 head MRI datasets for computer vision applications. European Radiology 122 2022;32(1):725–36. 123 15. Wood DA, Kafiabadi S, Al Busaidi A, et al. Labelling Imaging Datasets on the Basis of 124 Neuroradiology Reports: A Validation Study. Interpretable and Annotation-Efficient 125 Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, 126 Second International Workshop, MIL3ID 2020, and 5th International Workshop, LABELS 127 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, 128 Proceedings. Berlin, Heidelberg: Springer-Verlag; 2020;254–65. 129 https://doi.org/10.1007/978-3-030-61166-8_27. 130  131 Declaration of interests   ☐ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.   ☒ The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:  Siddharth Agarwal reports financial support was provided by Engineering and Physical Sciences Research Council. Marc Modat reports financial support was provided by Wellcome Trust. Thomas C Booth reports financial support was provided by Wellcome Trust. Thomas C Booth reports financial support was provided by Engineering and Physical Sciences Research Council. Thomas C Booth reports financial support was provided by Royal College of Radiologists. Thomas C Booth reports financial support was provided by King’s College Hospital. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.  Declaration of interest form

Letter to the Editor:What are the legal and ethical considerations of submitting radiology reports to ChatGPT?

https://kclpure.kcl.ac.uk/portal/files/279619592/CRAD-S-24-00424.pdf

Letter to the Editor:What are the legal and ethical considerations of submitting radiology reports to ChatGPT?

Abstract

Similar works

Full text

Available Versions

King's Research Portal