Extracting detailed oncologic history and treatment plan from medical oncology notes with large language models

Both medical care and observational studies in oncology require a thorough understanding of a patient's disease progression and treatment history, often elaborately documented in clinical notes. Despite their vital role, no current oncology information representation and annotation schema fully encapsulates the diversity of information recorded within these notes. Although large language models (LLMs) have recently exhibited impressive performance on various medical natural language processing tasks, due to the current lack of comprehensively annotated oncology datasets, an extensive evaluation of LLMs in extracting and reasoning with the complex rhetoric in oncology notes remains understudied. We developed a detailed schema for annotating textual oncology information, encompassing patient characteristics, tumor characteristics, tests, treatments, and temporality. Using a corpus of 10 de-identified breast cancer progress notes at University of California, San Francisco, we applied this schema to assess the abilities of three recently-released LLMs (GPT-4, GPT-3.5-turbo, and FLAN-UL2) to perform zero-shot extraction of detailed oncological history from two narrative sections of clinical progress notes. Our team annotated 2750 entities, 2874 modifiers, and 1623 relationships. The GPT-4 model exhibited overall best performance, with an average BLEU score of 0.69, an average ROUGE score of 0.72, and an average accuracy of 67% on complex tasks (expert manual evaluation). Notably, it was proficient in tumor characteristic and medication extraction, and demonstrated superior performance in inferring symptoms due to cancer and considerations of future medications. The analysis demonstrates that GPT-4 is potentially already usable to extract important facts from cancer progress notes needed for clinical research, complex population management, and documenting quality patient care.Comment: Source code available at: https://github.com/MadhumitaSushil/OncLLMExtractio

Knowledge Organization Systems (KOS) in the Semantic Web: A Multi-Dimensional Review

Since the Simple Knowledge Organization System (SKOS) specification and its SKOS eXtension for Labels (SKOS-XL) became formal W3C recommendations in 2009 a significant number of conventional knowledge organization systems (KOS) (including thesauri, classification schemes, name authorities, and lists of codes and terms, produced before the arrival of the ontology-wave) have made their journeys to join the Semantic Web mainstream. This paper uses "LOD KOS" as an umbrella term to refer to all of the value vocabularies and lightweight ontologies within the Semantic Web framework. The paper provides an overview of what the LOD KOS movement has brought to various communities and users. These are not limited to the colonies of the value vocabulary constructors and providers, nor the catalogers and indexers who have a long history of applying the vocabularies to their products. The LOD dataset producers and LOD service providers, the information architects and interface designers, and researchers in sciences and humanities, are also direct beneficiaries of LOD KOS. The paper examines a set of the collected cases (experimental or in real applications) and aims to find the usages of LOD KOS in order to share the practices and ideas among communities and users. Through the viewpoints of a number of different user groups, the functions of LOD KOS are examined from multiple dimensions. This paper focuses on the LOD dataset producers, vocabulary producers, and researchers (as end-users of KOS).Comment: 31 pages, 12 figures, accepted paper in International Journal on Digital Librarie

Haematological malignancy: are we measuring what is important to patients? A systematic review of quality of life instruments

© 2018 The Authors. This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.The wide range of health‐related quality‐of‐life (HRQoL) instruments used in haematology makes it challenging for haematologists and other care team members in practice to select, use and understand the scoring system and finally interpret the results. The main objectives of this study were to: (a) provide a comprehensive list of quality‐of‐life issues important to patients suffering from haematological malignancies, identified through the literature; (b) provide a list of health‐related quality‐of‐life (HRQoL) instruments used in haematological malignancies in both daily clinical practice and research; and (c) evaluate the relevance and comprehensibility of the identified instruments in haematological malignancies. Systematic literature review of two databases, followed by addition of articles by manual searching, was carried out. The articles focusing on the primary studies, which have used semi‐structured/structured interviews or surveys to identify issues important to HM patients, and other studies describing the results of testing measurement properties, such as reliability, validity and responsiveness of the instruments currently used to evaluate the HRQoL in different HMs, were included. Fifty‐seven studies reported development and validation of 30 HRQoL instruments, which have been used in haematology. Twenty‐four studies were identified using qualitative methods to report HRQoL issues and symptoms from a patient's perspective. No identified instrument captured all the issues identified from the qualitative studies. None of the instruments reviewed appeared to have been developed for use in clinical practice and specifically for patients with HM, except MyPOS. Furthermore, measurement properties were established, largely, in clinical trial scenarios. There is a need for development of a new HRQoL instrument entirely based on involvement of patients with haematological malignancies.Peer reviewe

Automatic symptoms identification from a massive volume of unstructured medical consultations using deep neural and BERT models

Automatic symptom identification plays a crucial role in assisting doctors during the diagnosis process in Telemedicine. In general, physicians spend considerable time on clinical documentation and symptom identification, which is unfeasible due to their full schedule. With text-based consultation services in telemedicine, the identification of symptoms from a user’s consultation is a sophisticated process and time-consuming. Moreover, at Altibbi, which is an Arabic telemedicine platform and the context of this work, users consult doctors and describe their conditions in different Arabic dialects which makes the problem more complex and challenging. Therefore, in this work, an advanced deep learning approach is developed consultations with multi-dialects. The approach is formulated as a multi-label multi-class classification using features extracted based on AraBERT and fine-tuned on the bidirectional long short-term memory (BiLSTM) network. The Fine-tuning of BiLSTM relies on features engineered based on different variants of the bidirectional encoder representations from transformers (BERT). Evaluating the models based on precision, recall, and a customized hit rate showed a successful identification of symptoms from Arabic texts with promising accuracy. Hence, this paves the way toward deploying an automated symptom identification model in production at Altibbi which can help general practitioners in telemedicine in providing more efficient and accurate consultations

Automated data analysis of unstructured grey literature in health research: A mapping review

\ua9 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd. The amount of grey literature and ‘softer’ intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews. We searched six databases to cover both health- and computer-science literature. After deduplication, 10% of the search results were screened by two reviewers, the remainder was single-screened up to an estimated 95% sensitivity; screening was stopped early after screening an additional 1000 results with no new includes. All full texts were retrieved, screened, and extracted by a single reviewer and 10% were checked in duplicate. We included 84 papers covering automation for health-related social media, internet fora, news, patents, government agencies and charities, or trial registers. From each paper, we extracted data about important functionalities for users of the tool or method; information about the level of support and reliability; and about practical challenges and research gaps. Poor availability of code, data, and usable tools leads to low transparency regarding performance and duplication of work. Financial implications, scalability, integration into downstream workflows, and meaningful evaluations should be carefully planned before starting to develop a tool, given the vast amounts of data and opportunities those tools offer to expedite research

Mining Social Media to Understand Consumers' Health Concerns and the Public's Opinion on Controversial Health Topics.

Social media websites are increasingly used by the general public as a venue to express health concerns and discuss controversial medical and public health issues. This information could be utilized for the purposes of public health surveillance as well as solicitation of public opinions. In this thesis, I developed methods to extract health-related information from multiple sources of social media data, and conducted studies to generate insights from the extracted information using text-mining techniques. To understand the availability and characteristics of health-related information in social media, I first identified the users who seek health information online and participate in online health community, and analyzed their motivations and behavior by two case studies of user-created groups on MedHelp and a diabetes online community on Twitter. Through a review of tweets mentioning eye-related medical concepts identified by MetaMap, I diagnosed the common reasons of tweets mislabeled by natural language processing tools tuned for biomedical texts, and trained a classifier to exclude non medically-relevant tweets to increase the precision of the extracted data. Furthermore, I conducted two studies to evaluate the effectiveness of understanding public opinions on controversial medical and public health issues from social media information using text-mining techniques. The first study applied topic modeling and text summarization to automatically distill users' key concerns about the purported link between autism and vaccines. The outputs of two methods cover most of the public concerns of MMR vaccines reported in previous survey studies. In the second study, I estimated the public's view on the ac{ACA} by applying sentiment analysis to four years of Twitter data, and demonstrated that the the rates of positive/negative responses measured by tweet sentiment are in general agreement with the results of Kaiser Family Foundation Poll. Finally, I designed and implemented a system which can automatically collect and analyze online news comments to help researchers, public health workers, and policy makers to better monitor and understand the public's opinion on issues such as controversial health-related topics.PhDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120714/1/owenliu_1.pd

From Text to Knowledge with Graphs: modelling, querying and exploiting textual content

This paper highlights the challenges, current trends, and open issues related to the representation, querying and analytics of content extracted from texts. The internet contains vast text-based information on various subjects, including commercial documents, medical records, scientific experiments, engineering tests, and events that impact urban and natural environments. Extracting knowledge from this text involves understanding the nuances of natural language and accurately representing the content without losing information. This allows knowledge to be accessed, inferred, or discovered. To achieve this, combining results from various fields, such as linguistics, natural language processing, knowledge representation, data storage, querying, and analytics, is necessary. The vision in this paper is that graphs can be a well-suited text content representation once annotated and the right querying and analytics techniques are applied. This paper discusses this hypothesis from the perspective of linguistics, natural language processing, graph models and databases and artificial intelligence provided by the panellists of the DOING session in the MADICS Symposium 2022

What are the views of hospital-based generalist palliative care professionals on what facilitates or hinders collaboration with in-patient specialist palliative care teams?:a systematically constructed narrative synthesis

Background: Hospital-based specialist palliative care services are common, yet existing evidence of inpatient generalist providers’ perceptions of collaborating with hospital-based specialist palliative care teams has never been systematically assessed. Aim: To assess the existing evidence of inpatient generalist palliative care providers’ perceptions of what facilitates or hinders collaboration with hospital-based specialist palliative care teams. Design: Narrative literature synthesis with systematically constructed search. Data sources: PsycINFO, PubMed, Web of Science, Cumulative Index of Nursing and Allied Health Literature and ProQuest Social Services databases were searched up to December 2014. Individual journal, citation and reference searching were also conducted. Papers with the views of generalist inpatient professional caregivers who utilised hospital-based specialist palliative care team services were included in the narrative synthesis. Hawker’s criteria were used to assess the quality of the included studies. Results: Studies included (n = 23) represented a variety of inpatient generalist palliative care professionals’ experiences of collaborating with specialist palliative care. Effective collaboration is experienced by many generalist professionals. Five themes were identified as improving or decreasing effective collaboration: model of care (integrated vs linear), professional onus, expertise and trust, skill building versus deskilling and specialist palliative care operations. Collaboration is fostered when specialist palliative care teams practice proactive communication, role negotiation and shared problem-solving and recognise generalists’ expertise. Conclusion: Fuller integration of specialist palliative care services, timely sharing of information and mutual respect increase generalists’ perceptions of effective collaboration. Further research is needed regarding the experiences of non-physician and non-nursing professionals as their views were either not included or not explicitly reported