433 research outputs found

    Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification

    Get PDF
    Recent advances in large language models (LLMs) have shown impressive ability in biomedical question-answering, but have not been adequately investigated for more specific biomedical applications. This study investigates the performance of LLMs such as the ChatGPT family of models (GPT-3.5s, GPT-4) in biomedical tasks beyond question-answering. Because no patient data can be passed to the OpenAI API public interface, we evaluated model performance with over 10000 samples as proxies for two fundamental tasks in the clinical domain - classification and reasoning. The first task is classifying whether statements of clinical and policy recommendations in scientific literature constitute health advice. The second task is causal relation detection from the biomedical literature. We compared LLMs with simpler models, such as bag-of-words (BoW) with logistic regression, and fine-tuned BioBERT models. Despite the excitement around viral ChatGPT, we found that fine-tuning for two fundamental NLP tasks remained the best strategy. The simple BoW model performed on par with the most complex LLM prompting. Prompt engineering required significant investment.Comment: 28 pages, 2 tables and 4 figures. Submitting for revie

    Natural language processing to automatically extract the presence and severity of esophagitis in notes of patients undergoing radiotherapy

    Full text link
    Radiotherapy (RT) toxicities can impair survival and quality-of-life, yet remain under-studied. Real-world evidence holds potential to improve our understanding of toxicities, but toxicity information is often only in clinical notes. We developed natural language processing (NLP) models to identify the presence and severity of esophagitis from notes of patients treated with thoracic RT. We fine-tuned statistical and pre-trained BERT-based models for three esophagitis classification tasks: Task 1) presence of esophagitis, Task 2) severe esophagitis or not, and Task 3) no esophagitis vs. grade 1 vs. grade 2-3. Transferability was tested on 345 notes from patients with esophageal cancer undergoing RT. Fine-tuning PubmedBERT yielded the best performance. The best macro-F1 was 0.92, 0.82, and 0.74 for Task 1, 2, and 3, respectively. Selecting the most informative note sections during fine-tuning improved macro-F1 by over 2% for all tasks. Silver-labeled data improved the macro-F1 by over 3% across all tasks. For the esophageal cancer notes, the best macro-F1 was 0.73, 0.74, and 0.65 for Task 1, 2, and 3, respectively, without additional fine-tuning. To our knowledge, this is the first effort to automatically extract esophagitis toxicity severity according to CTCAE guidelines from clinic notes. The promising performance provides proof-of-concept for NLP-based automated detailed toxicity monitoring in expanded domains.Comment: 17 pages, 6 tables, 1figure, submiting to JCO-CCI for revie

    Large Language Models to Identify Social Determinants of Health in Electronic Health Records

    Full text link
    Social determinants of health (SDoH) have an important impact on patient outcomes but are incompletely collected from the electronic health records (EHR). This study researched the ability of large language models to extract SDoH from free text in EHRs, where they are most commonly documented, and explored the role of synthetic clinical text for improving the extraction of these scarcely documented, yet extremely valuable, clinical data. 800 patient notes were annotated for SDoH categories, and several transformer-based models were evaluated. The study also experimented with synthetic data generation and assessed for algorithmic bias. Our best-performing models were fine-tuned Flan-T5 XL (macro-F1 0.71) for any SDoH, and Flan-T5 XXL (macro-F1 0.70). The benefit of augmenting fine-tuning with synthetic data varied across model architecture and size, with smaller Flan-T5 models (base and large) showing the greatest improvements in performance (delta F1 +0.12 to +0.23). Model performance was similar on the in-hospital system dataset but worse on the MIMIC-III dataset. Our best-performing fine-tuned models outperformed zero- and few-shot performance of ChatGPT-family models for both tasks. These fine-tuned models were less likely than ChatGPT to change their prediction when race/ethnicity and gender descriptors were added to the text, suggesting less algorithmic bias (p<0.05). At the patient-level, our models identified 93.8% of patients with adverse SDoH, while ICD-10 codes captured 2.0%. Our method can effectively extracted SDoH information from clinic notes, performing better compare to GPT zero- and few-shot settings. These models could enhance real-world evidence on SDoH and aid in identifying patients needing social support.Comment: 38 pages, 5 figures, 5 tables in main, submitted for revie

    The impact of responding to patient messages with large language model assistance

    Full text link
    Documentation burden is a major contributor to clinician burnout, which is rising nationally and is an urgent threat to our ability to care for patients. Artificial intelligence (AI) chatbots, such as ChatGPT, could reduce clinician burden by assisting with documentation. Although many hospitals are actively integrating such systems into electronic medical record systems, AI chatbots utility and impact on clinical decision-making have not been studied for this intended use. We are the first to examine the utility of large language models in assisting clinicians draft responses to patient questions. In our two-stage cross-sectional study, 6 oncologists responded to 100 realistic synthetic cancer patient scenarios and portal messages developed to reflect common medical situations, first manually, then with AI assistance. We find AI-assisted responses were longer, less readable, but provided acceptable drafts without edits 58% of time. AI assistance improved efficiency 77% of time, with low harm risk (82% safe). However, 7.7% unedited AI responses could severely harm. In 31% cases, physicians thought AI drafts were human-written. AI assistance led to more patient education recommendations, fewer clinical actions than manual responses. Results show promise for AI to improve clinician efficiency and patient care through assisting documentation, if used judiciously. Monitoring model outputs and human-AI interaction remains crucial for safe implementation.Comment: 4 figures and tables in main, submitted for revie

    Apoptosis at Inflection Point in Liquid Culture of Budding Yeasts

    Get PDF
    Budding yeasts are highly suitable for aging studies, because the number of bud scars (stage) proportionally correlates with age. Its maximum stages are known to reach at 20–30 stages on an isolated agar medium. However, their stage dynamics in a liquid culture is virtually unknown. We investigate the population dynamics by counting scars in each cell. Here one cell division produces one new cell and one bud scar. This simple rule leads to a conservation law: “The total number of bud scars is equal to the total number of cells.” We find a large discrepancy: extremely fewer cells with over 5 scars than expected. Almost all cells with 6 or more scars disappear within a short period of time in the late log phase (corresponds to the inflection point). This discrepancy is confirmed directly by the microscopic observations of broken cells. This finding implies apoptosis in older cells (6 scars or more)

    Aversive Learning in Honeybees Revealed by the Olfactory Conditioning of the Sting Extension Reflex

    Get PDF
    Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory

    Fibrotic Myofibroblasts Manifest Genome-Wide Derangements of Translational Control

    Get PDF
    Background: As a group, fibroproliferative disorders of the lung, liver, kidney, heart, vasculature and integument are common, progressive and refractory to therapy. They can emerge following toxic insults, but are frequently idiopathic. Their enigmatic propensity to resist therapy and progress to organ failure has focused attention on the myofibroblast–the primary effector of the fibroproliferative response. We have recently shown that aberrant beta 1 integrin signaling in fibrotic fibroblasts results in defective PTEN function, unrestrained Akt signaling and subsequent activation of the translation initiation machinery. How this pathological integrin signaling alters the gene expression pathway has not been elucidated. Results: Using a systems approach to study this question in a prototype fibrotic disease, Idiopathic Pulmonary Fibrosis (IPF); here we show organized changes in the gene expression pathway of primary lung myofibroblasts that persist for up to 9 sub-cultivations in vitro. When comparing IPF and control myofibroblasts in a 3-dimensional type I collagen matrix, more genes differed at the level of ribosome recruitment than at the level of transcript abundance, indicating pathological translational control as a major characteristic of IPF myofibroblasts. To determine the effect of matrix state on translational control, myofibroblasts were permitted to contract the matrix. Ribosome recruitment in control myofibroblasts was relatively stable. In contrast, IPF cells manifested large alterations in the ribosome recruitment pattern. Pathological studies suggest an epithelial origin for IPF myofibroblasts through the epithelial to mesenchymal transition (EMT). In accord wit

    The Effect of p38 Mitogen-Activated Protein Kinase Activation on Inflammatory Liver Damage following Hemorrhagic Shock in Rats

    Get PDF
    Hemorrhagic shock is a frequent cause of liver failure and often leads to a fatal outcome. Several studies have revealed that p38 MAPK is a key mediator in hemorrhagic damage of the primary organs through the activation of proinflammatory cytokines such as tumor necrosis factor (TNF)-α and interleukin (IL)-1β. However, the precise role of these factors in liver damage following hemorrhagic shock is unclear. In this study, we used FR167653, a specific inhibitor of p38 MAPK phosphorylation, to examine the role of p38 MAPK in liver damage occurring up to 5 hours after a hemorrhagic episode in a rat model. Activation of p38 MAPK in the liver as well as an increase in hepatic mRNA expression and serum concentrations of TNF-α and IL-1β occurred during the early phase after hemorrhage. Increased serum levels of hepatic enzymes, as well as histological damage and activated neutrophil accumulation in the liver, were observed in the late phase following hemorrhagic shock. FR167653 inhibited the inflammation-related hepatic injury following hemorrhagic shock. Bacterial lipopolysaccharide (LPS) derived from the gut appeared to have little effects on the hepatic damage. These results demonstrate that p38 MAPK activation is induced by hepatic ischemia during hemorrhagic shock and plays an important role both in the hepatic expression of proinflammatory cytokines and in the development of inflammation-related liver damage

    Auditory stimulation of opera music induced prolongation of murine cardiac allograft survival and maintained generation of regulatory CD4+CD25+ cells

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>Interactions between the immune response and brain functions such as olfactory, auditory, and visual sensations are likely. This study investigated the effect of sounds on alloimmune responses in a murine model of cardiac allograft transplantation.</p> <p>Methods</p> <p>Naïve CBA mice (H2<sup>k</sup>) underwent transplantation of a C57BL/6 (B6, H2<sup>b</sup>) heart and were exposed to one of three types of music--opera (<it>La Traviata</it>), classical (Mozart), and New Age (Enya)--or one of six different single sound frequencies, for 7 days. Additionally, we prepared two groups of CBA recipients with tympanic membrane perforation exposed to opera for 7 days and CBA recipients exposed to opera for 7 days before transplantation (pre-treatment). An adoptive transfer study was performed to determine whether regulatory cells were generated in allograft recipients. Immunohistochemical, cell-proliferation, cytokine, and flow cytometry assessments were also performed.</p> <p>Results</p> <p>CBA recipients of a B6 cardiac graft that were exposed to opera music and Mozart had significantly prolonged allograft survival (median survival times [MSTs], 26.5 and 20 days, respectively), whereas those exposed to a single sound frequency (100, 500, 1000, 5000, 10,000, or 20,000 Hz) or Enya did not (MSTs, 7.5, 8, 9, 8, 7.5, 8.5 and 11 days, respectively). Untreated, CBA mice with tympanic membrane perforations and CBA recipients exposed to opera for 7 days before transplantation (pre-treatment) rejected B6 cardiac grafts acutely (MSTs, 7, 8 and 8 days, respectively). Adoptive transfer of whole splenocytes, CD4<sup>+ </sup>cells, or CD4<sup>+</sup>CD25<sup>+ </sup>cells from opera-exposed primary allograft recipients resulted in significantly prolonged allograft survival in naive secondary recipients (MSTs, 36, 68, and > 100 days, respectively). Proliferation of splenocytes, interleukin (IL)-2 and interferon (IFN)-γ production was suppressed in opera-exposed mice, and production of IL-4 and IL-10 from opera-exposed transplant recipients increased compared to that from splenocytes of untreated recipients. Flow cytometry studies showed an increased CD4<sup>+</sup>CD25<sup>+ </sup>Forkhead box P3 (Foxp3)<sup>+ </sup>cell population in splenocytes from those mice.</p> <p>Conclusion</p> <p>Our findings indicate that exposure to opera music, such as La traviata, could affect such aspects of the peripheral immune response as generation of regulatory CD4<sup>+</sup>CD25<sup>+ </sup>cells and up-regulation of anti-inflammatory cytokines, resulting in prolonged allograft survival.</p
    corecore