433 research outputs found
Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification
Recent advances in large language models (LLMs) have shown impressive ability
in biomedical question-answering, but have not been adequately investigated for
more specific biomedical applications. This study investigates the performance
of LLMs such as the ChatGPT family of models (GPT-3.5s, GPT-4) in biomedical
tasks beyond question-answering. Because no patient data can be passed to the
OpenAI API public interface, we evaluated model performance with over 10000
samples as proxies for two fundamental tasks in the clinical domain -
classification and reasoning. The first task is classifying whether statements
of clinical and policy recommendations in scientific literature constitute
health advice. The second task is causal relation detection from the biomedical
literature. We compared LLMs with simpler models, such as bag-of-words (BoW)
with logistic regression, and fine-tuned BioBERT models. Despite the excitement
around viral ChatGPT, we found that fine-tuning for two fundamental NLP tasks
remained the best strategy. The simple BoW model performed on par with the most
complex LLM prompting. Prompt engineering required significant investment.Comment: 28 pages, 2 tables and 4 figures. Submitting for revie
Natural language processing to automatically extract the presence and severity of esophagitis in notes of patients undergoing radiotherapy
Radiotherapy (RT) toxicities can impair survival and quality-of-life, yet
remain under-studied. Real-world evidence holds potential to improve our
understanding of toxicities, but toxicity information is often only in clinical
notes. We developed natural language processing (NLP) models to identify the
presence and severity of esophagitis from notes of patients treated with
thoracic RT. We fine-tuned statistical and pre-trained BERT-based models for
three esophagitis classification tasks: Task 1) presence of esophagitis, Task
2) severe esophagitis or not, and Task 3) no esophagitis vs. grade 1 vs. grade
2-3. Transferability was tested on 345 notes from patients with esophageal
cancer undergoing RT.
Fine-tuning PubmedBERT yielded the best performance. The best macro-F1 was
0.92, 0.82, and 0.74 for Task 1, 2, and 3, respectively. Selecting the most
informative note sections during fine-tuning improved macro-F1 by over 2% for
all tasks. Silver-labeled data improved the macro-F1 by over 3% across all
tasks. For the esophageal cancer notes, the best macro-F1 was 0.73, 0.74, and
0.65 for Task 1, 2, and 3, respectively, without additional fine-tuning.
To our knowledge, this is the first effort to automatically extract
esophagitis toxicity severity according to CTCAE guidelines from clinic notes.
The promising performance provides proof-of-concept for NLP-based automated
detailed toxicity monitoring in expanded domains.Comment: 17 pages, 6 tables, 1figure, submiting to JCO-CCI for revie
Large Language Models to Identify Social Determinants of Health in Electronic Health Records
Social determinants of health (SDoH) have an important impact on patient
outcomes but are incompletely collected from the electronic health records
(EHR). This study researched the ability of large language models to extract
SDoH from free text in EHRs, where they are most commonly documented, and
explored the role of synthetic clinical text for improving the extraction of
these scarcely documented, yet extremely valuable, clinical data. 800 patient
notes were annotated for SDoH categories, and several transformer-based models
were evaluated. The study also experimented with synthetic data generation and
assessed for algorithmic bias. Our best-performing models were fine-tuned
Flan-T5 XL (macro-F1 0.71) for any SDoH, and Flan-T5 XXL (macro-F1 0.70). The
benefit of augmenting fine-tuning with synthetic data varied across model
architecture and size, with smaller Flan-T5 models (base and large) showing the
greatest improvements in performance (delta F1 +0.12 to +0.23). Model
performance was similar on the in-hospital system dataset but worse on the
MIMIC-III dataset. Our best-performing fine-tuned models outperformed zero- and
few-shot performance of ChatGPT-family models for both tasks. These fine-tuned
models were less likely than ChatGPT to change their prediction when
race/ethnicity and gender descriptors were added to the text, suggesting less
algorithmic bias (p<0.05). At the patient-level, our models identified 93.8% of
patients with adverse SDoH, while ICD-10 codes captured 2.0%. Our method can
effectively extracted SDoH information from clinic notes, performing better
compare to GPT zero- and few-shot settings. These models could enhance
real-world evidence on SDoH and aid in identifying patients needing social
support.Comment: 38 pages, 5 figures, 5 tables in main, submitted for revie
The impact of responding to patient messages with large language model assistance
Documentation burden is a major contributor to clinician burnout, which is
rising nationally and is an urgent threat to our ability to care for patients.
Artificial intelligence (AI) chatbots, such as ChatGPT, could reduce clinician
burden by assisting with documentation. Although many hospitals are actively
integrating such systems into electronic medical record systems, AI chatbots
utility and impact on clinical decision-making have not been studied for this
intended use. We are the first to examine the utility of large language models
in assisting clinicians draft responses to patient questions. In our two-stage
cross-sectional study, 6 oncologists responded to 100 realistic synthetic
cancer patient scenarios and portal messages developed to reflect common
medical situations, first manually, then with AI assistance.
We find AI-assisted responses were longer, less readable, but provided
acceptable drafts without edits 58% of time. AI assistance improved efficiency
77% of time, with low harm risk (82% safe). However, 7.7% unedited AI responses
could severely harm. In 31% cases, physicians thought AI drafts were
human-written. AI assistance led to more patient education recommendations,
fewer clinical actions than manual responses. Results show promise for AI to
improve clinician efficiency and patient care through assisting documentation,
if used judiciously. Monitoring model outputs and human-AI interaction remains
crucial for safe implementation.Comment: 4 figures and tables in main, submitted for revie
Apoptosis at Inflection Point in Liquid Culture of Budding Yeasts
Budding yeasts are highly suitable for aging studies, because the number of bud
scars (stage) proportionally correlates with age. Its maximum stages are known
to reach at 20–30 stages on an isolated agar medium. However, their stage
dynamics in a liquid culture is virtually unknown. We investigate the population
dynamics by counting scars in each cell. Here one cell division produces one new
cell and one bud scar. This simple rule leads to a conservation law: “The
total number of bud scars is equal to the total number of cells.” We find
a large discrepancy: extremely fewer cells with over 5 scars than expected.
Almost all cells with 6 or more scars disappear within a short period of time in
the late log phase (corresponds to the inflection point). This discrepancy is
confirmed directly by the microscopic observations of broken cells. This finding
implies apoptosis in older cells (6 scars or more)
Aversive Learning in Honeybees Revealed by the Olfactory Conditioning of the Sting Extension Reflex
Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory
Fibrotic Myofibroblasts Manifest Genome-Wide Derangements of Translational Control
Background: As a group, fibroproliferative disorders of the lung, liver, kidney, heart, vasculature and integument are common, progressive and refractory to therapy. They can emerge following toxic insults, but are frequently idiopathic. Their enigmatic propensity to resist therapy and progress to organ failure has focused attention on the myofibroblast–the primary effector of the fibroproliferative response. We have recently shown that aberrant beta 1 integrin signaling in fibrotic fibroblasts results in defective PTEN function, unrestrained Akt signaling and subsequent activation of the translation initiation machinery. How this pathological integrin signaling alters the gene expression pathway has not been elucidated. Results: Using a systems approach to study this question in a prototype fibrotic disease, Idiopathic Pulmonary Fibrosis (IPF); here we show organized changes in the gene expression pathway of primary lung myofibroblasts that persist for up to 9 sub-cultivations in vitro. When comparing IPF and control myofibroblasts in a 3-dimensional type I collagen matrix, more genes differed at the level of ribosome recruitment than at the level of transcript abundance, indicating pathological translational control as a major characteristic of IPF myofibroblasts. To determine the effect of matrix state on translational control, myofibroblasts were permitted to contract the matrix. Ribosome recruitment in control myofibroblasts was relatively stable. In contrast, IPF cells manifested large alterations in the ribosome recruitment pattern. Pathological studies suggest an epithelial origin for IPF myofibroblasts through the epithelial to mesenchymal transition (EMT). In accord wit
The Effect of p38 Mitogen-Activated Protein Kinase Activation on Inflammatory Liver Damage following Hemorrhagic Shock in Rats
Hemorrhagic shock is a frequent cause of liver failure and often leads to a fatal outcome. Several studies have revealed that p38 MAPK is a key mediator in hemorrhagic damage of the primary organs through the activation of proinflammatory cytokines such as tumor necrosis factor (TNF)-α and interleukin (IL)-1β. However, the precise role of these factors in liver damage following hemorrhagic shock is unclear. In this study, we used FR167653, a specific inhibitor of p38 MAPK phosphorylation, to examine the role of p38 MAPK in liver damage occurring up to 5 hours after a hemorrhagic episode in a rat model. Activation of p38 MAPK in the liver as well as an increase in hepatic mRNA expression and serum concentrations of TNF-α and IL-1β occurred during the early phase after hemorrhage. Increased serum levels of hepatic enzymes, as well as histological damage and activated neutrophil accumulation in the liver, were observed in the late phase following hemorrhagic shock. FR167653 inhibited the inflammation-related hepatic injury following hemorrhagic shock. Bacterial lipopolysaccharide (LPS) derived from the gut appeared to have little effects on the hepatic damage. These results demonstrate that p38 MAPK activation is induced by hepatic ischemia during hemorrhagic shock and plays an important role both in the hepatic expression of proinflammatory cytokines and in the development of inflammation-related liver damage
Auditory stimulation of opera music induced prolongation of murine cardiac allograft survival and maintained generation of regulatory CD4+CD25+ cells
<p>Abstract</p> <p>Background</p> <p>Interactions between the immune response and brain functions such as olfactory, auditory, and visual sensations are likely. This study investigated the effect of sounds on alloimmune responses in a murine model of cardiac allograft transplantation.</p> <p>Methods</p> <p>Naïve CBA mice (H2<sup>k</sup>) underwent transplantation of a C57BL/6 (B6, H2<sup>b</sup>) heart and were exposed to one of three types of music--opera (<it>La Traviata</it>), classical (Mozart), and New Age (Enya)--or one of six different single sound frequencies, for 7 days. Additionally, we prepared two groups of CBA recipients with tympanic membrane perforation exposed to opera for 7 days and CBA recipients exposed to opera for 7 days before transplantation (pre-treatment). An adoptive transfer study was performed to determine whether regulatory cells were generated in allograft recipients. Immunohistochemical, cell-proliferation, cytokine, and flow cytometry assessments were also performed.</p> <p>Results</p> <p>CBA recipients of a B6 cardiac graft that were exposed to opera music and Mozart had significantly prolonged allograft survival (median survival times [MSTs], 26.5 and 20 days, respectively), whereas those exposed to a single sound frequency (100, 500, 1000, 5000, 10,000, or 20,000 Hz) or Enya did not (MSTs, 7.5, 8, 9, 8, 7.5, 8.5 and 11 days, respectively). Untreated, CBA mice with tympanic membrane perforations and CBA recipients exposed to opera for 7 days before transplantation (pre-treatment) rejected B6 cardiac grafts acutely (MSTs, 7, 8 and 8 days, respectively). Adoptive transfer of whole splenocytes, CD4<sup>+ </sup>cells, or CD4<sup>+</sup>CD25<sup>+ </sup>cells from opera-exposed primary allograft recipients resulted in significantly prolonged allograft survival in naive secondary recipients (MSTs, 36, 68, and > 100 days, respectively). Proliferation of splenocytes, interleukin (IL)-2 and interferon (IFN)-γ production was suppressed in opera-exposed mice, and production of IL-4 and IL-10 from opera-exposed transplant recipients increased compared to that from splenocytes of untreated recipients. Flow cytometry studies showed an increased CD4<sup>+</sup>CD25<sup>+ </sup>Forkhead box P3 (Foxp3)<sup>+ </sup>cell population in splenocytes from those mice.</p> <p>Conclusion</p> <p>Our findings indicate that exposure to opera music, such as La traviata, could affect such aspects of the peripheral immune response as generation of regulatory CD4<sup>+</sup>CD25<sup>+ </sup>cells and up-regulation of anti-inflammatory cytokines, resulting in prolonged allograft survival.</p
- …