24 research outputs found
Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model
Large language models (LLMs) have made significant advancements in natural
language processing (NLP). Broad corpora capture diverse patterns but can
introduce irrelevance, while focused corpora enhance reliability by reducing
misleading information. Training LLMs on focused corpora poses computational
challenges. An alternative approach is to use a retrieval-augmentation (RetA)
method tested in a specific domain.
To evaluate LLM performance, OpenAI's GPT-3, GPT-4, Bing's Prometheus, and a
custom RetA model were compared using 19 questions on diffuse large B-cell
lymphoma (DLBCL) disease. Eight independent reviewers assessed responses based
on accuracy, relevance, and readability (rated 1-3).
The RetA model performed best in accuracy (12/19 3-point scores, total=47)
and relevance (13/19, 50), followed by GPT-4 (8/19, 43; 11/19, 49). GPT-4
received the highest readability scores (17/19, 55), followed by GPT-3 (15/19,
53) and the RetA model (11/19, 47). Prometheus underperformed in accuracy (34),
relevance (32), and readability (38).
Both GPT-3.5 and GPT-4 had more hallucinations in all 19 responses compared
to the RetA model and Prometheus. Hallucinations were mostly associated with
non-existent references or fabricated efficacy data.
These findings suggest that RetA models, supplemented with domain-specific
corpora, may outperform general-purpose LLMs in accuracy and relevance within
specific domains. However, this evaluation was limited to specific questions
and metrics and may not capture challenges in semantic search and other NLP
tasks. Further research will explore different LLM architectures, RetA
methodologies, and evaluation methods to assess strengths and limitations more
comprehensively
The TLR9 ligand, CpG-ODN, Induces Protection Against Cerebral Ischemia/Reperfusion Injury via Activation of pi3k/Akt Signaling.
Toll-like receptors (TLRs) have been shown to be involved in cerebral ischemia/reperfusion (I/R) injury. TLR9 is located in intracellular compartments and recognizes CpG-DNA. This study examined the effect of CpG-ODN on cerebral I/R injury. C57BL/6 mice were treated with CpG-ODN by i.p. injection 1 hour before the mice were subjected to cerebral ischemia (60 minutes) followed by reperfusion (24 hours). Scrambled-ODN served as control-ODN. Untreated mice, subjected to cerebral I/R, served as I/R control. The effect of inhibitory CpG-ODN (iCpG-ODN) on cerebral I/R injury was also examined. In addition, we examined the therapeutic effect of CpG-ODN on cerebral I/R injury by administration of CpG-ODN 15 minutes after cerebral ischemia. CpG-ODN administration significantly decreased cerebral I/R-induced infarct volume by 69.7% (6.4±1.80% vs 21.0±2.85%, P\u3c0.05), improved neurological scores, and increased survival rate, when compared with the untreated I/R group. Therapeutic administration of CpG-ODN also significantly reduced infarct volume by 44.7% (12.6±2.03% vs 22.8±2.54%, P\u3c0.05) compared with untreated I/R mice. Neither control-ODN, nor iCpG-ODN altered I/R-induced cerebral injury or neurological deficits. Nissl staining showed that CpG-ODN treatment preserved neuronal morphology in the ischemic hippocampus. Immunoblot showed that CpG-ODN administration increased Bcl-2 levels by 41% and attenuated I/R-increased levels of Bax and caspase-3 activity in ischemic brain tissues. Importantly, CpG-ODN treatment induced Akt and GSK-3β phosphorylation in brain tissue and cultured microglial cells. PI3K inhibition with LY294002 abolished CpG-ODN-induced protection. CpG-ODN significantly reduces cerebral I/R injury via a PI3K/Akt-dependent mechanism. Our data also indicate that CpG-ODN may be useful in the therapy of cerebral I/R injury
Assessing Reproducibility of Inherited Variants Detected With Short-Read Whole Genome Sequencing
Background: Reproducible detection of inherited variants with whole genome sequencing (WGS) is vital for the implementation of precision medicine and is a complicated process in which each step affects variant call quality. Systematically assessing reproducibility of inherited variants with WGS and impact of each step in the process is needed for understanding and improving quality of inherited variants from WGS.
Results: To dissect the impact of factors involved in detection of inherited variants with WGS, we sequence triplicates of eight DNA samples representing two populations on three short-read sequencing platforms using three library kits in six labs and call variants with 56 combinations of aligners and callers. We find that bioinformatics pipelines (callers and aligners) have a larger impact on variant reproducibility than WGS platform or library preparation. Single-nucleotide variants (SNVs), particularly outside difficult-to-map regions, are more reproducible than small insertions and deletions (indels), which are least reproducible when \u3e 5 bp. Increasing sequencing coverage improves indel reproducibility but has limited impact on SNVs above 30×.
Conclusions: Our findings highlight sources of variability in variant detection and the need for improvement of bioinformatics pipelines in the era of precision medicine with WGS
Assessing reproducibility of inherited variants detected with short-read whole genome sequencing
Background: Reproducible detection of inherited variants with whole genome sequencing (WGS) is vital for the implementation of precision medicine and is a complicated process in which each step affects variant call quality. Systematically assessing reproducibility of inherited variants with WGS and impact of each step in the process is needed for understanding and improving quality of inherited variants from WGS. Results: To dissect the impact of factors involved in detection of inherited variants with WGS, we sequence triplicates of eight DNA samples representing two populations on three short-read sequencing platforms using three library kits in six labs and call variants with 56 combinations of aligners and callers. We find that bioinformatics pipelines (callers and aligners) have a larger impact on variant reproducibility than WGS platform or library preparation. Single-nucleotide variants (SNVs), particularly outside difficult-to-map regions, are more reproducible than small insertions and deletions (indels), which are least reproducible when > 5 bp. Increasing sequencing coverage improves indel reproducibility but has limited impact on SNVs above 30x. Conclusions: Our findings highlight sources of variability in variant detection and the need for improvement of bioinformatics pipelines in the era of precision medicine with WGS.Peer reviewe
NonDL_models_clean
Python file used to perform classification with non-RNN models
LSTM_keras_03012018_clean
Python code to perform RNN models
NLP_Report_Process_clean
Python code to process raw clinical report