5 research outputs found
LawBench: Benchmarking Legal Knowledge of Large Language Models
Large language models (LLMs) have demonstrated strong capabilities in various
aspects. However, when applying them to the highly specialized, safe-critical
legal domain, it is unclear how much legal knowledge they possess and whether
they can reliably perform legal-related tasks. To address this gap, we propose
a comprehensive evaluation benchmark LawBench. LawBench has been meticulously
crafted to have precise assessment of the LLMs' legal capabilities from three
cognitive levels: (1) Legal knowledge memorization: whether LLMs can memorize
needed legal concepts, articles and facts; (2) Legal knowledge understanding:
whether LLMs can comprehend entities, events and relationships within legal
text; (3) Legal knowledge applying: whether LLMs can properly utilize their
legal knowledge and make necessary reasoning steps to solve realistic legal
tasks. LawBench contains 20 diverse tasks covering 5 task types: single-label
classification (SLC), multi-label classification (MLC), regression, extraction
and generation. We perform extensive evaluations of 51 LLMs on LawBench,
including 20 multilingual LLMs, 22 Chinese-oriented LLMs and 9 legal specific
LLMs. The results show that GPT-4 remains the best-performing LLM in the legal
domain, surpassing the others by a significant margin. While fine-tuning LLMs
on legal specific text brings certain improvements, we are still a long way
from obtaining usable and reliable LLMs in legal tasks. All data, model
predictions and evaluation code are released in
https://github.com/open-compass/LawBench/. We hope this benchmark provides
in-depth understanding of the LLMs' domain-specified capabilities and speed up
the development of LLMs in the legal domain
Mendelian randomization and clinical trial evidence supports TYK2 inhibition as a therapeutic target for autoimmune diseases
Background: To explore the associations of genetically proxied TYK2 inhibition with a wide range of disease outcomes and biomarkers to identify therapeutic repurposing opportunities, adverse effects, and biomarkers of efficacy. Methods: The loss-of-function missense variant rs34536443 in TYK2 gene was used as a genetic instrument to proxy the effect of TYK2 inhibition. A phenome-wide Mendelian randomization (MR) study was conducted to explore the associations of genetically-proxied TYK2 inhibition with 1473 disease outcomes in UK Biobank (N = 339,197). Identified associations were examined for replication in FinnGen (N = 260,405). We further performed tissue -specific gene expression MR, colocalization analyses, and MR with 247 blood biomarkers. A systematic review of randomized controlled trials (RCTs) on TYK2 inhibitor was performed to complement the genetic evidence. Findings: PheWAS-MR found that genetically-proxied TYK2 inhibition was associated with lower risk of a wide range of autoimmune diseases. The associations with hypothyroidism and psoriasis were confirmed in MR analysis of tissue-specific TYK2 gene expression and the associations with systemic lupus erythematosus, psoriasis, and rheumatoid arthritis were observed in colocalization analysis. There were nominal associations of genetically-proxied TYK2 inhibition with increased risk of prostate and breast cancer but not in tissue-specific expression MR or colocalization analyses. Thirty-seven blood biomarkers were associated with the TYK2 loss-of-function mutation. Evidence from RCTs confirmed the effectiveness of TYK2 inhibitors on plaque psoriasis and reported several adverse effects. Interpretation: This study supports TYK2 inhibitor as a potential treatment for psoriasis and several other autoim-mune diseases. Increased pharmacovigilance is warranted in relation to the potential adverse effects.De tvÄ första författarna delar förstaförfattarskapet.De tre sista författarna delar sistaförfattarskapet.</p
Molecular Variation and Genomic Function of Citrus Vein Enation Virus
In this study, we identified a new citrus vein enation virus (CVEV) isolate (named CVEV-DT1) through sRNA high-throughput sequencing and traditional sequencing. Phylogenetic analysis based on whole genome sequences of all known CVEV isolates revealed that CVEV-DT1 was in an evolutionary branch with other isolates from China. Molecular variation analysis showed that the single nucleotide variability along CVEV full-length sequences was less than 8%, with more transitions (60.55%) than transversions (39.43%), indicating a genetically homogeneous CVEV population. In addition, non-synonymous nucleotide mutations mainly occurred in ORF1 and ORF2. Based on disorder analysis of all encoded ORF by CVEV-DT1, we identified that the CVEV-DT1 coat protein (CP) formed spherical granules, mainly in the cell nucleus and partly throughout the cytoplasm, with liquid properties through subcellular localization and photobleaching assay. Furthermore, we also confirmed that the CVEV P0 protein has weak post-transcriptional RNA-silencing suppressor activity and could elicit a strong hypersensitive response (HR) in tobacco plants. Collectively, to the best of our knowledge, our study was the first to profile the genomic variation in all the reported CVEV isolates and reveal the functions of CVEV-DT1-encoded proteins