24 research outputs found
Multimodal Analogical Reasoning over Knowledge Graphs
Analogical reasoning is fundamental to human cognition and holds an important
place in various fields. However, previous studies mainly focus on single-modal
analogical reasoning and ignore taking advantage of structure knowledge.
Notably, the research in cognitive psychology has demonstrated that information
from multimodal sources always brings more powerful cognitive transfer than
single modality sources. To this end, we introduce the new task of multimodal
analogical reasoning over knowledge graphs, which requires multimodal reasoning
ability with the help of background knowledge. Specifically, we construct a
Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph
MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained
Transformer baselines, illustrating the potential challenges of the proposed
task. We further propose a novel model-agnostic Multimodal analogical reasoning
framework with Transformer (MarT) motivated by the structure mapping theory,
which can obtain better performance. Code and datasets are available in
https://github.com/zjunlp/MKG_Analogy.Comment: Accepted by ICLR 202
Contrastive Demonstration Tuning for Pre-trained Language Models
Pretrained language models can be effectively stimulated by textual prompts
or demonstrations, especially in low-data scenarios. Recent works have focused
on automatically searching discrete or continuous prompts or optimized
verbalizers, yet studies for the demonstration are still limited. Concretely,
the demonstration examples are crucial for an excellent final performance of
prompt-tuning. In this paper, we propose a novel pluggable, extensible, and
efficient approach named contrastive demonstration tuning, which is free of
demonstration sampling. Furthermore, the proposed approach can be: (i) Plugged
to any previous prompt-tuning approaches; (ii) Extended to widespread
classification tasks with a large number of categories. Experimental results on
16 datasets illustrate that our method integrated with previous approaches
LM-BFF and P-tuning can yield better performance. Code is available in
https://github.com/zjunlp/PromptKG/tree/main/research/Demo-Tuning.Comment: Work in progres
Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
Large Language Models (LLMs), with their remarkable task-handling
capabilities and innovative outputs, have catalyzed significant advancements
across a spectrum of fields. However, their proficiency within specialized
domains such as biomolecular studies remains limited. To address this
challenge, we introduce Mol-Instructions, a meticulously curated, comprehensive
instruction dataset expressly designed for the biomolecular realm.
Mol-Instructions is composed of three pivotal components: molecule-oriented
instructions, protein-oriented instructions, and biomolecular text
instructions, each curated to enhance the understanding and prediction
capabilities of LLMs concerning biomolecular features and behaviors. Through
extensive instruction tuning experiments on the representative LLM, we
underscore the potency of Mol-Instructions to enhance the adaptability and
cognitive acuity of large models within the complex sphere of biomolecular
studies, thereby promoting advancements in the biomolecular research community.
Mol-Instructions is made publicly accessible for future research endeavors and
will be subjected to continual updates for enhanced applicability.Comment: Project homepage: https://github.com/zjunlp/Mol-Instructions. Add
quantitative evaluation
Relphormer: Relational Graph Transformer for Knowledge Graph Representations
Transformers have achieved remarkable performance in widespread fields,
including natural language processing, computer vision and graph mining.
However, vanilla Transformer architectures have not yielded promising
improvements in the Knowledge Graph (KG) representations, where the
translational distance paradigm dominates this area. Note that vanilla
Transformer architectures struggle to capture the intrinsically heterogeneous
structural and semantic information of knowledge graphs. To this end, we
propose a new variant of Transformer for knowledge graph representations dubbed
Relphormer. Specifically, we introduce Triple2Seq which can dynamically sample
contextualized sub-graph sequences as the input to alleviate the heterogeneity
issue. We propose a novel structure-enhanced self-attention mechanism to encode
the relational information and keep the semantic information within entities
and relations. Moreover, we utilize masked knowledge modeling for general
knowledge graph representation learning, which can be applied to various
KG-based tasks including knowledge graph completion, question answering, and
recommendation. Experimental results on six datasets show that Relphormer can
obtain better performance compared with baselines. Code is available in
https://github.com/zjunlp/Relphormer.Comment: Work in progres
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Artificial Intelligence (AI), along with the recent progress in biomedical
language understanding, is gradually changing medical practice. With the
development of biomedical language understanding benchmarks, AI applications
are widely used in the medical field. However, most benchmarks are limited to
English, which makes it challenging to replicate many of the successes in
English for other languages. To facilitate research in this direction, we
collect real-world biomedical data and present the first Chinese Biomedical
Language Understanding Evaluation (CBLUE) benchmark: a collection of natural
language understanding tasks including named entity recognition, information
extraction, clinical diagnosis normalization, single-sentence/sentence-pair
classification, and an associated online platform for model evaluation,
comparison, and analysis. To establish evaluation on these tasks, we report
empirical results with the current 11 pre-trained Chinese models, and
experimental results show that state-of-the-art neural models perform by far
worse than the human ceiling. Our benchmark is released at
\url{https://tianchi.aliyun.com/dataset/dataDetail?dataId=95414&lang=en-us}
Electroacupuncture Mitigates Skeletal Muscular Lipid Metabolism Disorder Related to High-Fat-Diet Induced Insulin Resistance through the AMPK/ACC Signaling Pathway
The aim of this work is to investigate the effect of electroacupuncture (EA) on insulin sensitivity in high-fat diet (HFD) induced insulin resistance (IR) rats and to evaluate expression of AMPK/ACC signaling components. Thirty-two male Sprague-Dawley rats were randomized into control group, HFD group, HFD+Pi (oral gavage of pioglitazone) group, and HFD+EA group. Acupuncture was subcutaneously applied to Zusanli (ST40) and Sanyinjiao (SP6). For Zusanli (ST40) and Sanyinjiao (SP6), needles were connected to an electroacupuncture (EA) apparatus. Fasting plasma glucose was measured by glucose oxidase method. Plasma fasting insulin (FINS) and adiponectin (ADP) were determined by ELISA. Triglyceride (TG) and cholesterol (TC) were determined by Gpo-pap. Proteins of adiponectin receptor 1 (adipoR1), AMP-activated Protein Kinase (AMPK), and acetyl-CoA carboxylase (ACC) were determined by Western blot, respectively. Compared with the control group, HFD group exhibits increased levels of FPG, FINS, and homeostatic model assessment of insulin resistance (HOMA-IR) and decreased level of ADP and insulin sensitivity index (ISI). These changes were reversed by both EA and pioglitazone. Proteins of adipoR1 and AMPK were decreased, while ACC were increased in HFD group compared to control group. Proteins of these molecules were restored back to normal levels upon EA and pioglitazone. EA can improve the insulin sensitivity of insulin resistance rats; the positive regulation of the AMPK/ACC pathway in the skeletal muscle may be a possible mechanism of EA in the treatment of IR
Multi-Modal Protein Knowledge Graph Construction and Applications (Student Abstract)
Existing data-centric methods for protein science generally cannot sufficiently capture and leverage biology knowledge, which may be crucial for many protein tasks. To facilitate research in this field, we create ProteinKG65, a knowledge graph for protein science. Using gene ontology and Uniprot knowledge base as a basis, we transform and integrate various kinds of knowledge with aligned descriptions and protein sequences, respectively, to GO terms and protein entities. ProteinKG65 is mainly dedicated to providing a specialized protein knowledge graph, bringing the knowledge of Gene Ontology to protein function and structure prediction. We also illustrate the potential applications of ProteinKG65 with a prototype. Our dataset can be downloaded at https://w3id.org/proteinkg65
OntoProtein: Protein Pretraining With Gene Ontology Embedding
Self-supervised protein language models have proved their effectiveness in
learning the proteins representations. With the increasing computational power,
current protein language models pre-trained with millions of diverse sequences
can advance the parameter scale from million-level to billion-level and achieve
remarkable improvement. However, those prevailing approaches rarely consider
incorporating knowledge graphs (KGs), which can provide rich structured
knowledge facts for better protein representations. We argue that informative
biology knowledge in KGs can enhance protein representation with external
knowledge. In this work, we propose OntoProtein, the first general framework
that makes use of structure in GO (Gene Ontology) into protein pre-training
models. We construct a novel large-scale knowledge graph that consists of GO
and its related proteins, and gene annotation texts or protein sequences
describe all nodes in the graph. We propose novel contrastive learning with
knowledge-aware negative sampling to jointly optimize the knowledge graph and
protein embedding during pre-training. Experimental results show that
OntoProtein can surpass state-of-the-art methods with pre-trained protein
language models in TAPE benchmark and yield better performance compared with
baselines in protein-protein interaction and protein function prediction. Code
and datasets are available in https://github.com/zjunlp/OntoProtein.Comment: Accepted by ICLR 202
IFI16 Inhibits Porcine Reproductive and Respiratory Syndrome Virus 2 Replication in a MAVS-Dependent Manner in MARC-145 Cells
Porcine reproductive and respiratory syndrome virus (PRRSV) is a single-stranded positive-sense RNA virus, and the current strategies for controlling PRRSV are limited. Interferon gamma-inducible protein 16 (IFI16) has been reported to have a broader role in the regulation of the type I interferons (IFNs) response to RNA and DNA viruses. However, the function of IFI16 in PRRSV infection is unclear. Here, we revealed that IFI16 acts as a novel antiviral protein against PRRSV-2. IFI16 could be induced by interferon-beta (IFN-β). Overexpression of IFI16 could significantly suppress PRRSV-2 replication, and silencing the expression of endogenous IFI16 by small interfering RNAs led to the promotion of PRRSV-2 replication in MARC-145 cells. Additionally, IFI16 could promote mitochondrial antiviral signaling protein (MAVS)-mediated production of type I interferon and interact with MAVS. More importantly, IFI16 exerted anti-PRRSV effects in a MAVS-dependent manner. In conclusion, our data demonstrated that IFI16 has an inhibitory effect on PRRSV-2, and these findings contribute to understanding the role of cellular proteins in regulating PRRSV replication and may have implications for the future antiviral strategies