107 research outputs found
Modeling Relation Paths for Representation Learning of Knowledge Bases
Representation learning of knowledge bases (KBs) aims to embed both entities
and relations into a low-dimensional space. Most existing methods only consider
direct relations in representation learning. We argue that multiple-step
relation paths also contain rich inference patterns between entities, and
propose a path-based representation learning model. This model considers
relation paths as translations between entities for representation learning,
and addresses two key challenges: (1) Since not all relation paths are
reliable, we design a path-constraint resource allocation algorithm to measure
the reliability of relation paths. (2) We represent relation paths via semantic
composition of relation embeddings. Experimental results on real-world datasets
show that, as compared with baselines, our model achieves significant and
consistent improvements on knowledge base completion and relation extraction
from text.Comment: 10 page
Graph Neural Networks with Generated Parameters for Relation Extraction
Recently, progress has been made towards improving relational reasoning in
machine learning field. Among existing models, graph neural networks (GNNs) is
one of the most effective approaches for multi-hop relational reasoning. In
fact, multi-hop relational reasoning is indispensable in many natural language
processing tasks such as relation extraction. In this paper, we propose to
generate the parameters of graph neural networks (GP-GNNs) according to natural
language sentences, which enables GNNs to process relational reasoning on
unstructured text inputs. We verify GP-GNNs in relation extraction from text.
Experimental results on a human-annotated dataset and two distantly supervised
datasets show that our model achieves significant improvements compared to
baselines. We also perform a qualitative analysis to demonstrate that our model
could discover more accurate relations by multi-hop relational reasoning
Representation Learning for Natural Language Processing
This open access book provides an overview of the recent advances in representation learning theory, algorithms and applications for natural language processing (NLP). It is divided into three parts. Part I presents the representation learning techniques for multiple language entries, including words, phrases, sentences and documents. Part II then introduces the representation techniques for those objects that are closely related to NLP, including entity-based world knowledge, sememe-based linguistic knowledge, networks, and cross-modal entries. Lastly, Part III provides open resource tools for representation learning techniques, and discusses the remaining challenges and future research directions. The theories and algorithms of representation learning presented can also benefit other related domains such as machine learning, social network analysis, semantic Web, information retrieval, data mining and computational biology. This book is intended for advanced undergraduate and graduate students, post-doctoral fellows, researchers, lecturers, and industrial engineers, as well as anyone interested in representation learning and natural language processing
Characteristics of multicystic biliary hamartoma: A case report
IntroductionMulticystic biliary hamartoma (MCBH) is a very rare hepatic benign neoplasm that manifests as a localized cystic-solid mass. Only 17 cases have been described in the literature to date. MCBH diagnosis is currently dependent on imaging and pathology following surgical resection and no precise standards are in place.Case PresentationThis case study involves a middle-aged male patient with a history of drinking but no other liver diseases. A routine ultrasound examination showed a 6.0 × 5.5 cm inhomogeneous echo mass in the right lobe of the liver. The patient experienced no discomfort or other symptoms, and blood tests were normal. Imaging revealed a localized cystic-solid neoplasm in segment 6 of the liver that did not have the features of a malignant tumor. Surgical resection was performed. Based on imaging, macroscopic examination, and histological results, a final diagnosis of MCBH was made.ConclusionThe imaging and pathological features of MCBH were summarized based on the published case reports to date. As a non-invasive examination, the imaging features will aid in the diagnosis of MCBH. Furthermore, these features, along with tumor size and patient symptoms, will facilitate clinicians in selecting surgical resection or follow-up for individual patients
Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View
Graph Neural Networks (GNNs) have achieved promising performance on a wide
range of graph-based tasks. Despite their success, one severe limitation of
GNNs is the over-smoothing issue (indistinguishable representations of nodes in
different classes). In this work, we present a systematic and quantitative
study on the over-smoothing issue of GNNs. First, we introduce two quantitative
metrics, MAD and MADGap, to measure the smoothness and over-smoothness of the
graph nodes representations, respectively. Then, we verify that smoothing is
the nature of GNNs and the critical factor leading to over-smoothness is the
low information-to-noise ratio of the message received by the nodes, which is
partially determined by the graph topology. Finally, we propose two methods to
alleviate the over-smoothing issue from the topological view: (1) MADReg which
adds a MADGap-based regularizer to the training objective;(2) AdaGraph which
optimizes the graph topology based on the model predictions. Extensive
experiments on 7 widely-used graph datasets with 10 typical GNN models show
that the two proposed methods are effective for relieving the over-smoothing
issue, thus improving the performance of various GNN models.Comment: Accepted by AAAI 2020. This complete version contains the appendi
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Parameter-efficient tuning methods (PETs) have achieved promising results in
tuning large pre-trained language models (PLMs). By formalizing frozen PLMs and
additional tunable parameters as systems and controls respectively, PETs can be
theoretically grounded to optimal control and further viewed as optimizing the
terminal cost and running cost in the optimal control literature. Despite the
elegance of this theoretical grounding, in practice, existing PETs often ignore
the running cost and only optimize the terminal cost, i.e., focus on optimizing
the loss function of the output state, regardless of the running cost that
depends on the intermediate states. Since it is non-trivial to directly model
the intermediate states and design a running cost function, we propose to use
latent stochastic bridges to regularize the intermediate states and use the
regularization as the running cost of PETs. As the first work to propose
regularized PETs that use stochastic bridges as the regularizers (running
costs) for the intermediate states, we show the effectiveness and generality of
this regularization across different tasks, PLMs and PETs. In view of the great
potential and capacity, we believe more sophisticated regularizers can be
designed for PETs and better performance can be achieved in the future. The
code is released at
\url{https://github.com/thunlp/stochastic-bridge-pet/tree/main}.Comment: ACL 2023 Finding
- …