683 research outputs found
Zero-Shot Learning with Common Sense Knowledge Graphs
Zero-shot learning relies on semantic class representations such as
hand-engineered attributes or learned embeddings to predict classes without any
labeled examples. We propose to learn class representations from common sense
knowledge graphs. Common sense knowledge graphs are an untapped source of
explicit high-level knowledge that requires little human effort to apply to a
range of tasks. To capture the knowledge in the graph, we introduce ZSL-KG, a
general-purpose framework with a novel transformer graph convolutional network
(TrGCN) for generating class representations. Our proposed TrGCN architecture
computes non-linear combinations of the node neighbourhood and shows
improvements on zero-shot learning tasks in language and vision. Our results
show ZSL-KG outperforms the best performing graph-based zero-shot learning
framework by an average of 2.1 accuracy points with improvements as high as 3.4
accuracy points. Our ablation study on ZSL-KG with alternate graph neural
networks shows that our TrGCN adds up to 1.2 accuracy points improvement on
these tasks
Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains
Accurately typing entity mentions from text segments is a fundamental task
for various natural language processing applications. Many previous approaches
rely on massive human-annotated data to perform entity typing. Nevertheless,
collecting such data in highly specialized science and engineering domains
(e.g., software engineering and security) can be time-consuming and costly,
without mentioning the domain gaps between training and inference data if the
model needs to be applied to confidential datasets. In this paper, we study the
task of seed-guided fine-grained entity typing in science and engineering
domains, which takes the name and a few seed entities for each entity type as
the only supervision and aims to classify new entity mentions into both seen
and unseen types (i.e., those without seed entities). To solve this problem, we
propose SEType which first enriches the weak supervision by finding more
entities for each seen type from an unlabeled corpus using the contextualized
representations of pre-trained language models. It then matches the enriched
entities to unlabeled text to get pseudo-labeled samples and trains a textual
entailment model that can make inferences for both seen and unseen types.
Extensive experiments on two datasets covering four domains demonstrate the
effectiveness of SEType in comparison with various baselines.Comment: 9 pages; Accepted to AAAI 2024 (Code:
https://github.com/yuzhimanhua/SEType
Fine-Grained Entity Typing in Hyperbolic Space
How can we represent hierarchical information present in large type
inventories for entity typing? We study the ability of hyperbolic embeddings to
capture hierarchical relations between mentions in context and their target
types in a shared vector space. We evaluate on two datasets and investigate two
different techniques for creating a large hierarchical entity type inventory:
from an expert-generated ontology and by automatically mining type
co-occurrences. We find that the hyperbolic model yields improvements over its
Euclidean counterpart in some, but not all cases. Our analysis suggests that
the adequacy of this geometry depends on the granularity of the type inventory
and the way hierarchical relations are inferred.Comment: 12 pages, 4 figures, final version, accepted at the 4th Workshop on
Representation Learning for NLP (RepL4NLP), held in conjunction with ACL 201
- …