569,456 research outputs found
Schema-aware Reference as Prompt Improves Data-Efficient Relational Triple and Event Extraction
Information Extraction, which aims to extract structural relational triple or
event from unstructured texts, often suffers from data scarcity issues. With
the development of pre-trained language models, many prompt-based approaches to
data-efficient information extraction have been proposed and achieved
impressive performance. However, existing prompt learning methods for
information extraction are still susceptible to several potential limitations:
(i) semantic gap between natural language and output structure knowledge with
pre-defined schema; (ii) representation learning with locally individual
instances limits the performance given the insufficient features. In this
paper, we propose a novel approach of schema-aware Reference As Prompt (RAP),
which dynamically leverage schema and knowledge inherited from global
(few-shot) training data for each sample. Specifically, we propose a
schema-aware reference store, which unifies symbolic schema and relevant
textual instances. Then, we employ a dynamic reference integration module to
retrieve pertinent knowledge from the datastore as prompts during training and
inference. Experimental results demonstrate that RAP can be plugged into
various existing models and outperforms baselines in low-resource settings on
four datasets of relational triple extraction and event extraction. In
addition, we provide comprehensive empirical ablations and case analysis
regarding different types and scales of knowledge in order to better understand
the mechanisms of RAP. Code is available in https://github.com/zjunlp/RAP.Comment: Work in progres
Generating Faithful Text From a Knowledge Graph with Noisy Reference Text
Knowledge Graph (KG)-to-Text generation aims at generating fluent
natural-language text that accurately represents the information of a given
knowledge graph. While significant progress has been made in this task by
exploiting the power of pre-trained language models (PLMs) with appropriate
graph structure-aware modules, existing models still fall short of generating
faithful text, especially when the ground-truth natural-language text contains
additional information that is not present in the graph. In this paper, we
develop a KG-to-text generation model that can generate faithful
natural-language text from a given graph, in the presence of noisy reference
text. Our framework incorporates two core ideas: Firstly, we utilize
contrastive learning to enhance the model's ability to differentiate between
faithful and hallucinated information in the text, thereby encouraging the
decoder to generate text that aligns with the input graph. Secondly, we empower
the decoder to control the level of hallucination in the generated text by
employing a controllable text generation technique. We evaluate our model's
performance through the standard quantitative metrics as well as a
ChatGPT-based quantitative and qualitative analysis. Our evaluation
demonstrates the superior performance of our model over state-of-the-art
KG-to-text models on faithfulness
Change Patterns for Process Families
The increasing adoption of process-aware information systems (PAISs), together with the variability of business processes (BPs), has resulted in large collections of related process model variants (i.e., process families). To effectively deal with process families, several proposals (e.g., C-EPC, Provop) exist that extend BP modeling languages with variability-specific constructs. While fostering reuse and reducing modeling efforts, respective constructs imply additional complexity and demand proper support for process designers when creating and modifying process families. Recently, generic and language-independent adaptation patterns were successfully introduced for creating and evolving single BP models. However, they are not sufficient to cope with the specific needs for modeling and evolving process families. This paper suggests a complementary set of generic and language-independent change patterns specifically tailored to the needs of process families. When used in combination with existing adaptation patterns, change patterns for process families will enable the modeling and evolution of process families at a high-level of abstraction. Further, they will serve as reference for implementing tools or comparing proposals managing process families
Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation
This paper demonstrates that word sense disambiguation (WSD) can improve
neural machine translation (NMT) by widening the source context considered when
modeling the senses of potentially ambiguous words. We first introduce three
adaptive clustering algorithms for WSD, based on k-means, Chinese restaurant
processes, and random walks, which are then applied to large word contexts
represented in a low-rank space and evaluated on SemEval shared-task data. We
then learn word vectors jointly with sense vectors defined by our best WSD
method, within a state-of-the-art NMT system. We show that the concatenation of
these vectors, and the use of a sense selection mechanism based on the weighted
average of sense vectors, outperforms several baselines including sense-aware
ones. This is demonstrated by translation on five language pairs. The
improvements are above one BLEU point over strong NMT baselines, +4% accuracy
over all ambiguous nouns and verbs, or +20% when scored manually over several
challenging words.Comment: To appear in TAC
- …