344 research outputs found
A Survey on Knowledge-Enhanced Pre-trained Language Models
Natural Language Processing (NLP) has been revolutionized by the use of
Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in
nearly every NLP task, PLMs still face a number of challenges including poor
interpretability, weak reasoning capability, and the need for a lot of
expensive annotated data when applied to downstream tasks. By integrating
external knowledge into PLMs,
\textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained
\underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to
overcome the above-mentioned limitations. In this paper, we examine KEPLMs
systematically through a series of studies. Specifically, we outline the common
types and different formats of knowledge to be integrated into KEPLMs, detail
the existing methods for building and evaluating KEPLMS, present the
applications of KEPLMs in downstream tasks, and discuss the future research
directions. Researchers will benefit from this survey by gaining a quick and
comprehensive overview of the latest developments in this field.Comment: 19 pages, 12 figures, 192 reference
Event Extraction: A Survey
Extracting the reported events from text is one of the key research themes in
natural language processing. This process includes several tasks such as event
detection, argument extraction, role labeling. As one of the most important
topics in natural language processing and natural language understanding, the
applications of event extraction spans across a wide range of domains such as
newswire, biomedical domain, history and humanity, and cyber security. This
report presents a comprehensive survey for event detection from textual
documents. In this report, we provide the task definition, the evaluation
method, as well as the benchmark datasets and a taxonomy of methodologies for
event extraction. We also present our vision of future research direction in
event detection.Comment: 20 page
Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey
Storytelling and narrative are fundamental to human experience, intertwined
with our social and cultural engagement. As such, researchers have long
attempted to create systems that can generate stories automatically. In recent
years, powered by deep learning and massive data resources, automatic story
generation has shown significant advances. However, considerable challenges,
like the need for global coherence in generated stories, still hamper
generative models from reaching the same storytelling ability as human
narrators. To tackle these challenges, many studies seek to inject structured
knowledge into the generation process, which is referred to as structure
knowledge-enhanced story generation. Incorporating external knowledge can
enhance the logical coherence among story events, achieve better knowledge
grounding, and alleviate over-generalization and repetition problems in
stories. This survey provides the latest and comprehensive review of this
research field: (i) we present a systematical taxonomy regarding how existing
methods integrate structured knowledge into story generation; (ii) we summarize
involved story corpora, structured knowledge datasets, and evaluation metrics;
(iii) we give multidimensional insights into the challenges of
knowledge-enhanced story generation and cast light on promising directions for
future study
A survey on knowledge-enhanced multimodal learning
Multimodal learning has been a field of increasing interest, aiming to
combine various modalities in a single joint representation. Especially in the
area of visiolinguistic (VL) learning multiple models and techniques have been
developed, targeting a variety of tasks that involve images and text. VL models
have reached unprecedented performances by extending the idea of Transformers,
so that both modalities can learn from each other. Massive pre-training
procedures enable VL models to acquire a certain level of real-world
understanding, although many gaps can be identified: the limited comprehension
of commonsense, factual, temporal and other everyday knowledge aspects
questions the extendability of VL tasks. Knowledge graphs and other knowledge
sources can fill those gaps by explicitly providing missing information,
unlocking novel capabilities of VL models. In the same time, knowledge graphs
enhance explainability, fairness and validity of decision making, issues of
outermost importance for such complex implementations. The current survey aims
to unify the fields of VL representation learning and knowledge graphs, and
provides a taxonomy and analysis of knowledge-enhanced VL models
All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparison
Public opinion is shaped by the information news media provide, and that
information in turn may be shaped by the ideological preferences of media
outlets. But while much attention has been devoted to media bias via overt
ideological language or topic selection, a more unobtrusive way in which the
media shape opinion is via the strategic inclusion or omission of partisan
events that may support one side or the other. We develop a latent
variable-based framework to predict the ideology of news articles by comparing
multiple articles on the same story and identifying partisan events whose
inclusion or omission reveals ideology. Our experiments first validate the
existence of partisan event selection, and then show that article alignment and
cross-document comparison detect partisan events and article ideology better
than competitive baselines. Our results reveal the high-level form of media
bias, which is present even among mainstream media with strong norms of
objectivity and nonpartisanship. Our codebase and dataset are available at
https://github.com/launchnlp/ATC.Comment: EMNLP'23 Main Conferenc
- …