3,247 research outputs found
Bayesian Information Extraction Network
Dynamic Bayesian networks (DBNs) offer an elegant way to integrate various
aspects of language in one model. Many existing algorithms developed for
learning and inference in DBNs are applicable to probabilistic language
modeling. To demonstrate the potential of DBNs for natural language processing,
we employ a DBN in an information extraction task. We show how to assemble
wealth of emerging linguistic instruments for shallow parsing, syntactic and
semantic tagging, morphological decomposition, named entity recognition etc. in
order to incrementally build a robust information extraction system. Our method
outperforms previously published results on an established benchmark domain.Comment: 6 page
Semantically-Enhanced Information Extraction
Information Extraction using Natural Language Processing (NLP) produces entities along with some of the relationships that may exist among them. To be semantically useful, however, such discrete extractions must be put into context through some form of intelligent analysis. This paper1,2 offers a two-part architecture that employs the statistical methods of traditional NLP to extract discrete information elements in a relatively domain-agnostic manner, which are then injected into an inference-enabled environment where they can be semantically analyzed. Within this semantic environment, extractions are woven into the contextual fabric of a user-provided, domain-centric ontology where users together with user-provided logic can analyze these extractions within a more contextually complete picture. Our demonstration system infers the possibility of a terrorist plot by extracting key events and relationships from a collection of news articles and intelligence reports
Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection
Event detection is a crucial information extraction task in many domains,
such as Wikipedia or news. The task typically relies on trigger detection (TD)
-- identifying token spans in the text that evoke specific events. While the
notion of triggers should ideally be universal across domains, domain transfer
for TD from high- to low-resource domains results in significant performance
drops. We address the problem of negative transfer in TD by coupling triggers
between domains using subject-object relations obtained from a rule-based open
information extraction (OIE) system. We demonstrate that OIE relations injected
through multi-task training can act as mediators between triggers in different
domains, enhancing zero- and few-shot TD domain transfer and reducing
performance drops, in particular when transferring from a high-resource source
domain (Wikipedia) to a low(er)-resource target domain (news). Additionally, we
combine this improved transfer with masked language modeling on the target
domain, observing further TD transfer gains. Finally, we demonstrate that the
gains are robust to the choice of the OIE system.Comment: Accepted at EACL 2024 Finding
Validating Large Language Models with ReLM
Although large language models (LLMs) have been touted for their ability to
generate natural-sounding text, there are growing concerns around possible
negative effects of LLMs such as data memorization, bias, and inappropriate
language. Unfortunately, the complexity and generation capacities of LLMs make
validating (and correcting) such concerns difficult. In this work, we introduce
ReLM, a system for validating and querying LLMs using standard regular
expressions. ReLM formalizes and enables a broad range of language model
evaluations, reducing complex evaluation rules to simple regular expression
queries. Our results exploring queries surrounding memorization, gender bias,
toxicity, and language understanding show that ReLM achieves up to 15x higher
system efficiency, 2.5x data efficiency, and increased statistical and
prompt-tuning coverage compared to state-of-the-art ad-hoc queries. ReLM offers
a competitive and general baseline for the increasingly important problem of
LLM validation
Inference of Regular Expressions for Text Extraction from Examples
A large class of entity extraction tasks from text that is either semistructured or fully unstructured may be addressed by regular expressions, because in many practical cases the relevant entities follow an underlying syntactical pattern and this pattern may be described by a regular expression. In this work we consider the long-standing problem of synthesizing such expressions automatically, based solely on examples of the desired behavior. We present the design and implementation of a system capable of addressing extraction tasks of realistic complexity. Our system is based on an evolutionary procedure carefully tailored to the specific needs of regular expression generation by examples. The procedure executes a search driven by a multiobjective optimization strategy aimed at simultaneously improving multiple performance indexes of candidate solutions while at the same time ensuring an adequate exploration of the huge solution space. We assess our proposal experimentally in great depth, on a number of challenging datasets. The accuracy of the obtained solutions seems to be adequate for practical usage and improves over earlier proposals significantly. Most importantly, our results are highly competitive even with respect to human operators. A prototype is available as a web application at http://regex.inginf.units.it
- …