16,979 research outputs found
Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction
We propose a joint event and temporal relation extraction model with shared
representation learning and structured prediction. The proposed method has two
advantages over existing work. First, it improves event representation by
allowing the event and relation modules to share the same contextualized
embeddings and neural representation learner. Second, it avoids error
propagation in the conventional pipeline systems by leveraging structured
inference and learning methods to assign both the event labels and the temporal
relation labels jointly. Experiments show that the proposed method can improve
both event extraction and temporal relation extraction over state-of-the-art
systems, with the end-to-end F1 improved by 10% and 6.8% on two benchmark
datasets respectively.Comment: Published at EMNLP'1
Word-Level Loss Extensions for Neural Temporal Relation Classification
Unsupervised pre-trained word embeddings are used effectively for many tasks
in natural language processing to leverage unlabeled textual data. Often these
embeddings are either used as initializations or as fixed word representations
for task-specific classification models. In this work, we extend our
classification model's task loss with an unsupervised auxiliary loss on the
word-embedding level of the model. This is to ensure that the learned word
representations contain both task-specific features, learned from the
supervised loss component, and more general features learned from the
unsupervised loss component. We evaluate our approach on the task of temporal
relation extraction, in particular, narrative containment relation extraction
from clinical records, and show that continued training of the embeddings on
the unsupervised objective together with the task objective gives better
task-specific embeddings, and results in an improvement over the state of the
art on the THYME dataset, using only a general-domain part-of-speech tagger as
linguistic resource.Comment: Accepted at the 27th International Conference on Computational
Linguistics (COLING 2018
Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction
We submitted two systems to the SemEval-2016 Task 12: Clinical TempEval
challenge, participating in Phase 1, where we identified text spans of time and
event expressions in clinical notes and Phase 2, where we predicted a relation
between an event and its parent document creation time.
For temporal entity extraction, we find that a joint inference-based approach
using structured prediction outperforms a vanilla recurrent neural network that
incorporates word embeddings trained on a variety of large clinical document
sets. For document creation time relations, we find that a combination of date
canonicalization and distant supervision rules for predicting relations on both
events and time expressions improves classification, though gains are limited,
likely due to the small scale of training data.Comment: NAACL HLT 2016, SemEval-2016 Task 12 submissio
Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
The past decade has seen an explosion in the amount of digital information
stored in electronic health records (EHR). While primarily designed for
archiving patient clinical information and administrative healthcare tasks,
many researchers have found secondary use of these records for various clinical
informatics tasks. Over the same period, the machine learning community has
seen widespread advances in deep learning techniques, which also have been
successfully applied to the vast amount of EHR data. In this paper, we review
these deep EHR systems, examining architectures, technical aspects, and
clinical applications. We also identify shortcomings of current techniques and
discuss avenues of future research for EHR-based deep learning.Comment: Accepted for publication with Journal of Biomedical and Health
Informatics: http://ieeexplore.ieee.org/abstract/document/8086133
Structured Minimally Supervised Learning for Neural Relation Extraction
We present an approach to minimally supervised relation extraction that
combines the benefits of learned representations and structured learning, and
accurately predicts sentence-level relation mentions given only
proposition-level supervision from a KB. By explicitly reasoning about missing
data during learning, our approach enables large-scale training of 1D
convolutional neural networks while mitigating the issue of label noise
inherent in distant supervision. Our approach achieves state-of-the-art results
on minimally supervised sentential relation extraction, outperforming a number
of baselines, including a competitive approach that uses the attention layer of
a purely neural model.Comment: Accepted to NAACL 2019. This version improves the model
description(present original "Bag-Size Adaptive Learning Rate" as "Bag-Size
Weighting Function"). No result/conclusion chang
Natural Language Processing for Information Extraction
With rise of digital age, there is an explosion of information in the form of
news, articles, social media, and so on. Much of this data lies in unstructured
form and manually managing and effectively making use of it is tedious, boring
and labor intensive. This explosion of information and need for more
sophisticated and efficient information handling tools gives rise to
Information Extraction(IE) and Information Retrieval(IR) technology.
Information Extraction systems takes natural language text as input and
produces structured information specified by certain criteria, that is relevant
to a particular application. Various sub-tasks of IE such as Named Entity
Recognition, Coreference Resolution, Named Entity Linking, Relation Extraction,
Knowledge Base reasoning forms the building blocks of various high end Natural
Language Processing (NLP) tasks such as Machine Translation, Question-Answering
System, Natural Language Understanding, Text Summarization and Digital
Assistants like Siri, Cortana and Google Now. This paper introduces Information
Extraction technology, its various sub-tasks, highlights state-of-the-art
research in various IE subtasks, current challenges and future research
directions.Comment: 24 pages, 1 figur
Deep Structured Neural Network for Event Temporal Relation Extraction
We propose a novel deep structured learning framework for event temporal
relation extraction. The model consists of 1) a recurrent neural network (RNN)
to learn scoring functions for pair-wise relations, and 2) a structured support
vector machine (SSVM) to make joint predictions. The neural network
automatically learns representations that account for long-term contexts to
provide robust features for the structured model, while the SSVM incorporates
domain knowledge such as transitive closure of temporal relations as
constraints to make better globally consistent decisions. By jointly training
the two components, our model combines the benefits of both data-driven
learning and knowledge exploitation. Experimental results on three high-quality
event temporal relation datasets (TCR, MATRES, and TB-Dense) demonstrate that
incorporated with pre-trained contextualized embeddings, the proposed model
achieves significantly better performances than the state-of-the-art methods on
all three datasets. We also provide thorough ablation studies to investigate
our model.Comment: This paper will be published in CoNLL 201
EventPlus: A Temporal Event Understanding Pipeline
We present EventPlus, a temporal event understanding pipeline that integrates
various state-of-the-art event understanding components including event trigger
and type detection, event argument detection, event duration and temporal
relation extraction. Event information, especially event temporal knowledge, is
a type of common sense knowledge that helps people understand how stories
evolve and provides predictive hints for future events. EventPlus as the first
comprehensive temporal event understanding pipeline provides a convenient tool
for users to quickly obtain annotations about events and their temporal
information for any user-provided document. Furthermore, we show EventPlus can
be easily adapted to other domains (e.g., biomedical domain). We make EventPlus
publicly available to facilitate event-related information extraction and
downstream applications
State of the Art, Evaluation and Recommendations regarding "Document Processing and Visualization Techniques"
Several Networks of Excellence have been set up in the framework of the
European FP5 research program. Among these Networks of Excellence, the NEMIS
project focuses on the field of Text Mining.
Within this field, document processing and visualization was identified as
one of the key topics and the WG1 working group was created in the NEMIS
project, to carry out a detailed survey of techniques associated with the text
mining process and to identify the relevant research topics in related research
areas.
In this document we present the results of this comprehensive survey. The
report includes a description of the current state-of-the-art and practice, a
roadmap for follow-up research in the identified areas, and recommendations for
anticipated technological development in the domain of text mining.Comment: 54 pages, Report of Working Group 1 for the European Network of
Excellence (NoE) in Text Mining and its Applications in Statistics (NEMIS
Structured Prediction as Translation between Augmented Natural Languages
We propose a new framework, Translation between Augmented Natural Languages
(TANL), to solve many structured prediction language tasks including joint
entity and relation extraction, nested named entity recognition, relation
classification, semantic role labeling, event extraction, coreference
resolution, and dialogue state tracking. Instead of tackling the problem by
training task-specific discriminative classifiers, we frame it as a translation
task between augmented natural languages, from which the task-relevant
information can be easily extracted. Our approach can match or outperform
task-specific models on all tasks, and in particular, achieves new
state-of-the-art results on joint entity and relation extraction (CoNLL04, ADE,
NYT, and ACE2005 datasets), relation classification (FewRel and TACRED), and
semantic role labeling (CoNLL-2005 and CoNLL-2012). We accomplish this while
using the same architecture and hyperparameters for all tasks and even when
training a single model to solve all tasks at the same time (multi-task
learning). Finally, we show that our framework can also significantly improve
the performance in a low-resource regime, thanks to better use of label
semantics
- …