9,645 research outputs found
Candidate sentence selection for language learning exercises: from a comprehensive framework to an empirical evaluation
We present a framework and its implementation relying on Natural Language
Processing methods, which aims at the identification of exercise item
candidates from corpora. The hybrid system combining heuristics and machine
learning methods includes a number of relevant selection criteria. We focus on
two fundamental aspects: linguistic complexity and the dependence of the
extracted sentences on their original context. Previous work on exercise
generation addressed these two criteria only to a limited extent, and a refined
overall candidate sentence selection framework appears also to be lacking. In
addition to a detailed description of the system, we present the results of an
empirical evaluation conducted with language teachers and learners which
indicate the usefulness of the system for educational purposes. We have
integrated our system into a freely available online learning platform.Comment: To appear in Traitement Automatique des Langues (TAL) Journal,
Special issue on NLP for Learning and Teachin
Predicting the Relative Difficulty of Single Sentences With and Without Surrounding Context
The problem of accurately predicting relative reading difficulty across a set
of sentences arises in a number of important natural language applications,
such as finding and curating effective usage examples for intelligent language
tutoring systems. Yet while significant research has explored document- and
passage-level reading difficulty, the special challenges involved in assessing
aspects of readability for single sentences have received much less attention,
particularly when considering the role of surrounding passages. We introduce
and evaluate a novel approach for estimating the relative reading difficulty of
a set of sentences, with and without surrounding context. Using different sets
of lexical and grammatical features, we explore models for predicting pairwise
relative difficulty using logistic regression, and examine rankings generated
by aggregating pairwise difficulty labels using a Bayesian rating system to
form a final ranking. We also compare rankings derived for sentences assessed
with and without context, and find that contextual features can help predict
differences in relative difficulty judgments across these two conditions.Comment: EMNLP 2016 Long Pape
Text Analysis in Adversarial Settings: Does Deception Leave a Stylistic Trace?
Textual deception constitutes a major problem for online security. Many
studies have argued that deceptiveness leaves traces in writing style, which
could be detected using text classification techniques. By conducting an
extensive literature review of existing empirical work, we demonstrate that
while certain linguistic features have been indicative of deception in certain
corpora, they fail to generalize across divergent semantic domains. We suggest
that deceptiveness as such leaves no content-invariant stylistic trace, and
textual similarity measures provide superior means of classifying texts as
potentially deceptive. Additionally, we discuss forms of deception beyond
semantic content, focusing on hiding author identity by writing style
obfuscation. Surveying the literature on both author identification and
obfuscation techniques, we conclude that current style transformation methods
fail to achieve reliable obfuscation while simultaneously ensuring semantic
faithfulness to the original text. We propose that future work in style
transformation should pay particular attention to disallowing semantically
drastic changes.Comment: 35 pages To appear in ACM Computing Surveys (CSUR
Clinically Accurate Chest X-Ray Report Generation
The automatic generation of radiology reports given medical radiographs has
significant potential to operationally and improve clinical patient care. A
number of prior works have focused on this problem, employing advanced methods
from computer vision and natural language generation to produce readable
reports. However, these works often fail to account for the particular nuances
of the radiology domain, and, in particular, the critical importance of
clinical accuracy in the resulting generated reports. In this work, we present
a domain-aware automatic chest X-ray radiology report generation system which
first predicts what topics will be discussed in the report, then conditionally
generates sentences corresponding to these topics. The resulting system is
fine-tuned using reinforcement learning, considering both readability and
clinical accuracy, as assessed by the proposed Clinically Coherent Reward. We
verify this system on two datasets, Open-I and MIMIC-CXR, and demonstrate that
our model offers marked improvements on both language generation metrics and
CheXpert assessed accuracy over a variety of competitive baselines
SnapToGrid: From Statistical to Interpretable Models for Biomedical Information Extraction
We propose an approach for biomedical information extraction that marries the
advantages of machine learning models, e.g., learning directly from data, with
the benefits of rule-based approaches, e.g., interpretability. Our approach
starts by training a feature-based statistical model, then converts this model
to a rule-based variant by converting its features to rules, and "snapping to
grid" the feature weights to discrete votes. In doing so, our proposal takes
advantage of the large body of work in machine learning, but it produces an
interpretable model, which can be directly edited by experts. We evaluate our
approach on the BioNLP 2009 event extraction task. Our results show that there
is a small performance penalty when converting the statistical model to rules,
but the gain in interpretability compensates for that: with minimal effort,
human experts improve this model to have similar performance to the statistical
model that served as starting point
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
Sentence simplification aims to improve readability and understandability,
based on several operations such as splitting, deletion, and paraphrasing.
However, a valid simplified sentence should also be logically entailed by its
input sentence. In this work, we first present a strong pointer-copy mechanism
based sequence-to-sequence sentence simplification model, and then improve its
entailment and paraphrasing capabilities via multi-task learning with related
auxiliary tasks of entailment and paraphrase generation. Moreover, we propose a
novel 'multi-level' layered soft sharing approach where each auxiliary task
shares different (higher versus lower) level layers of the sentence
simplification model, depending on the task's semantic versus lexico-syntactic
nature. We also introduce a novel multi-armed bandit based training approach
that dynamically learns how to effectively switch across tasks during
multi-task learning. Experiments on multiple popular datasets demonstrate that
our model outperforms competitive simplification systems in SARI and FKGL
automatic metrics, and human evaluation. Further, we present several ablation
analyses on alternative layer sharing methods, soft versus hard sharing,
dynamic multi-armed bandit sampling approaches, and our model's learned
entailment and paraphrasing skills.Comment: COLING 2018 (15 pages
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Current lexical simplification approaches rely heavily on heuristics and
corpus level features that do not always align with human judgment. We create a
human-rated word-complexity lexicon of 15,000 English words and propose a novel
neural readability ranking model with a Gaussian-based feature vectorization
layer that utilizes these human ratings to measure the complexity of any given
word or phrase. Our model performs better than the state-of-the-art systems for
different lexical simplification tasks and evaluation datasets. Additionally,
we also produce SimplePPDB++, a lexical resource of over 10 million simplifying
paraphrase rules, by applying our model to the Paraphrase Database (PPDB).Comment: 12 pages; EMNLP 201
Fake News Early Detection: An Interdisciplinary Study
Massive dissemination of fake news and its potential to erode democracy has
increased the demand for accurate fake news detection. Recent advancements in
this area have proposed novel techniques that aim to detect fake news by
exploring how it propagates on social networks. Nevertheless, to detect fake
news at an early stage, i.e., when it is published on a news outlet but not yet
spread on social media, one cannot rely on news propagation information as it
does not exist. Hence, there is a strong need to develop approaches that can
detect fake news by focusing on news content. In this paper, a theory-driven
model is proposed for fake news detection. The method investigates news content
at various levels: lexicon-level, syntax-level, semantic-level and
discourse-level. We represent news at each level, relying on well-established
theories in social and forensic psychology. Fake news detection is then
conducted within a supervised machine learning framework. As an
interdisciplinary research, our work explores potential fake news patterns,
enhances the interpretability in fake news feature engineering, and studies the
relationships among fake news, deception/disinformation, and clickbaits.
Experiments conducted on two real-world datasets indicate the proposed method
can outperform the state-of-the-art and enable fake news early detection when
there is limited content information.Comment: 25 page
Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme
In this paper, we propose a joint architecture that captures language, rhyme
and meter for sonnet modelling. We assess the quality of generated poems using
crowd and expert judgements. The stress and rhyme models perform very well, as
generated poems are largely indistinguishable from human-written poems. Expert
evaluation, however, reveals that a vanilla language model captures meter
implicitly, and that machine-generated poems still underperform in terms of
readability and emotion. Our research shows the importance expert evaluation
for poetry generation, and that future research should look beyond rhyme/meter
and focus on poetic language.Comment: 11 pages; ACL201
Catching Attention with Automatic Pull Quote Selection
Pull quotes are an effective component of a captivating news article. These
spans of text are selected from an article and provided with more salient
presentation, with the aim of attracting readers with intriguing phrases and
making the article more visually interesting. In this paper, we introduce the
novel task of automatic pull quote selection, construct a dataset, and
benchmark the performance of a number of approaches ranging from hand-crafted
features to state-of-the-art sentence embeddings to cross-task models. We show
that pre-trained Sentence-BERT embeddings outperform all other approaches,
however the benefit over n-gram models is marginal. By closely examining the
results of simple models, we also uncover many unexpected properties of pull
quotes that should serve as inspiration for future approaches. We believe the
benefits of exploring this problem further are clear: pull quotes have been
found to increase enjoyment and readability, shape reader perceptions, and
facilitate learning.Comment: 14 pages (11 + 3 for refs), 3 figures, 6 table
- …