4,521 research outputs found
Evaluating prose style transfer with the Bible
In the prose style transfer task a system, provided with text input and a
target prose style, produces output which preserves the meaning of the input
text but alters the style. These systems require parallel data for evaluation
of results and usually make use of parallel data for training. Currently, there
are few publicly available corpora for this task. In this work, we identify a
high-quality source of aligned, stylistically distinct text in different
versions of the Bible. We provide a standardized split, into training,
development and testing data, of the public domain versions in our corpus. This
corpus is highly parallel since many Bible versions are included. Sentences are
aligned due to the presence of chapter and verse numbers within all versions of
the text. In addition to the corpus, we present the results, as measured by the
BLEU and PINC metrics, of several models trained on our data which can serve as
baselines for future research. While we present these data as a style transfer
corpus, we believe that it is of unmatched quality and may be useful for other
natural language tasks as well
Learning Criteria and Evaluation Metrics for Textual Transfer between Non-Parallel Corpora
We consider the problem of automatically generating textual paraphrases with
modified attributes or stylistic properties, focusing on the setting without
parallel data (Hu et al., 2017; Shen et al., 2017). This setting poses
challenges for learning and evaluation. We show that the metric of
post-transfer classification accuracy is insufficient on its own, and propose
additional metrics based on semantic content preservation and fluency. For
reliable evaluation, all three metric categories must be taken into account. We
contribute new loss functions and training strategies to address the new
metrics. Semantic preservation is addressed by adding a cyclic consistency loss
and a loss based on paraphrase pairs, while fluency is improved by integrating
losses based on style-specific language models. Automatic and manual evaluation
show large improvements over the baseline method of Shen et al. (2017). Our
hope is that these losses and metrics can be general and useful tools for a
range of textual transfer settings without parallel corpora
Text Style Transfer: A Review and Experimental Evaluation
The stylistic properties of text have intrigued computational linguistics
researchers in recent years. Specifically, researchers have investigated the
Text Style Transfer (TST) task, which aims to change the stylistic properties
of the text while retaining its style independent content. Over the last few
years, many novel TST algorithms have been developed, while the industry has
leveraged these algorithms to enable exciting TST applications. The field of
TST research has burgeoned because of this symbiosis. This article aims to
provide a comprehensive review of recent research efforts on text style
transfer. More concretely, we create a taxonomy to organize the TST models and
provide a comprehensive summary of the state of the art. We review the existing
evaluation methodologies for TST tasks and conduct a large-scale
reproducibility study where we experimentally benchmark 19 state-of-the-art TST
algorithms on two publicly available datasets. Finally, we expand on current
trends and provide new perspectives on the new and exciting developments in the
TST field
- …