5,550 research outputs found
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation
This paper surveys the current state of the art in Natural Language
Generation (NLG), defined as the task of generating text or speech from
non-linguistic input. A survey of NLG is timely in view of the changes that the
field has undergone over the past decade or so, especially in relation to new
(usually data-driven) methods, as well as new applications of NLG technology.
This survey therefore aims to (a) give an up-to-date synthesis of research on
the core tasks in NLG and the architectures adopted in which such tasks are
organised; (b) highlight a number of relatively recent research topics that
have arisen partly as a result of growing synergies between NLG and other areas
of artificial intelligence; (c) draw attention to the challenges in NLG
evaluation, relating them to similar challenges faced in other areas of Natural
Language Processing, with an emphasis on different evaluation methods and the
relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118
pages, 8 figures, 1 tabl
Text Style Transfer: A Review and Experimental Evaluation
The stylistic properties of text have intrigued computational linguistics
researchers in recent years. Specifically, researchers have investigated the
Text Style Transfer (TST) task, which aims to change the stylistic properties
of the text while retaining its style independent content. Over the last few
years, many novel TST algorithms have been developed, while the industry has
leveraged these algorithms to enable exciting TST applications. The field of
TST research has burgeoned because of this symbiosis. This article aims to
provide a comprehensive review of recent research efforts on text style
transfer. More concretely, we create a taxonomy to organize the TST models and
provide a comprehensive summary of the state of the art. We review the existing
evaluation methodologies for TST tasks and conduct a large-scale
reproducibility study where we experimentally benchmark 19 state-of-the-art TST
algorithms on two publicly available datasets. Finally, we expand on current
trends and provide new perspectives on the new and exciting developments in the
TST field
External Reasoning: Towards Multi-Large-Language-Models Interchangeable Assistance with Human Feedback
Memory is identified as a crucial human faculty that allows for the retention
of visual and linguistic information within the hippocampus and neurons in the
brain, which can subsequently be retrieved to address real-world challenges
that arise through a lifetime of learning. The resolution of complex AI tasks
through the application of acquired knowledge represents a stride toward the
realization of artificial general intelligence. However, despite the prevalence
of Large Language Models (LLMs) like GPT-3.5 and GPT-4 , which have displayed
remarkable capabilities in language comprehension, generation, interaction, and
reasoning, they are inhibited by constraints on context length that preclude
the processing of extensive, continually evolving knowledge bases. This paper
proposes that LLMs could be augmented through the selective integration of
knowledge from external repositories, and in doing so, introduces a novel
methodology for External Reasoning, exemplified by ChatPDF. Central to this
approach is the establishment of a tiered policy for \textbf{External Reasoning
based on Multiple LLM Interchange Assistance}, where the level of support
rendered is modulated across entry, intermediate, and advanced tiers based on
the complexity of the query, with adjustments made in response to human
feedback. A comprehensive evaluation of this methodology is conducted using
multiple LLMs and the results indicate state-of-the-art performance, surpassing
existing solutions including ChatPDF.com. Moreover, the paper emphasizes that
this approach is more efficient compared to the direct processing of full text
by LLMs.Comment: technical repor
Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems
Sharing ideas through communication with peers is the primary mode of human
interaction. Consequently, extensive research has been conducted in the area of
conversational AI, leading to an increase in the availability and diversity of
conversational tasks, datasets, and methods. However, with numerous tasks being
explored simultaneously, the current landscape of conversational AI becomes
fragmented. Therefore, initiating a well-thought-out model for a dialogue agent
can pose significant challenges for a practitioner. Towards highlighting the
critical ingredients needed for a practitioner to design a dialogue agent from
scratch, the current study provides a comprehensive overview of the primary
characteristics of a dialogue agent, the supporting tasks, their corresponding
open-domain datasets, and the methods used to benchmark these datasets. We
observe that different methods have been used to tackle distinct dialogue
tasks. However, building separate models for each task is costly and does not
leverage the correlation among the several tasks of a dialogue agent. As a
result, recent trends suggest a shift towards building unified foundation
models. To this end, we propose UNIT, a UNified dIalogue dataseT constructed
from conversations of existing datasets for different dialogue tasks capturing
the nuances for each of them. We also examine the evaluation strategies used to
measure the performance of dialogue agents and highlight the scope for future
research in the area of conversational AI.Comment: 21 pages, 3 figures, 3 table
STEER: Unified Style Transfer with Expert Reinforcement
While text style transfer has many applications across natural language
processing, the core premise of transferring from a single source style is
unrealistic in a real-world setting. In this work, we focus on arbitrary style
transfer: rewriting a text from an arbitrary, unknown style to a target style.
We propose STEER: Unified Style Transfer with Expert Reinforcement, a unified
frame-work developed to overcome the challenge of limited parallel data for
style transfer. STEER involves automatically generating a corpus of
style-transfer pairs using a product of experts during decoding. The generated
offline data is then used to pre-train an initial policy before switching to
online, off-policy reinforcement learning for further improvements via
fine-grained reward signals. STEER is unified and can transfer to multiple
target styles from an arbitrary, unknown source style, making it particularly
flexible and efficient.
Experimental results on a challenging dataset with text from a diverse set of
styles demonstrate state-of-the-art results compared to competitive baselines.
Remarkably, STEER outperforms the 175B parameter instruction-tuned GPT-3 on
overall style transfer quality, despite being 226 times smaller in size. We
also show STEER is robust, maintaining its style transfer capabilities on
out-of-domain data, and surpassing nearly all baselines across various styles.
The success of our method highlights the potential of RL algorithms when
augmented with controllable decoding to overcome the challenge of limited data
supervision.Comment: for associated code, see
https://github.com/shallinan1/STEERStyleTransfe
Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema
Task-oriented dialog systems empower users to accomplish their goals by
facilitating intuitive and expressive natural language interactions.
State-of-the-art approaches in task-oriented dialog systems formulate the
problem as a conditional sequence generation task and fine-tune pre-trained
causal language models in the supervised setting. This requires labeled
training data for each new domain or task, and acquiring such data is
prohibitively laborious and expensive, thus making it a bottleneck for scaling
systems to a wide range of domains. To overcome this challenge, we introduce a
novel Zero-Shot generalizable end-to-end Task-oriented Dialog system, ZS-ToD,
that leverages domain schemas to allow for robust generalization to unseen
domains and exploits effective summarization of the dialog history. We employ
GPT-2 as a backbone model and introduce a two-step training process where the
goal of the first step is to learn the general structure of the dialog data and
the second step optimizes the response generation as well as intermediate
outputs, such as dialog state and system actions. As opposed to
state-of-the-art systems that are trained to fulfill certain intents in the
given domains and memorize task-specific conversational patterns, ZS-ToD learns
generic task-completion skills by comprehending domain semantics via domain
schemas and generalizing to unseen domains seamlessly. We conduct an extensive
experimental evaluation on SGD and SGD-X datasets that span up to 20 unique
domains and ZS-ToD outperforms state-of-the-art systems on key metrics, with an
improvement of +17% on joint goal accuracy and +5 on inform. Additionally, we
present a detailed ablation study to demonstrate the effectiveness of the
proposed components and training mechanis
- …