57,795 research outputs found
Generating Text with Deep Reinforcement Learning
We introduce a novel schema for sequence to sequence learning with a Deep
Q-Network (DQN), which decodes the output sequence iteratively. The aim here is
to enable the decoder to first tackle easier portions of the sequences, and
then turn to cope with difficult parts. Specifically, in each iteration, an
encoder-decoder Long Short-Term Memory (LSTM) network is employed to, from the
input sequence, automatically create features to represent the internal states
of and formulate a list of potential actions for the DQN. Take rephrasing a
natural sentence as an example. This list can contain ranked potential words.
Next, the DQN learns to make decision on which action (e.g., word) will be
selected from the list to modify the current decoded sequence. The newly
modified output sequence is subsequently used as the input to the DQN for the
next decoding iteration. In each iteration, we also bias the reinforcement
learning's attention to explore sequence portions which are previously
difficult to be decoded. For evaluation, the proposed strategy was trained to
decode ten thousands natural sentences. Our experiments indicate that, when
compared to a left-to-right greedy beam search LSTM decoder, the proposed
method performed competitively well when decoding sentences from the training
set, but significantly outperformed the baseline when decoding unseen
sentences, in terms of BLEU score obtained.Comment: Accepted to the NIPS2015 Deep Reinforcement Learning Worksho
Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model
Recently, generating adversarial examples has become an important means of
measuring robustness of a deep learning model. Adversarial examples help us
identify the susceptibilities of the model and further counter those
vulnerabilities by applying adversarial training techniques. In natural
language domain, small perturbations in the form of misspellings or paraphrases
can drastically change the semantics of the text. We propose a reinforcement
learning based approach towards generating adversarial examples in black-box
settings. We demonstrate that our method is able to fool well-trained models
for (a) IMDB sentiment classification task and (b) AG's news corpus news
categorization task with significantly high success rates. We find that the
adversarial examples generated are semantics-preserving perturbations to the
original text.Comment: 16 pages, 3 figures, ECML PKDD 201
AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions
In the last two decades, the landscape of text generation has undergone
tremendous changes and is being reshaped by the success of deep learning. New
technologies for text generation ranging from template-based methods to neural
network-based methods emerged. Meanwhile, the research objectives have also
changed from generating smooth and coherent sentences to infusing personalized
traits to enrich the diversification of newly generated content. With the rapid
development of text generation solutions, one comprehensive survey is urgent to
summarize the achievements and track the state of the arts. In this survey
paper, we present the general systematical framework, illustrate the widely
utilized models and summarize the classic applications of text generation.Comment: Accepted by IEEE UIC 201
A Comprehensive Survey of Deep Learning for Image Captioning
Generating a description of an image is called image captioning. Image
captioning requires to recognize the important objects, their attributes and
their relationships in an image. It also needs to generate syntactically and
semantically correct sentences. Deep learning-based techniques are capable of
handling the complexities and challenges of image captioning. In this survey
paper, we aim to present a comprehensive review of existing deep learning-based
image captioning techniques. We discuss the foundation of the techniques to
analyze their performances, strengths and limitations. We also discuss the
datasets and the evaluation metrics popularly used in deep learning based
automatic image captioning.Comment: 36 Pages, Accepted as a Journal Paper in ACM Computing Surveys
(October 2018
Image Captioning based on Deep Reinforcement Learning
Recently it has shown that the policy-gradient methods for reinforcement
learning have been utilized to train deep end-to-end systems on natural
language processing tasks. What's more, with the complexity of understanding
image content and diverse ways of describing image content in natural language,
image captioning has been a challenging problem to deal with. To the best of
our knowledge, most state-of-the-art methods follow a pattern of sequential
model, such as recurrent neural networks (RNN). However, in this paper, we
propose a novel architecture for image captioning with deep reinforcement
learning to optimize image captioning tasks. We utilize two networks called
"policy network" and "value network" to collaboratively generate the captions
of images. The experiments are conducted on Microsoft COCO dataset, and the
experimental results have verified the effectiveness of the proposed method
OptiGAN: Generative Adversarial Networks for Goal Optimized Sequence Generation
One of the challenging problems in sequence generation tasks is the optimized
generation of sequences with specific desired goals. Current sequential
generative models mainly generate sequences to closely mimic the training data,
without direct optimization of desired goals or properties specific to the
task. We introduce OptiGAN, a generative model that incorporates both
Generative Adversarial Networks (GAN) and Reinforcement Learning (RL) to
optimize desired goal scores using policy gradients. We apply our model to text
and real-valued sequence generation, where our model is able to achieve higher
desired scores out-performing GAN and RL baselines, while not sacrificing
output sample diversity.Comment: Preprint for accepted conference paper at International Joint
Conference on Neural Networks (IJCNN) 202
Music Generation by Deep Learning - Challenges and Directions
In addition to traditional tasks such as prediction, classification and
translation, deep learning is receiving growing attention as an approach for
music generation, as witnessed by recent research groups such as Magenta at
Google and CTRL (Creator Technology Research Lab) at Spotify. The motivation is
in using the capacity of deep learning architectures and training techniques to
automatically learn musical styles from arbitrary musical corpora and then to
generate samples from the estimated distribution. However, a direct application
of deep learning to generate content rapidly reaches limits as the generated
content tends to mimic the training set without exhibiting true creativity.
Moreover, deep learning architectures do not offer direct ways for controlling
generation (e.g., imposing some tonality or other arbitrary constraints).
Furthermore, deep learning architectures alone are autistic automata which
generate music autonomously without human user interaction, far from the
objective of interactively assisting musicians to compose and refine music.
Issues such as: control, structure, creativity and interactivity are the focus
of our analysis. In this paper, we select some limitations of a direct
application of deep learning to music generation, analyze why the issues are
not fulfilled and how to address them by possible approaches. Various examples
of recent systems are cited as examples of promising directions.Comment: 17 pages. arXiv admin note: substantial text overlap with
arXiv:1709.01620. Accepted for publication in Special Issue on Deep learning
for music and audio, Neural Computing & Applications, Springer Nature, 201
Multimodal Storytelling via Generative Adversarial Imitation Learning
Deriving event storylines is an effective summarization method to succinctly
organize extensive information, which can significantly alleviate the pain of
information overload. The critical challenge is the lack of widely recognized
definition of storyline metric. Prior studies have developed various approaches
based on different assumptions about users' interests. These works can extract
interesting patterns, but their assumptions do not guarantee that the derived
patterns will match users' preference. On the other hand, their exclusiveness
of single modality source misses cross-modality information. This paper
proposes a method, multimodal imitation learning via generative adversarial
networks(MIL-GAN), to directly model users' interests as reflected by various
data. In particular, the proposed model addresses the critical challenge by
imitating users' demonstrated storylines. Our proposed model is designed to
learn the reward patterns given user-provided storylines and then applies the
learned policy to unseen data. The proposed approach is demonstrated to be
capable of acquiring the user's implicit intent and outperforming competing
methods by a substantial margin with a user study.Comment: IJCAI 201
A Survey on Dialogue Systems: Recent Advances and New Frontiers
Dialogue systems have attracted more and more attention. Recent advances on
dialogue systems are overwhelmingly contributed by deep learning techniques,
which have been employed to enhance a wide range of big data applications such
as computer vision, natural language processing, and recommender systems. For
dialogue systems, deep learning can leverage a massive amount of data to learn
meaningful feature representations and response generation strategies, while
requiring a minimum amount of hand-crafting. In this article, we give an
overview to these recent advances on dialogue systems from various perspectives
and discuss some possible research directions. In particular, we generally
divide existing dialogue systems into task-oriented and non-task-oriented
models, then detail how deep learning techniques help them with representative
algorithms and finally discuss some appealing research directions that can
bring the dialogue system research into a new frontier.Comment: 13 pages. arXiv admin note: text overlap with arXiv:1703.01008 by
other author
Multi-Task Generative Adversarial Nets with Shared Memory for Cross-Domain Coordination Control
Generating sequential decision process from huge amounts of measured process
data is a future research direction for collaborative factory automation,
making full use of those online or offline process data to directly design
flexible make decisions policy, and evaluate performance. The key challenges
for the sequential decision process is to online generate sequential
decision-making policy directly, and transferring knowledge across tasks
domain. Most multi-task policy generating algorithms often suffer from
insufficient generating cross-task sharing structure at discrete-time nonlinear
systems with applications. This paper proposes the multi-task generative
adversarial nets with shared memory for cross-domain coordination control,
which can generate sequential decision policy directly from raw sensory input
of all of tasks, and online evaluate performance of system actions in
discrete-time nonlinear systems. Experiments have been undertaken using a
professional flexible manufacturing testbed deployed within a smart factory of
Weichai Power in China. Results on three groups of discrete-time nonlinear
control tasks show that our proposed model can availably improve the
performance of task with the help of other related tasks
- …