38 research outputs found
Dual Learning: Theoretical Study and an Algorithmic Extension
Dual learning has been successfully applied in many machine learning
applications including machine translation, image-to-image transformation, etc.
The high-level idea of dual learning is very intuitive: if we map an from
one domain to another and then map it back, we should recover the original .
Although its effectiveness has been empirically verified, theoretical
understanding of dual learning is still very limited. In this paper, we aim at
understanding why and when dual learning works. Based on our theoretical
analysis, we further extend dual learning by introducing more related mappings
and propose multi-step dual learning, in which we leverage feedback signals
from additional domains to improve the qualities of the mappings. We prove that
multi-step dual learn-ing can boost the performance of standard dual learning
under mild conditions. Experiments on WMT 14 EnglishGerman and
MultiUNEnglishFrench translations verify our theoretical
findings on dual learning, and the results on the translations among English,
French, and Spanish of MultiUN demonstrate the effectiveness of multi-step dual
learning.Comment: 11 pages, 2 figure
Incentivizing High-quality Content from Heterogeneous Users: On the Existence of Nash Equilibrium
We study the existence of pure Nash equilibrium (PNE) for the mechanisms used
in Internet services (e.g., online reviews and question-answer websites) to
incentivize users to generate high-quality content. Most existing work assumes
that users are homogeneous and have the same ability. However, real-world users
are heterogeneous and their abilities can be very different from each other due
to their diverse background, culture, and profession. In this work, we consider
heterogeneous users with the following framework: (1) the users are
heterogeneous and each of them has a private type indicating the best quality
of the content she can generate; (2) there is a fixed amount of reward to
allocate to the participated users. Under this framework, we study the
existence of pure Nash equilibrium of several mechanisms composed by different
allocation rules, action spaces, and information settings. We prove the
existence of PNE for some mechanisms and the non-existence of PNE for some
mechanisms. We also discuss how to find a PNE for those mechanisms with PNE
either through a constructive way or a search algorithm
Image-to-Image Translation with Multi-Path Consistency Regularization
Image translation across different domains has attracted much attention in
both machine learning and computer vision communities. Taking the translation
from source domain to target domain as an
example, existing algorithms mainly rely on two kinds of loss for training: One
is the discrimination loss, which is used to differentiate images generated by
the models and natural images; the other is the reconstruction loss, which
measures the difference between an original image and the reconstructed version
through translation. In this
work, we introduce a new kind of loss, multi-path consistency loss, which
evaluates the differences between direct translation
and indirect translation
with as an
auxiliary domain, to regularize training. For multi-domain translation (at
least, three) which focuses on building translation models between any two
domains, at each training iteration, we randomly select three domains, set them
respectively as the source, auxiliary and target domains, build the multi-path
consistency loss and optimize the network. For two-domain translation, we need
to introduce an additional auxiliary domain and construct the multi-path
consistency loss. We conduct various experiments to demonstrate the
effectiveness of our proposed methods, including face-to-face translation,
paint-to-photo translation, and de-raining/de-noising translation.Comment: 8 pages, 6 figures. Accepted by the 28th International Joint
Conference on Artificial Intelligence (IJCAI-2019
Thompson Sampling for Budgeted Multi-armed Bandits
Thompson sampling is one of the earliest randomized algorithms for
multi-armed bandits (MAB). In this paper, we extend the Thompson sampling to
Budgeted MAB, where there is random cost for pulling an arm and the total cost
is constrained by a budget. We start with the case of Bernoulli bandits, in
which the random rewards (costs) of an arm are independently sampled from a
Bernoulli distribution. To implement the Thompson sampling algorithm in this
case, at each round, we sample two numbers from the posterior distributions of
the reward and cost for each arm, obtain their ratio, select the arm with the
maximum ratio, and then update the posterior distributions. We prove that the
distribution-dependent regret bound of this algorithm is , where
denotes the budget. By introducing a Bernoulli trial, we further extend this
algorithm to the setting that the rewards (costs) are drawn from general
distributions, and prove that its regret bound remains almost the same. Our
simulation results demonstrate the effectiveness of the proposed algorithm
Conditional Image-to-Image Translation
Image-to-image translation tasks have been widely investigated with
Generative Adversarial Networks (GANs) and dual learning. However, existing
models lack the ability to control the translated results in the target domain
and their results usually lack of diversity in the sense that a fixed image
usually leads to (almost) deterministic translation result. In this paper, we
study a new problem, conditional image-to-image translation, which is to
translate an image from the source domain to the target domain conditioned on a
given image in the target domain. It requires that the generated image should
inherit some domain-specific features of the conditional image from the target
domain. Therefore, changing the conditional image in the target domain will
lead to diverse translation results for a fixed input image from the source
domain, and therefore the conditional input image helps to control the
translation results. We tackle this problem with unpaired data based on GANs
and dual learning. We twist two conditional translation models (one translation
from A domain to B domain, and the other one from B domain to A domain)
together for inputs combination and reconstruction while preserving domain
independent features. We carry out experiments on men's faces from-to women's
faces translation and edges to shoes&bags translations. The results demonstrate
the effectiveness of our proposed method.Comment: 9 pages, 9 figures, IEEE Conference on Computer Vision and Pattern
Recognition (CVPR
Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation
Image-to-image translation tasks have been widely investigated with
Generative Adversarial Networks (GANs). However, existing approaches are mostly
designed in an unsupervised manner while little attention has been paid to
domain information within unpaired data. In this paper, we treat domain
information as explicit supervision and design an unpaired image-to-image
translation framework, Domain-supervised GAN (DosGAN), which takes the first
step towards the exploration of explicit domain supervision. In contrast to
representing domain characteristics using different generators or domain codes,
we pre-train a classification network to explicitly classify the domain of an
image. After pre-training, this network is used to extract the domain-specific
features of each image. Such features, together with the domain-independent
features extracted by another encoder (shared across different domains), are
used to generate image in target domain. Extensive experiments on multiple
facial attribute translation, multiple identity translation, multiple season
translation and conditional edges-to-shoes/handbags demonstrate the
effectiveness of our method. In addition, we can transfer the domain-specific
feature extractor obtained on the Facescrub dataset with domain supervision
information to unseen domains, such as faces in the CelebA dataset. We also
succeed in achieving conditional translation with any two images in CelebA,
while previous models like StarGAN cannot handle this task.Comment: Accepted by IEEE Transaction on Pattern Analysis and Machine
Intelligence (TPAMI).13 pages, 11 figures, 7 Table
Dual Supervised Learning
Many supervised learning tasks are emerged in dual forms, e.g.,
English-to-French translation vs. French-to-English translation, speech
recognition vs. text to speech, and image classification vs. image generation.
Two dual tasks have intrinsic connections with each other due to the
probabilistic correlation between their models. This connection is, however,
not effectively utilized today, since people usually train the models of two
dual tasks separately and independently. In this work, we propose training the
models of two dual tasks simultaneously, and explicitly exploiting the
probabilistic correlation between them to regularize the training process. For
ease of reference, we call the proposed approach \emph{dual supervised
learning}. We demonstrate that dual supervised learning can improve the
practical performances of both tasks, for various applications including
machine translation, image processing, and sentiment analysis.Comment: ICML 201
Efficient Bidirectional Neural Machine Translation
The encoder-decoder based neural machine translation usually generates a
target sequence token by token from left to right. Due to error propagation,
the tokens in the right side of the generated sequence are usually of poorer
quality than those in the left side. In this paper, we propose an efficient
method to generate a sequence in both left-to-right and right-to-left manners
using a single encoder and decoder, combining the advantages of both generation
directions. Experiments on three translation tasks show that our method
achieves significant improvements over conventional unidirectional approach.
Compared with ensemble methods that train and combine two models with different
generation directions, our method saves 50% model parameters and about 40%
training time, and also improve inference speed
Learning to Teach with Dynamic Loss Functions
Teaching is critical to human society: it is with teaching that prospective
students are educated and human civilization can be inherited and advanced. A
good teacher not only provides his/her students with qualified teaching
materials (e.g., textbooks), but also sets up appropriate learning objectives
(e.g., course projects and exams) considering different situations of a
student. When it comes to artificial intelligence, treating machine learning
models as students, the loss functions that are optimized act as perfect
counterparts of the learning objective set by the teacher. In this work, we
explore the possibility of imitating human teaching behaviors by dynamically
and automatically outputting appropriate loss functions to train machine
learning models. Different from typical learning settings in which the loss
function of a machine learning model is predefined and fixed, in our framework,
the loss function of a machine learning model (we call it student) is defined
by another machine learning model (we call it teacher). The ultimate goal of
teacher model is cultivating the student to have better performance measured on
development dataset. Towards that end, similar to human teaching, the teacher,
a parametric model, dynamically outputs different loss functions that will be
used and optimized by its student model at different training stages. We
develop an efficient learning method for the teacher model that makes gradient
based optimization possible, exempt of the ineffective solutions such as policy
optimization. We name our method as "learning to teach with dynamic loss
functions" (L2T-DLF for short). Extensive experiments on real world tasks
including image classification and neural machine translation demonstrate that
our method significantly improves the quality of various student models.Comment: NIPS 201
Adversarial Neural Machine Translation
In this paper, we study a new learning paradigm for Neural Machine
Translation (NMT). Instead of maximizing the likelihood of the human
translation as in previous works, we minimize the distinction between human
translation and the translation given by an NMT model. To achieve this goal,
inspired by the recent success of generative adversarial networks (GANs), we
employ an adversarial training architecture and name it as Adversarial-NMT. In
Adversarial-NMT, the training of the NMT model is assisted by an adversary,
which is an elaborately designed Convolutional Neural Network (CNN). The goal
of the adversary is to differentiate the translation result generated by the
NMT model from that by human. The goal of the NMT model is to produce high
quality translations so as to cheat the adversary. A policy gradient method is
leveraged to co-train the NMT model and the adversary. Experimental results on
EnglishFrench and GermanEnglish translation tasks
show that Adversarial-NMT can achieve significantly better translation quality
than several strong baselines.Comment: ACML 201