4 research outputs found
An Adversarial Non-Autoregressive Model for Text Generation with Incomplete Information
Non-autoregressive models have been widely studied in the Complete
Information Scenario (CIS), in which the input has complete information of
corresponding output. However, their explorations in the Incomplete Information
Scenario (IIS) are extremely limited. Our analyses reveal that the IIS's
incomplete input information will augment the inherent limitations of existing
non-autoregressive models trained under Maximum Likelihood Estimation. In this
paper, we propose for the IIS an Adversarial Non-autoregressive Transformer
(ANT) which has two features: 1) Position-Aware Self-Modulation to provide more
reasonable hidden representations, and 2) Dependency Feed Forward Network to
strengthen its capacity in dependency modeling. We compare ANT with other
mainstream models in the IIS and demonstrate that ANT can achieve comparable
performance with much fewer decoding iterations. Furthermore, we show its great
potential in various applications like latent interpolation and semi-supervised
learning
Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient
Backtracking linesearch is the de facto approach for minimizing continuously
differentiable functions with locally Lipschitz gradient. In recent years, it
has been shown that in the convex setting it is possible to avoid linesearch
altogether, and to allow the stepsize to adapt based on a local smoothness
estimate without any backtracks or evaluations of the function value. In this
work we propose an adaptive proximal gradient method, adaPG, that uses novel
estimates of the local smoothness modulus which leads to less conservative
stepsize updates and that can additionally cope with nonsmooth terms. This idea
is extended to the primal-dual setting where an adaptive three term primal-dual
algorithm, adaPD, is proposed which can be viewed as an extension of the PDHG
method. Moreover, in this setting the ``essentially'' fully adaptive variant
adaPD is proposed that avoids evaluating the linear operator norm by
invoking a backtracking procedure, that, remarkably, does not require extra
gradient evaluations. Numerical simulations demonstrate the effectiveness of
the proposed algorithms compared to the state of the art
A relaxed inertial forward-backward-forward algorithm for solving monotone inclusions with application to GANs
We introduce a relaxed inertial forward-backward-forward (RIFBF) splitting algorithm for approaching the set of zeros of the sum of a maximally monotone operator and a single-valued monotone and Lipschitz continuous operator. This work aims to extend Tseng's forward-backward-forward method by both using inertial effects as well as relaxation parameters. We formulate first a second order dynamical system that approaches the solution set of the monotone inclusion problem to be solved and provide an asymptotic analysis for its trajectories. We provide for RIFBF, which follows by explicit time discretization, a convergence analysis in the general monotone case as well as when applied to the solving of pseudo-monotone variational inequalities. We illustrate the proposed method by applications to a bilinear saddle point problem, in the context of which we also emphasize the interplay between the inertial and the relaxation parameters, and to the training of Generative Adversarial Networks (GANs)