3,150 research outputs found
Wyner VAE: Joint and Conditional Generation with Succinct Common Representation Learning
A new variational autoencoder (VAE) model is proposed that learns a succinct
common representation of two correlated data variables for conditional and
joint generation tasks. The proposed Wyner VAE model is based on two
information theoretic problems---distributed simulation and channel
synthesis---in which Wyner's common information arises as the fundamental limit
of the succinctness of the common representation. The Wyner VAE decomposes a
pair of correlated data variables into their common representation (e.g., a
shared concept) and local representations that capture the remaining randomness
(e.g., texture and style) in respective data variables by imposing the mutual
information between the data variables and the common representation as a
regularization term. The utility of the proposed approach is demonstrated
through experiments for joint and conditional generation with and without style
control using synthetic data and real images. Experimental results show that
learning a succinct common representation achieves better generative
performance and that the proposed model outperforms existing VAE variants and
the variational information bottleneck method.Comment: 24 pages, 18 figure
Denoising Autoencoders for fast Combinatorial Black Box Optimization
Estimation of Distribution Algorithms (EDAs) require flexible probability
models that can be efficiently learned and sampled. Autoencoders (AE) are
generative stochastic networks with these desired properties. We integrate a
special type of AE, the Denoising Autoencoder (DAE), into an EDA and evaluate
the performance of DAE-EDA on several combinatorial optimization problems with
a single objective. We asses the number of fitness evaluations as well as the
required CPU times. We compare the results to the performance to the Bayesian
Optimization Algorithm (BOA) and RBM-EDA, another EDA which is based on a
generative neural network which has proven competitive with BOA. For the
considered problem instances, DAE-EDA is considerably faster than BOA and
RBM-EDA, sometimes by orders of magnitude. The number of fitness evaluations is
higher than for BOA, but competitive with RBM-EDA. These results show that DAEs
can be useful tools for problems with low but non-negligible fitness evaluation
costs.Comment: corrected typos and small inconsistencie
Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
In high-dimensional model selection problems, penalized simple least-square
approaches have been extensively used. This paper addresses the question of
both robustness and efficiency of penalized model selection methods, and
proposes a data-driven weighted linear combination of convex loss functions,
together with weighted -penalty. It is completely data-adaptive and does
not require prior knowledge of the error distribution. The weighted
-penalty is used both to ensure the convexity of the penalty term and to
ameliorate the bias caused by the -penalty. In the setting with
dimensionality much larger than the sample size, we establish a strong oracle
property of the proposed method that possesses both the model selection
consistency and estimation efficiency for the true non-zero coefficients. As
specific examples, we introduce a robust method of composite L1-L2, and optimal
composite quantile method and evaluate their performance in both simulated and
real data examples
- …