621 research outputs found
Unsupervised Domain Adaptation with Copula Models
We study the task of unsupervised domain adaptation, where no labeled data
from the target domain is provided during training time. To deal with the
potential discrepancy between the source and target distributions, both in
features and labels, we exploit a copula-based regression framework. The
benefits of this approach are two-fold: (a) it allows us to model a broader
range of conditional predictive densities beyond the common exponential family,
(b) we show how to leverage Sklar's theorem, the essence of the copula
formulation relating the joint density to the copula dependency functions, to
find effective feature mappings that mitigate the domain mismatch. By
transforming the data to a copula domain, we show on a number of benchmark
datasets (including human emotion estimation), and using different regression
models for prediction, that we can achieve a more robust and accurate
estimation of target labels, compared to recently proposed feature
transformation (adaptation) methods.Comment: IEEE International Workshop On Machine Learning for Signal Processing
201
Neural Likelihoods via Cumulative Distribution Functions
We leverage neural networks as universal approximators of monotonic functions
to build a parameterization of conditional cumulative distribution functions
(CDFs). By the application of automatic differentiation with respect to
response variables and then to parameters of this CDF representation, we are
able to build black box CDF and density estimators. A suite of families is
introduced as alternative constructions for the multivariate case. At one
extreme, the simplest construction is a competitive density estimator against
state-of-the-art deep learning methods, although it does not provide an easily
computable representation of multivariate CDFs. At the other extreme, we have a
flexible construction from which multivariate CDF evaluations and
marginalizations can be obtained by a simple forward pass in a deep neural net,
but where the computation of the likelihood scales exponentially with
dimensionality. Alternatives in between the extremes are discussed. We evaluate
the different representations empirically on a variety of tasks involving tail
area probabilities, tail dependence and (partial) density estimation.Comment: 10 page
CVAE: Gaussian Copula-based VAE Differing Disentangled from Coupled Representations with Contrastive Posterior
We present a self-supervised variational autoencoder (VAE) to jointly learn
disentangled and dependent hidden factors and then enhance disentangled
representation learning by a self-supervised classifier to eliminate coupled
representations in a contrastive manner. To this end, a Contrastive Copula VAE
(CVAE) is introduced without relying on prior knowledge about data in the
probabilistic principle and involving strong modeling assumptions on the
posterior in the neural architecture. CVAE simultaneously factorizes the
posterior (evidence lower bound, ELBO) with total correlation (TC)-driven
decomposition for learning factorized disentangled representations and extracts
the dependencies between hidden features by a neural Gaussian copula for copula
coupled representations. Then, a self-supervised contrastive classifier
differentiates the disentangled representations from the coupled
representations, where a contrastive loss regularizes this contrastive
classification together with the TC loss for eliminating entangled factors and
strengthening disentangled representations. CVAE demonstrates a strong
effect in enhancing disentangled representation learning. CVAE further
contributes to improved optimization addressing the TC-based VAE instability
and the trade-off between reconstruction and representation
Scenario Generation for Cooling, Heating, and Power Loads Using Generative Moment Matching Networks
Scenario generations of cooling, heating, and power loads are of great
significance for the economic operation and stability analysis of integrated
energy systems. In this paper, a novel deep generative network is proposed to
model cooling, heating, and power load curves based on a generative moment
matching networks (GMMN) where an auto-encoder transforms high-dimensional load
curves into low-dimensional latent variables and the maximum mean discrepancy
represents the similarity metrics between the generated samples and the real
samples. After training the model, the new scenarios are generated by feeding
Gaussian noises to the scenario generator of the GMMN. Unlike the explicit
density models, the proposed GMMN does not need to artificially assume the
probability distribution of the load curves, which leads to stronger
universality. The simulation results show that the GMMN not only fits the
probability distribution of multi-class load curves well, but also accurately
captures the shape (e.g., large peaks, fast ramps, and fluctuation),
frequency-domain characteristics, and temporal-spatial correlations of cooling,
heating, and power loads. Furthermore, the energy consumption of generated
samples closely resembles that of real samples.Comment: This paper has been accepted by CSEE Journal of Power and Energy
System
Learning Invariant Representations for Deep Latent Variable Models
Deep latent variable models introduce a new class of generative models which are able to handle unstructured data and encode non-linear dependencies. Despite their known flexibility, these models are frequently not invariant against target-specific transformations. Therefore, they suffer from model mismatches and are challenging to interpret or control. We employ the concept of symmetry transformations from physics to formally describe these invariances. In this thesis, we investigate how we can model invariances when a symmetry transformation is either known or unknown. As a consequence, we make contributions in the domain of variable compression under side information and generative modelling. In our first contribution, we investigate the problem where a symmetry transformation is known yet not implicitly learned by the model. Specifically, we consider the task of estimating mutual information in the context of the deep information bottleneck which is not invariant against monotone transformations. To address this limitation, we extend the deep information bottleneck with a copula construction. In our second contribution, we address the problem of learning target-invariant subspaces for generative models. In this case, the symmetry transformation is unknown and has to be learned from data. We achieve this by formulating a deep information bottleneck with a target and a target-invariant subspace. To ensure invariance, we provide a continuous mutual information regulariser based on adversarial training. In our last contribution, we introduce an improved method for learning unknown symmetry transformations with cycle-consistency. To do so, we employ the equivalent deep information bottleneck method with a partitioned latent space. However, we ensure target-invariance by utilizing a cycle-consistency loss in the latent space. As a result, we overcome potential convergence issues introduced by adversarial training and are able to deal with mixed data. In summary, each of our presented models provide an attempt to better control and understand deep latent variables models by learning symmetry transformations. We demonstrated the effectiveness of our contributions with an extensive evaluation on both artificial and real-world experiments
- …