77 research outputs found
Normalizing Flow with Variational Latent Representation
Normalizing flow (NF) has gained popularity over traditional maximum
likelihood based methods due to its strong capability to model complex data
distributions. However, the standard approach, which maps the observed data to
a normal distribution, has difficulty in handling data distributions with
multiple relatively isolated modes. To overcome this issue, we propose a new
framework based on variational latent representation to improve the practical
performance of NF. The idea is to replace the standard normal latent variable
with a more general latent representation, jointly learned via Variational
Bayes. For example, by taking the latent representation as a discrete sequence,
our framework can learn a Transformer model that generates the latent sequence
and an NF model that generates continuous data distribution conditioned on the
sequence. The resulting method is significantly more powerful than the standard
normalization flow approach for generating data distributions with multiple
modes. Extensive experiments have shown the advantages of NF with variational
latent representation.Comment: 24 pages, 7 figure
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
Particle-based variational inference (VI) minimizes the KL divergence between
model samples and the target posterior with gradient flow estimates. With the
popularity of Stein variational gradient descent (SVGD), the focus of
particle-based VI algorithms has been on the properties of functions in
Reproducing Kernel Hilbert Space (RKHS) to approximate the gradient flow.
However, the requirement of RKHS restricts the function class and algorithmic
flexibility. This paper remedies the problem by proposing a general framework
to obtain tractable functional gradient flow estimates. The functional gradient
flow in our framework can be defined by a general functional regularization
term that includes the RKHS norm as a special case. We use our framework to
propose a new particle-based VI algorithm: preconditioned functional gradient
flow (PFG). Compared with SVGD, the proposed method has several advantages:
larger function class; greater scalability in large particle-size scenarios;
better adaptation to ill-conditioned distributions; provable continuous-time
convergence in KL divergence. Non-linear function classes such as neural
networks can be incorporated to estimate the gradient flow. Both theory and
experiments have shown the effectiveness of our framework.Comment: 34 pages, 8 figure
Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo
To sample from a general target distribution beyond the
isoperimetric condition, Huang et al. (2023) proposed to perform sampling
through reverse diffusion, giving rise to Diffusion-based Monte Carlo (DMC).
Specifically, DMC follows the reverse SDE of a diffusion process that
transforms the target distribution to the standard Gaussian, utilizing a
non-parametric score estimation. However, the original DMC algorithm
encountered high gradient complexity, resulting in an exponential dependency on
the error tolerance of the obtained samples. In this paper, we
demonstrate that the high complexity of DMC originates from its redundant
design of score estimation, and proposed a more efficient algorithm, called
RS-DMC, based on a novel recursive score estimation method. In particular, we
first divide the entire diffusion process into multiple segments and then
formulate the score estimation step (at any time step) as a series of
interconnected mean estimation and sampling subproblems accordingly, which are
correlated in a recursive manner. Importantly, we show that with a proper
design of the segment decomposition, all sampling subproblems will only need to
tackle a strongly log-concave distribution, which can be very efficient to
solve using the Langevin-based samplers with a provably rapid convergence rate.
As a result, we prove that the gradient complexity of RS-DMC only has a
quasi-polynomial dependency on , which significantly improves
exponential gradient complexity in Huang et al. (2023). Furthermore, under
commonly used dissipative conditions, our algorithm is provably much faster
than the popular Langevin-based algorithms. Our algorithm design and
theoretical framework illuminate a novel direction for addressing sampling
problems, which could be of broader applicability in the community.Comment: 54 page
Disentangled Generative Causal Representation Learning
This paper proposes a Disentangled gEnerative cAusal Representation (DEAR)
learning method. Unlike existing disentanglement methods that enforce
independence of the latent variables, we consider the general case where the
underlying factors of interests can be causally correlated. We show that
previous methods with independent priors fail to disentangle causally
correlated factors. Motivated by this finding, we propose a new disentangled
learning method called DEAR that enables causal controllable generation and
causal representation learning. The key ingredient of this new formulation is
to use a structural causal model (SCM) as the prior for a bidirectional
generative model. The prior is then trained jointly with a generator and an
encoder using a suitable GAN loss incorporated with supervision. We provide
theoretical justification on the identifiability and asymptotic consistency of
the proposed method, which guarantees disentangled causal representation
learning under appropriate conditions. We conduct extensive experiments on both
synthesized and real data sets to demonstrate the effectiveness of DEAR in
causal controllable generation, and the benefits of the learned representations
for downstream tasks in terms of sample efficiency and distributional
robustness
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
Generative foundation models are susceptible to implicit biases that can
arise from extensive unsupervised training data. Such biases can produce
suboptimal samples, skewed outcomes, and unfairness, with potentially
significant repercussions. Consequently, aligning these models with human
ethics and preferences is an essential step toward ensuring their responsible
and effective deployment in real-world applications. Prior research has
primarily employed Reinforcement Learning from Human Feedback (RLHF) as a means
of addressing this problem, wherein generative models are fine-tuned using RL
algorithms guided by a human-feedback-informed reward model. However, the
inefficiencies and instabilities associated with RL algorithms frequently
present substantial obstacles to the successful alignment of generative models,
necessitating the development of a more robust and streamlined approach. To
this end, we introduce a new framework, Reward rAnked FineTuning (RAFT),
designed to align generative models more effectively. Utilizing a reward model
and a sufficient number of samples, our approach selects the high-quality
samples, discarding those that exhibit undesired behavior, and subsequently
assembles a streaming dataset. This dataset serves as the basis for aligning
the generative model and can be employed under both offline and online
settings. Notably, the sample generation process within RAFT is gradient-free,
rendering it compatible with black-box generators. Through extensive
experiments, we demonstrate that our proposed algorithm exhibits strong
performance in the context of both large language models and diffusion models
- …