2 research outputs found
Composite Logconcave Sampling with a Restricted Gaussian Oracle
We consider sampling from composite densities on of the form
for well-conditioned and convex (but
possibly non-smooth) , a family generalizing restrictions to a convex set,
through the abstraction of a restricted Gaussian oracle. For with condition
number , our algorithm runs in iterations, each querying a gradient of and a
restricted Gaussian oracle, to achieve total variation distance . The
restricted Gaussian oracle, which draws samples from a distribution whose
negative log-likelihood sums a quadratic and , has been previously studied
and is a natural extension of the proximal oracle used in composite
optimization. Our algorithm is conceptually simple and obtains stronger
provable guarantees and greater generality than existing methods for composite
sampling. We conduct experiments showing our algorithm vastly improves upon the
hit-and-run algorithm for sampling the restriction of a (non-diagonal) Gaussian
to the positive orthant
Structured Logconcave Sampling with a Restricted Gaussian Oracle
We give algorithms for sampling several structured logconcave families to
high accuracy. We further develop a reduction framework, inspired by proximal
point methods in convex optimization, which bootstraps samplers for regularized
densities to improve dependences on problem conditioning. A key ingredient in
our framework is the notion of a "restricted Gaussian oracle" (RGO) for , which is a sampler for distributions
whose negative log-likelihood sums a quadratic and . By combining our
reduction framework with our new samplers, we obtain the following bounds for
sampling structured distributions to total variation distance . For
composite densities , where has condition number
and convex (but possibly non-smooth) admits an RGO, we obtain a
mixing time of , matching the
state-of-the-art non-composite bound; no composite samplers with better mixing
than general-purpose logconcave samplers were previously known. For logconcave
finite sums , where
has condition number , we give a sampler querying gradient oracles to ; no
high-accuracy samplers with nontrivial gradient query complexity were
previously known. For densities with condition number , we give an
algorithm obtaining mixing time ,
improving the prior state-of-the-art by a logarithmic factor with a
significantly simpler analysis; we also show a zeroth-order algorithm attains
the same query complexity.Comment: 58 pages. The results of Section 5 of this paper, as well as an
empirical evaluation, appeared earlier as arXiv:2006.05976. This version
fixes an error in the proof of Theorem 1, see Section 1.