1,160 research outputs found
Convergence Rates for Non-Log-Concave Sampling and Log-Partition Estimation
Sampling from Gibbs distributions and
computing their log-partition function are fundamental tasks in statistics,
machine learning, and statistical physics. However, while efficient algorithms
are known for convex potentials , the situation is much more difficult in
the non-convex case, where algorithms necessarily suffer from the curse of
dimensionality in the worst case. For optimization, which can be seen as a
low-temperature limit of sampling, it is known that smooth functions allow
faster convergence rates. Specifically, for -times differentiable functions
in dimensions, the optimal rate for algorithms with function
evaluations is known to be , where the constant can potentially
depend on and the function to be optimized. Hence, the curse of
dimensionality can be alleviated for smooth functions at least in terms of the
convergence rate. Recently, it has been shown that similarly fast rates can
also be achieved with polynomial runtime , where the exponent
is independent of or . Hence, it is natural to ask whether similar rates
for sampling and log-partition computation are possible, and whether they can
be realized in polynomial time with an exponent independent of and . We
show that the optimal rates for sampling and log-partition computation are
sometimes equal and sometimes faster than for optimization. We then analyze
various polynomial-time sampling algorithms, including an extension of a recent
promising optimization approach, and find that they sometimes exhibit
interesting behavior but no near-optimal rates. Our results also give further
insights on the relation between sampling, log-partition, and optimization
problems.Comment: Changes in v2: Minor corrections and formatting changes. Plots can be
reproduced using the code at
https://github.com/dholzmueller/sampling_experiment
Adaptive multiscale detection of filamentary structures in a background of uniform random points
We are given a set of points that might be uniformly distributed in the
unit square . We wish to test whether the set, although mostly
consisting of uniformly scattered points, also contains a small fraction of
points sampled from some (a priori unknown) curve with -norm
bounded by . An asymptotic detection threshold exists in this problem;
for a constant , if the number of points sampled from the
curve is smaller than , reliable detection
is not possible for large . We describe a multiscale significant-runs
algorithm that can reliably detect concentration of data near a smooth curve,
without knowing the smoothness information or in advance,
provided that the number of points on the curve exceeds
. This algorithm therefore has an optimal
detection threshold, up to a factor . At the heart of our approach is
an analysis of the data by counting membership in multiscale multianisotropic
strips. The strips will have area and exhibit a variety of lengths,
orientations and anisotropies. The strips are partitioned into anisotropy
classes; each class is organized as a directed graph whose vertices all are
strips of the same anisotropy and whose edges link such strips to their ``good
continuations.'' The point-cloud data are reduced to counts that measure
membership in strips. Each anisotropy graph is reduced to a subgraph that
consist of strips with significant counts. The algorithm rejects
whenever some such subgraph contains a path that connects many consecutive
significant counts.Comment: Published at http://dx.doi.org/10.1214/009053605000000787 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …