5,787 research outputs found
Many Sparse Cuts via Higher Eigenvalues
Cheeger's fundamental inequality states that any edge-weighted graph has a
vertex subset such that its expansion (a.k.a. conductance) is bounded as
follows: \phi(S) \defeq \frac{w(S,\bar{S})}{\min \set{w(S), w(\bar{S})}}
\leq 2\sqrt{\lambda_2} where is the total edge weight of a subset or a
cut and is the second smallest eigenvalue of the normalized
Laplacian of the graph. Here we prove the following natural generalization: for
any integer , there exist disjoint subsets ,
such that where
is the smallest eigenvalue of the normalized Laplacian and
are suitable absolute constants. Our proof is via a polynomial-time
algorithm to find such subsets, consisting of a spectral projection and a
randomized rounding. As a consequence, we get the same upper bound for the
small set expansion problem, namely for any , there is a subset whose
weight is at most a \bigO(1/k) fraction of the total weight and . Both results are the best possible up to constant
factors.
The underlying algorithmic problem, namely finding subsets such that the
maximum expansion is minimized, besides extending sparse cuts to more than one
subset, appears to be a natural clustering problem in its own right
Improved Cheeger's Inequality: Analysis of Spectral Partitioning Algorithms through Higher Order Spectral Gap
Let \phi(G) be the minimum conductance of an undirected graph G, and let
0=\lambda_1 <= \lambda_2 <=... <= \lambda_n <= 2 be the eigenvalues of the
normalized Laplacian matrix of G. We prove that for any graph G and any k >= 2,
\phi(G) = O(k) \lambda_2 / \sqrt{\lambda_k}, and this performance guarantee
is achieved by the spectral partitioning algorithm. This improves Cheeger's
inequality, and the bound is optimal up to a constant factor for any k. Our
result shows that the spectral partitioning algorithm is a constant factor
approximation algorithm for finding a sparse cut if \lambda_k$ is a constant
for some constant k. This provides some theoretical justification to its
empirical performance in image segmentation and clustering problems. We extend
the analysis to other graph partitioning problems, including multi-way
partition, balanced separator, and maximum cut
Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications
In computer vision, many problems such as image segmentation, pixel
labelling, and scene parsing can be formulated as binary quadratic programs
(BQPs). For submodular problems, cuts based methods can be employed to
efficiently solve large-scale problems. However, general nonsubmodular problems
are significantly more challenging to solve. Finding a solution when the
problem is of large size to be of practical interest, however, typically
requires relaxation. Two standard relaxation methods are widely used for
solving general BQPs--spectral methods and semidefinite programming (SDP), each
with their own advantages and disadvantages. Spectral relaxation is simple and
easy to implement, but its bound is loose. Semidefinite relaxation has a
tighter bound, but its computational complexity is high, especially for large
scale problems. In this work, we present a new SDP formulation for BQPs, with
two desirable properties. First, it has a similar relaxation bound to
conventional SDP formulations. Second, compared with conventional SDP methods,
the new SDP formulation leads to a significantly more efficient and scalable
dual optimization approach, which has the same degree of complexity as spectral
methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton
methods, for the dual problem. Both of them are significantly more efficiently
than standard interior-point methods. In practice, the smoothing Newton solver
is faster than the quasi-Newton solver for dense or medium-sized problems,
while the quasi-Newton solver is preferable for large sparse/structured
problems. Our experiments on a few computer vision applications including
clustering, image segmentation, co-segmentation and registration show the
potential of our SDP formulation for solving large-scale BQPs.Comment: Fixed some typos. 18 pages. Accepted to IEEE Transactions on Pattern
Analysis and Machine Intelligenc
Graph Spectral Image Processing
Recent advent of graph signal processing (GSP) has spurred intensive studies
of signals that live naturally on irregular data kernels described by graphs
(e.g., social networks, wireless sensor networks). Though a digital image
contains pixels that reside on a regularly sampled 2D grid, if one can design
an appropriate underlying graph connecting pixels with weights that reflect the
image structure, then one can interpret the image (or image patch) as a signal
on a graph, and apply GSP tools for processing and analysis of the signal in
graph spectral domain. In this article, we overview recent graph spectral
techniques in GSP specifically for image / video processing. The topics covered
include image compression, image restoration, image filtering and image
segmentation
Robust Low-Rank Subspace Segmentation with Semidefinite Guarantees
Recently there is a line of research work proposing to employ Spectral
Clustering (SC) to segment (group){Throughout the paper, we use segmentation,
clustering, and grouping, and their verb forms, interchangeably.}
high-dimensional structural data such as those (approximately) lying on
subspaces {We follow {liu2010robust} and use the term "subspace" to denote both
linear subspaces and affine subspaces. There is a trivial conversion between
linear subspaces and affine subspaces as mentioned therein.} or low-dimensional
manifolds. By learning the affinity matrix in the form of sparse
reconstruction, techniques proposed in this vein often considerably boost the
performance in subspace settings where traditional SC can fail. Despite the
success, there are fundamental problems that have been left unsolved: the
spectrum property of the learned affinity matrix cannot be gauged in advance,
and there is often one ugly symmetrization step that post-processes the
affinity for SC input. Hence we advocate to enforce the symmetric positive
semidefinite constraint explicitly during learning (Low-Rank Representation
with Positive SemiDefinite constraint, or LRR-PSD), and show that factually it
can be solved in an exquisite scheme efficiently instead of general-purpose SDP
solvers that usually scale up poorly. We provide rigorous mathematical
derivations to show that, in its canonical form, LRR-PSD is equivalent to the
recently proposed Low-Rank Representation (LRR) scheme {liu2010robust}, and
hence offer theoretic and practical insights to both LRR-PSD and LRR, inviting
future research. As per the computational cost, our proposal is at most
comparable to that of LRR, if not less. We validate our theoretic analysis and
optimization scheme by experiments on both synthetic and real data sets.Comment: 10 pages, 4 figures. Accepted by ICDM Workshop on Optimization Based
Methods for Emerging Data Mining Problems (OEDM), 2010. Main proof simplified
and typos corrected. Experimental data slightly adde
- …