10 research outputs found
Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit
Subspace clustering methods based on , or nuclear norm
regularization have become very popular due to their simplicity, theoretical
guarantees and empirical success. However, the choice of the regularizer can
greatly impact both theory and practice. For instance, regularization
is guaranteed to give a subspace-preserving affinity (i.e., there are no
connections between points from different subspaces) under broad conditions
(e.g., arbitrary subspaces and corrupted data). However, it requires solving a
large scale convex optimization problem. On the other hand, and
nuclear norm regularization provide efficient closed form solutions, but
require very strong assumptions to guarantee a subspace-preserving affinity,
e.g., independent subspaces and uncorrupted data. In this paper we study a
subspace clustering method based on orthogonal matching pursuit. We show that
the method is both computationally efficient and guaranteed to give a
subspace-preserving affinity under broad conditions. Experiments on synthetic
data verify our theoretical analysis, and applications in handwritten digit and
face clustering show that our approach achieves the best trade off between
accuracy and efficiency.Comment: 13 pages, 1 figure, 2 tables. Accepted to CVPR 2016 as an oral
presentatio
Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering
State-of-the-art subspace clustering methods are based on expressing each
data point as a linear combination of other data points while regularizing the
matrix of coefficients with , or nuclear norms.
regularization is guaranteed to give a subspace-preserving affinity (i.e.,
there are no connections between points from different subspaces) under broad
theoretical conditions, but the clusters may not be connected. and
nuclear norm regularization often improve connectivity, but give a
subspace-preserving affinity only for independent subspaces. Mixed ,
and nuclear norm regularizations offer a balance between the
subspace-preserving and connectedness properties, but this comes at the cost of
increased computational complexity. This paper studies the geometry of the
elastic net regularizer (a mixture of the and norms) and uses
it to derive a provably correct and scalable active set method for finding the
optimal coefficients. Our geometric analysis also provides a theoretical
justification and a geometric interpretation for the balance between the
connectedness (due to regularization) and subspace-preserving (due to
regularization) properties for elastic net subspace clustering. Our
experiments show that the proposed active set method not only achieves
state-of-the-art clustering performance, but also efficiently handles
large-scale datasets.Comment: 15 pages, 6 figures, accepted to CVPR 2016 for oral presentatio
Improved full-waveform inversion for seismic data in the presence of noise based on the K-support norm
Full-waveform inversion (FWI) is known as a seismic data processing method
that achieves high-resolution imaging. In the inversion part of the method that
brings high resolution in finding a convergence point in the model space, a
local numerical optimization algorithm minimizes the objective function based
on the norm using the least-square form. Since the norm is sensitive to
outliers and noise, the method may often lead to inaccurate imaging results.
Thus, a new regulation form with a more practical relaxation form is proposed
to solve the overfitting drawback caused by the use of the norm,, namely the
K-support norm, which has the form of more reasonable and tighter constraints.
In contrast to the least-square method that minimizes the norm, our K-support
constraints combine the and the norms. Then, a quadratic penalty method is
adopted to linearize the non-linear problem to lighten the computational load.
This paper introduces the concept of the K-support norm and integrates this
scheme with the quadratic penalty problem to improve the convergence and
robustness against background noise. In the numerical example, two synthetic
models are tested to clarify the effectiveness of the K-support norm by
comparison to the conventional norm with noisy data set. Experimental results
indicate that the modified FWI based on the new regularization form effectively
improves inversion accuracy and stability, which significantly enhances the
lateral resolution of depth inversion even with data with a low signal-to-noise
ratio (SNR).Comment: 54 pages, 21 figure
Subspace clustering via good neighbors
Finding the informative clusters of a high-dimensional dataset is at the core of numerous applications in computer vision, where spectral based subspace clustering algorithm is arguably the most widely-studied methods due to its empirical performance and provable guarantees under various assumptions. It is well-known that sparsity and connectivity of the affinity graph play important rules for effective subspace clustering. However, it is difficult to simultaneously optimize both factors due to their conflicting nature, and most existing methods are designed to deal with only one factor. In this paper, we propose an algorithm to optimize both sparsity and connectivity by finding good neighbors which induce key connections among samples within a subspace. First, an initial coefficient matrix is generated from the input dataset. For each sample, we find its good neighbors which not only have large coefficients but are strongly connected to each other. We reassign the coefficients of good neighbors and eliminate other entries to generate a new coefficient matrix, which can be used by spectral clustering methods. Experiments on five benchmark datasets show that the proposed algorithm performs favorably against the state-of-the-art methods in terms of accuracy with a negligible increase in speed
Simultaneous subspace clustering and cluster number estimating based on triplet relationship
In this paper we propose a unified framework to discover the number of clusters and group the data points into different clusters using subspace clustering simultaneously. Real data distributed in a high-dimensional space can be disentangled into a union of low-dimensional subspaces, which can benefit various applications. To explore such intrinsic structure, stateof- the-art subspace clustering approaches often optimize a selfrepresentation problem among all samples, to construct a pairwise affinity graph for spectral clustering. However, a graph with pairwise similarities lacks robustness for segmentation, especially for samples which lie on the intersection of two subspaces. To address this problem, we design a hyper-correlation based data structure termed as the triplet relationship, which reveals high relevance and local compactness among three samples. The triplet relationship can be derived from the self-representation matrix, and be utilized to iteratively assign the data points to clusters. Based on the triplet relationship, we propose a unified optimizing scheme to automatically calculate clustering assignments. Specifically, we optimize a model selection reward and a fusion reward by simultaneously maximizing the similarity of triplets from different clusters while minimizing the correlation of triplets from same cluster. The proposed algorithm also automatically reveals the number of clusters and fuses groups to avoid over-segmentation. Extensive experimental results on both synthetic and real-world datasets validate the effectiveness and robustness of the proposed method
Robust Estimation of Motion Parameters and Scene Geometry : Minimal Solvers and Convexification of Regularisers for Low-Rank Approximation
In the dawning age of autonomous driving, accurate and robust tracking of vehicles is a quintessential part. This is inextricably linked with the problem of Simultaneous Localisation and Mapping (SLAM), in which one tries to determine the position of a vehicle relative to its surroundings without prior knowledge of them. The more you know about the object you wish to track—through sensors or mechanical construction—the more likely you are to get good positioning estimates. In the first part of this thesis, we explore new ways of improving positioning for vehicles travelling on a planar surface. This is done in several different ways: first, we generalise the work done for monocular vision to include two cameras, we propose ways of speeding up the estimation time with polynomial solvers, and we develop an auto-calibration method to cope with radially distorted images, without enforcing pre-calibration procedures.We continue to investigate the case of constrained motion—this time using auxiliary data from inertial measurement units (IMUs) to improve positioning of unmanned aerial vehicles (UAVs). The proposed methods improve the state-of-the-art for partially calibrated cases (with unknown focal length) for indoor navigation. Furthermore, we propose the first-ever real-time compatible minimal solver for simultaneous estimation of radial distortion profile, focal length, and motion parameters while utilising the IMU data.In the third and final part of this thesis, we develop a bilinear framework for low-rank regularisation, with global optimality guarantees under certain conditions. We also show equivalence between the linear and the bilinear framework, in the sense that the objectives are equal. This enables users of alternating direction method of multipliers (ADMM)—or other subgradient or splitting methods—to transition to the new framework, while being able to enjoy the benefits of second order methods. Furthermore, we propose a novel regulariser fusing two popular methods. This way we are able to combine the best of two worlds by encouraging bias reduction while enforcing low-rank solutions