31,921 research outputs found
Subspace Learning with Partial Information
The goal of subspace learning is to find a -dimensional subspace of
, such that the expected squared distance between instance
vectors and the subspace is as small as possible. In this paper we study
subspace learning in a partial information setting, in which the learner can
only observe attributes from each instance vector. We propose several
efficient algorithms for this task, and analyze their sample complexit
Constrained Sparse Subspace Clustering with Side-Information
Subspace clustering refers to the problem of segmenting high dimensional data
drawn from a union of subspaces into the respective subspaces. In some
applications, partial side-information to indicate "must-link" or "cannot-link"
in clustering is available. This leads to the task of subspace clustering with
side-information. However, in prior work the supervision value of the
side-information for subspace clustering has not been fully exploited. To this
end, in this paper, we present an enhanced approach for constrained subspace
clustering with side-information, termed Constrained Sparse Subspace Clustering
plus (CSSC+), in which the side-information is used not only in the stage of
learning an affinity matrix but also in the stage of spectral clustering.
Moreover, we propose to estimate clustering accuracy based on the partial
side-information and theoretically justify the connection to the ground-truth
clustering accuracy in terms of the Rand index. We conduct experiments on three
cancer gene expression datasets to validate the effectiveness of our proposals.Comment: 8 pages, 2 figures, and 3 tables. This work has been accepted by ICPR
2018 as oral presentatio
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm
This work presents a new algorithm for empirical risk minimization. The
algorithm bridges the gap between first- and second-order methods by computing
a search direction that uses a second-order-type update in one subspace,
coupled with a scaled steepest descent step in the orthogonal complement. To
this end, partial curvature information is incorporated to help with
ill-conditioning, while simultaneously allowing the algorithm to scale to the
large problem dimensions often encountered in machine learning applications.
Theoretical results are presented to confirm that the algorithm converges to a
stationary point in both the strongly convex and nonconvex cases. A stochastic
variant of the algorithm is also presented, along with corresponding
theoretical guarantees. Numerical results confirm the strengths of the new
approach on standard machine learning problems.Comment: 38 pages, 74 figure
Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval
Sketch-based image retrieval (SBIR) is challenging due to the inherent
domain-gap between sketch and photo. Compared with pixel-perfect depictions of
photos, sketches are iconic renderings of the real world with highly abstract.
Therefore, matching sketch and photo directly using low-level visual clues are
unsufficient, since a common low-level subspace that traverses semantically
across the two modalities is non-trivial to establish. Most existing SBIR
studies do not directly tackle this cross-modal problem. This naturally
motivates us to explore the effectiveness of cross-modal retrieval methods in
SBIR, which have been applied in the image-text matching successfully. In this
paper, we introduce and compare a series of state-of-the-art cross-modal
subspace learning methods and benchmark them on two recently released
fine-grained SBIR datasets. Through thorough examination of the experimental
results, we have demonstrated that the subspace learning can effectively model
the sketch-photo domain-gap. In addition we draw a few key insights to drive
future research.Comment: Accepted by Neurocomputin
Partial Sum Minimization of Singular Values Representation on Grassmann Manifolds
As a significant subspace clustering method, low rank representation (LRR)
has attracted great attention in recent years. To further improve the
performance of LRR and extend its applications, there are several issues to be
resolved. The nuclear norm in LRR does not sufficiently use the prior knowledge
of the rank which is known in many practical problems. The LRR is designed for
vectorial data from linear spaces, thus not suitable for high dimensional data
with intrinsic non-linear manifold structure. This paper proposes an extended
LRR model for manifold-valued Grassmann data which incorporates prior knowledge
by minimizing partial sum of singular values instead of the nuclear norm,
namely Partial Sum minimization of Singular Values Representation (GPSSVR). The
new model not only enforces the global structure of data in low rank, but also
retains important information by minimizing only smaller singular values. To
further maintain the local structures among Grassmann points, we also integrate
the Laplacian penalty with GPSSVR. An effective algorithm is proposed to solve
the optimization problem based on the GPSSVR model. The proposed model and
algorithms are assessed on some widely used human action video datasets and a
real scenery dataset. The experimental results show that the proposed methods
obviously outperform other state-of-the-art methods.Comment: Submitting to ACM Transactions on Knowledge Discovery from Data with
minor revisio
A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture
Data sparsity is a common issue to train machine learning tools such as
neural networks for engineering and scientific applications, where experiments
and simulations are expensive. Recently physics-constrained neural networks
(PCNNs) were developed to reduce the required amount of training data. However,
the weights of different losses from data and physical constraints are adjusted
empirically in PCNNs. In this paper, a new physics-constrained neural network
with the minimax architecture (PCNN-MM) is proposed so that the weights of
different losses can be adjusted systematically. The training of the PCNN-MM is
searching the high-order saddle points of the objective function. A novel
saddle point search algorithm called Dual-Dimer method is developed. It is
demonstrated that the Dual-Dimer method is computationally more efficient than
the gradient descent ascent method for nonconvex-nonconcave functions and
provides additional eigenvalue information to verify search results. A heat
transfer example also shows that the convergence of PCNN-MMs is faster than
that of traditional PCNNs.Comment: 34 pages, 5 figures, accepted by neural network
Online Supervised Subspace Tracking
We present a framework for supervised subspace tracking, when there are two
time series and , one being the high-dimensional predictors and the
other being the response variables and the subspace tracking needs to take into
consideration of both sequences. It extends the classic online subspace
tracking work which can be viewed as tracking of only. Our online
sufficient dimensionality reduction (OSDR) is a meta-algorithm that can be
applied to various cases including linear regression, logistic regression,
multiple linear regression, multinomial logistic regression, support vector
machine, the random dot product model and the multi-scale union-of-subspace
model. OSDR reduces data-dimensionality on-the-fly with low-computational
complexity and it can also handle missing data and dynamic data. OSDR uses an
alternating minimization scheme and updates the subspace via gradient descent
on the Grassmannian manifold. The subspace update can be performed efficiently
utilizing the fact that the Grassmannian gradient with respect to the subspace
in many settings is rank-one (or low-rank in certain cases). The optimization
problem for OSDR is non-convex and hard to analyze in general; we provide
convergence analysis of OSDR in a simple linear regression setting. The good
performance of OSDR compared with the conventional unsupervised subspace
tracking are demonstrated via numerical examples on simulated and real data.Comment: Submitted for journal publicatio
Extracting low-dimensional dynamics from multiple large-scale neural population recordings by learning to predict correlations
A powerful approach for understanding neural population dynamics is to
extract low-dimensional trajectories from population recordings using
dimensionality reduction methods. Current approaches for dimensionality
reduction on neural data are limited to single population recordings, and can
not identify dynamics embedded across multiple measurements. We propose an
approach for extracting low-dimensional dynamics from multiple, sequential
recordings. Our algorithm scales to data comprising millions of observed
dimensions, making it possible to access dynamics distributed across large
populations or multiple brain areas. Building on subspace-identification
approaches for dynamical systems, we perform parameter estimation by minimizing
a moment-matching objective using a scalable stochastic gradient descent
algorithm: The model is optimized to predict temporal covariations across
neurons and across time. We show how this approach naturally handles missing
data and multiple partial recordings, and can identify dynamics and predict
correlations even in the presence of severe subsampling and small overlap
between recordings. We demonstrate the effectiveness of the approach both on
simulated data and a whole-brain larval zebrafish imaging dataset
Visual Tracking via Shallow and Deep Collaborative Model
In this paper, we propose a robust tracking method based on the collaboration
of a generative model and a discriminative classifier, where features are
learned by shallow and deep architectures, respectively. For the generative
model, we introduce a block-based incremental learning scheme, in which a local
binary mask is constructed to deal with occlusion. The similarity degrees
between the local patches and their corresponding subspace are integrated to
formulate a more accurate global appearance model. In the discriminative model,
we exploit the advances of deep learning architectures to learn generic
features which are robust to both background clutters and foreground appearance
variations. To this end, we first construct a discriminative training set from
auxiliary video sequences. A deep classification neural network is then trained
offline on this training set. Through online fine-tuning, both the hierarchical
feature extractor and the classifier can be adapted to the appearance change of
the target for effective online tracking. The collaboration of these two models
achieves a good balance in handling occlusion and target appearance change,
which are two contradictory challenging factors in visual tracking. Both
quantitative and qualitative evaluations against several state-of-the-art
algorithms on challenging image sequences demonstrate the accuracy and the
robustness of the proposed tracker.Comment: Undergraduate Thesis, appearing in Pattern Recognitio
Sampling and multilevel coarsening algorithms for fast matrix approximations
This paper addresses matrix approximation problems for matrices that are
large, sparse and/or that are representations of large graphs. To tackle these
problems, we consider algorithms that are based primarily on coarsening
techniques, possibly combined with random sampling. A multilevel coarsening
technique is proposed which utilizes a hypergraph associated with the data
matrix and a graph coarsening strategy based on column matching. Theoretical
results are established that characterize the quality of the dimension
reduction achieved by a coarsening step, when a proper column matching strategy
is employed. We consider a number of standard applications of this technique as
well as a few new ones. Among the standard applications we first consider the
problem of computing the partial SVD for which a combination of sampling and
coarsening yields significantly improved SVD results relative to sampling
alone. We also consider the Column subset selection problem, a popular low rank
approximation method used in data related applications, and show how multilevel
coarsening can be adapted for this problem. Similarly, we consider the problem
of graph sparsification and show how coarsening techniques can be employed to
solve it. Numerical experiments illustrate the performances of the methods in
various applications
- …