3 research outputs found
A Self-consistent-field Iteration for Orthogonal Canonical Correlation Analysis
We propose an efficient algorithm for solving orthogonal canonical
correlation analysis (OCCA) in the form of trace-fractional structure and
orthogonal linear projections. Even though orthogonality has been widely used
and proved to be a useful criterion for pattern recognition and feature
extraction, existing methods for solving OCCA problem are either numerical
unstable by relying on a deflation scheme, or less efficient by directly using
generic optimization methods. In this paper, we propose an alternating
numerical scheme whose core is the sub-maximization problem in the
trace-fractional form with an orthogonal constraint. A customized
self-consistent-field (SCF) iteration for this sub-maximization problem is
devised. It is proved that the SCF iteration is globally convergent to a KKT
point and that the alternating numerical scheme always converges. We further
formulate a new trace-fractional maximization problem for orthogonal multiset
CCA (OMCCA) and then propose an efficient algorithm with an either Jacobi-style
or Gauss-Seidel-style updating scheme based on the same SCF iteration.
Extensive experiments are conducted to evaluate the proposed algorithms against
existing methods including two real world applications: multi-label
classification and multi-view feature extraction. Experimental results show
that our methods not only perform competitively to or better than baselines but
also are more efficient
Optimal Convergence Rate of Self-Consistent Field Iteration for Solving Eigenvector-dependent Nonlinear Eigenvalue Problems
We present a comprehensive convergence analysis for Self-Consistent Field
(SCF) iteration to solve a class of nonlinear eigenvalue problems with
eigenvector-dependency (NEPv). Using a tangent-angle matrix as an intermediate
measure for approximation error, we establish new formulas for two fundamental
quantities that optimally characterize the local convergence of the plain SCF:
the local contraction factor and the local average contraction factor. In
comparison with previously established results, new convergence rate estimates
provide much sharper bounds on the convergence speed. As an application, we
extend the convergence analysis to a popular SCF variant -- the level-shifted
SCF. The effectiveness of the convergence rate estimates is demonstrated
numerically for NEPv arising from solving the Kohn-Sham equation in electronic
structure calculation and the Gross-Pitaevskii equation in the modeling of
Bose-Einstein condensation
Uncorrelated Semi-paired Subspace Learning
Multi-view datasets are increasingly collected in many real-world
applications, and we have seen better learning performance by existing
multi-view learning methods than by conventional single-view learning methods
applied to each view individually. But, most of these multi-view learning
methods are built on the assumption that at each instance no view is missing
and all data points from all views must be perfectly paired. Hence they cannot
handle unpaired data but ignore them completely from their learning process.
However, unpaired data can be more abundant in reality than paired ones and
simply ignoring all unpaired data incur tremendous waste in resources. In this
paper, we focus on learning uncorrelated features by semi-paired subspace
learning, motivated by many existing works that show great successes of
learning uncorrelated features. Specifically, we propose a generalized
uncorrelated multi-view subspace learning framework, which can naturally
integrate many proven learning criteria on the semi-paired data. To showcase
the flexibility of the framework, we instantiate five new semi-paired models
for both unsupervised and semi-supervised learning. We also design a successive
alternating approximation (SAA) method to solve the resulting optimization
problem and the method can be combined with the powerful Krylov subspace
projection technique if needed. Extensive experimental results on multi-view
feature extraction and multi-modality classification show that our proposed
models perform competitively to or better than the baselines