OPML: A One-Pass Closed-Form Solution for Online Metric Learning

To achieve a low computational cost when performing online metric learning for large-scale data, we present a one-pass closed-form solution namely OPML in this paper. Typically, the proposed OPML first adopts a one-pass triplet construction strategy, which aims to use only a very small number of triplets to approximate the representation ability of whole original triplets obtained by batch-manner methods. Then, OPML employs a closed-form solution to update the metric for new coming samples, which leads to a low space (i.e., $O(d)$) and time (i.e., $O(d^2)$) complexity, where $d$ is the feature dimensionality. In addition, an extension of OPML (namely COPML) is further proposed to enhance the robustness when in real case the first several samples come from the same class (i.e., cold start problem). In the experiments, we have systematically evaluated our methods (OPML and COPML) on three typical tasks, including UCI data classification, face verification, and abnormal event detection in videos, which aims to fully evaluate the proposed methods on different sample number, different feature dimensionalities and different feature extraction ways (i.e., hand-crafted and deeply-learned). The results show that OPML and COPML can obtain the promising performance with a very low computational cost. Also, the effectiveness of COPML under the cold start setting is experimentally verified.Comment: 12 page

Rigorous a-posteriori analysis using numerical eigenvalue bounds in a surface growth model

In order to prove numerically the global existence and uniqueness of smooth solutions of a fourth order, nonlinear PDE, we derive rigorous a-posteriori upper bounds on the supremum of the numerical range of the linearized operator. These bounds also have to be easily computable in order to be applicable to our rigorous a-posteriori methods, as we use them in each time-step of the numerical discretization. The final goal is to establish global bounds on smooth local solutions, which then establish global uniqueness.Comment: 19 pages, 9 figure

Robust Low-Rank Subspace Segmentation with Semidefinite Guarantees

Recently there is a line of research work proposing to employ Spectral Clustering (SC) to segment (group){Throughout the paper, we use segmentation, clustering, and grouping, and their verb forms, interchangeably.} high-dimensional structural data such as those (approximately) lying on subspaces {We follow {liu2010robust} and use the term "subspace" to denote both linear subspaces and affine subspaces. There is a trivial conversion between linear subspaces and affine subspaces as mentioned therein.} or low-dimensional manifolds. By learning the affinity matrix in the form of sparse reconstruction, techniques proposed in this vein often considerably boost the performance in subspace settings where traditional SC can fail. Despite the success, there are fundamental problems that have been left unsolved: the spectrum property of the learned affinity matrix cannot be gauged in advance, and there is often one ugly symmetrization step that post-processes the affinity for SC input. Hence we advocate to enforce the symmetric positive semidefinite constraint explicitly during learning (Low-Rank Representation with Positive SemiDefinite constraint, or LRR-PSD), and show that factually it can be solved in an exquisite scheme efficiently instead of general-purpose SDP solvers that usually scale up poorly. We provide rigorous mathematical derivations to show that, in its canonical form, LRR-PSD is equivalent to the recently proposed Low-Rank Representation (LRR) scheme {liu2010robust}, and hence offer theoretic and practical insights to both LRR-PSD and LRR, inviting future research. As per the computational cost, our proposal is at most comparable to that of LRR, if not less. We validate our theoretic analysis and optimization scheme by experiments on both synthetic and real data sets.Comment: 10 pages, 4 figures. Accepted by ICDM Workshop on Optimization Based Methods for Emerging Data Mining Problems (OEDM), 2010. Main proof simplified and typos corrected. Experimental data slightly adde

Network Density of States

Spectral analysis connects graph structure to the eigenvalues and eigenvectors of associated matrices. Much of spectral graph theory descends directly from spectral geometry, the study of differentiable manifolds through the spectra of associated differential operators. But the translation from spectral geometry to spectral graph theory has largely focused on results involving only a few extreme eigenvalues and their associated eigenvalues. Unlike in geometry, the study of graphs through the overall distribution of eigenvalues - the spectral density - is largely limited to simple random graph models. The interior of the spectrum of real-world graphs remains largely unexplored, difficult to compute and to interpret. In this paper, we delve into the heart of spectral densities of real-world graphs. We borrow tools developed in condensed matter physics, and add novel adaptations to handle the spectral signatures of common graph motifs. The resulting methods are highly efficient, as we illustrate by computing spectral densities for graphs with over a billion edges on a single compute node. Beyond providing visually compelling fingerprints of graphs, we show how the estimation of spectral densities facilitates the computation of many common centrality measures, and use spectral densities to estimate meaningful information about graph structure that cannot be inferred from the extremal eigenpairs alone.Comment: 10 pages, 7 figure

A stability criterion for high-frequency oscillations

We show that a simple Levi compatibility condition determines stability of WKB solutions to semilinear hyperbolic initial-value problems issued from highly-oscillating initial data with large amplitudes. The compatibility condition involves the hyperbolic operator, the fundamental phase associated with the initial oscillation, and the semilinear source term; it states roughly that hyperbolicity is preserved around resonances. If the compatibility condition is satisfied, the solutions are defined over time intervals independent of the wavelength, and the associated WKB solutions are stable under a large class of initial perturbations. If the compatibility condition is not satisfied, resonances are exponentially amplified, and arbitrarily small initial perturbations can destabilize the WKB solutions in small time. The amplification mechanism is based on the observation that in frequency space, resonances correspond to points of weak hyperbolicity. At such points, the behavior of the system depends on the lower order terms through the compatibility condition. The analysis relies, in the unstable case, on a short-time Duhamel representation formula for solutions of zeroth-order pseudo-differential equations. Our examples include coupled Klein-Gordon systems, and systems describing Raman and Brillouin instabilities.Comment: Final version, to appear in M\'em. Soc. Math. F

An inverse Sturm-Liouville problem with a fractional derivative

In this paper, we numerically investigate an inverse problem of recovering the potential term in a fractional Sturm-Liouville problem from one spectrum. The qualitative behaviors of the eigenvalues and eigenfunctions are discussed, and numerical reconstructions of the potential with a Newton method from finite spectral data are presented. Surprisingly, it allows very satisfactory reconstructions for both smooth and discontinuous potentials, provided that the order $\alpha\in(1,2)$ of fractional derivative is sufficiently away from 2.Comment: 16 pages, 6 figures, accepted for publication in Journal of Computational Physic