500,594 research outputs found
Sparse Inverse Covariance Estimation for Chordal Structures
In this paper, we consider the Graphical Lasso (GL), a popular optimization
problem for learning the sparse representations of high-dimensional datasets,
which is well-known to be computationally expensive for large-scale problems.
Recently, we have shown that the sparsity pattern of the optimal solution of GL
is equivalent to the one obtained from simply thresholding the sample
covariance matrix, for sparse graphs under different conditions. We have also
derived a closed-form solution that is optimal when the thresholded sample
covariance matrix has an acyclic structure. As a major generalization of the
previous result, in this paper we derive a closed-form solution for the GL for
graphs with chordal structures. We show that the GL and thresholding
equivalence conditions can significantly be simplified and are expected to hold
for high-dimensional problems if the thresholded sample covariance matrix has a
chordal structure. We then show that the GL and thresholding equivalence is
enough to reduce the GL to a maximum determinant matrix completion problem and
drive a recursive closed-form solution for the GL when the thresholded sample
covariance matrix has a chordal structure. For large-scale problems with up to
450 million variables, the proposed method can solve the GL problem in less
than 2 minutes, while the state-of-the-art methods converge in more than 2
hours
Non-sparse Linear Representations for Visual Tracking with Online Reservoir Metric Learning
Most sparse linear representation-based trackers need to solve a
computationally expensive L1-regularized optimization problem. To address this
problem, we propose a visual tracker based on non-sparse linear
representations, which admit an efficient closed-form solution without
sacrificing accuracy. Moreover, in order to capture the correlation information
between different feature dimensions, we learn a Mahalanobis distance metric in
an online fashion and incorporate the learned metric into the optimization
problem for obtaining the linear representation. We show that online metric
learning using proximity comparison significantly improves the robustness of
the tracking, especially on those sequences exhibiting drastic appearance
changes. Furthermore, in order to prevent the unbounded growth in the number of
training samples for the metric learning, we design a time-weighted reservoir
sampling method to maintain and update limited-sized foreground and background
sample buffers for balancing sample diversity and adaptability. Experimental
results on challenging videos demonstrate the effectiveness and robustness of
the proposed tracker.Comment: Appearing in IEEE Conf. Computer Vision and Pattern Recognition, 201
Multiplicative Updates for Nonnegative Quadratic Programming
Many problems in neural computation and statistical learning involve optimizations with nonnegativity constraints. In this article, we study convex problems in quadratic programming where the optimization is confined to an axis-aligned region in the nonnegative orthant. For these problems, we derive multiplicative updates that improve the value of the objective function at each iteration and converge monotonically to the global minimum. The updates have a simple closed form and do not involve any heuristics or free parameters that must be tuned to ensure convergence. Despite their simplicity, they differ strikingly in form from other multiplicative updates used in machine learning.We provide complete proofs of convergence for these updates and describe their application to problems in signal processing and pattern recognition
Graph learning under spectral sparsity constraints
Graph inference plays an essential role in machine learning, pattern
recognition, and classification. Signal processing based approaches in
literature generally assume some variational property of the observed data on
the graph. We make a case for inferring graphs on which the observed data has
high variation. We propose a signal processing based inference model that
allows for wideband frequency variation in the data and propose an algorithm
for graph inference. The proposed inference algorithm consists of two steps: 1)
learning orthogonal eigenvectors of a graph from the data; 2) recovering the
adjacency matrix of the graph topology from the given graph eigenvectors. The
first step is solved by an iterative algorithm with a closed-form solution. In
the second step, the adjacency matrix is inferred from the eigenvectors by
solving a convex optimization problem. Numerical results on synthetic data show
the proposed inference algorithm can effectively capture the meaningful graph
topology from observed data under the wideband assumption
Synaptic state matching: a dynamical architecture for predictive internal representation and feature perception
Here we consider the possibility that a fundamental function of sensory cortex is the generation of an internal simulation of sensory environment in real-time. A logical elaboration of this idea leads to a dynamical neural architecture that oscillates between two fundamental network states, one driven by external input, and the other by recurrent synaptic drive in the absence of sensory input. Synaptic strength is modified by a proposed synaptic state matching (SSM) process that ensures equivalence of spike statistics between the two network states. Remarkably, SSM, operating locally at individual synapses, generates accurate and stable network-level predictive internal representations, enabling pattern completion and unsupervised feature detection from noisy sensory input. SSM is a biologically plausible substrate for learning and memory because it brings together sequence learning, feature detection, synaptic homeostasis, and network oscillations under a single parsimonious computational framework. Beyond its utility as a potential model of cortical computation, artificial networks based on this principle have remarkable capacity for internalizing dynamical systems, making them useful in a variety of application domains including time-series prediction and machine intelligence
- …