12,850 research outputs found
Communications inspired linear discriminant analysis
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label. By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods, and comparisons are also made with a method in which Rényi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets. Copyright 2012 by the author(s)/owner(s)
Discrimination on the Grassmann Manifold: Fundamental Limits of Subspace Classifiers
We present fundamental limits on the reliable classification of linear and
affine subspaces from noisy, linear features. Drawing an analogy between
discrimination among subspaces and communication over vector wireless channels,
we propose two Shannon-inspired measures to characterize asymptotic classifier
performance. First, we define the classification capacity, which characterizes
necessary and sufficient conditions for the misclassification probability to
vanish as the signal dimension, the number of features, and the number of
subspaces to be discerned all approach infinity. Second, we define the
diversity-discrimination tradeoff which, by analogy with the
diversity-multiplexing tradeoff of fading vector channels, characterizes
relationships between the number of discernible subspaces and the
misclassification probability as the noise power approaches zero. We derive
upper and lower bounds on these measures which are tight in many regimes.
Numerical results, including a face recognition application, validate the
results in practice.Comment: 19 pages, 4 figures. Revised submission to IEEE Transactions on
Information Theor
Neural-inspired sensors enable sparse, efficient classification of spatiotemporal data
Sparse sensor placement is a central challenge in the efficient
characterization of complex systems when the cost of acquiring and processing
data is high. Leading sparse sensing methods typically exploit either spatial
or temporal correlations, but rarely both. This work introduces a new sparse
sensor optimization that is designed to leverage the rich spatiotemporal
coherence exhibited by many systems. Our approach is inspired by the remarkable
performance of flying insects, which use a few embedded strain-sensitive
neurons to achieve rapid and robust flight control despite large gust
disturbances. Specifically, we draw on nature to identify targeted
neural-inspired sensors on a flapping wing to detect body rotation. This task
is particularly challenging as the rotational twisting mode is three
orders-of-magnitude smaller than the flapping modes. We show that nonlinear
filtering in time, built to mimic strain-sensitive neurons, is essential to
detect rotation, whereas instantaneous measurements fail. Optimized sparse
sensor placement results in efficient classification with approximately ten
sensors, achieving the same accuracy and noise robustness as full measurements
consisting of hundreds of sensors. Sparse sensing with neural inspired encoding
establishes a new paradigm in hyper-efficient, embodied sensing of
spatiotemporal data and sheds light on principles of biological sensing for
agile flight control.Comment: 21 pages, 19 figure
Compressive Classification
This paper derives fundamental limits associated with compressive
classification of Gaussian mixture source models. In particular, we offer an
asymptotic characterization of the behavior of the (upper bound to the)
misclassification probability associated with the optimal Maximum-A-Posteriori
(MAP) classifier that depends on quantities that are dual to the concepts of
diversity gain and coding gain in multi-antenna communications. The diversity,
which is shown to determine the rate at which the probability of
misclassification decays in the low noise regime, is shown to depend on the
geometry of the source, the geometry of the measurement system and their
interplay. The measurement gain, which represents the counterpart of the coding
gain, is also shown to depend on geometrical quantities. It is argued that the
diversity order and the measurement gain also offer an optimization criterion
to perform dictionary learning for compressive classification applications.Comment: 5 pages, 3 figures, submitted to the 2013 IEEE International
Symposium on Information Theory (ISIT 2013
Generalizing Lieb's Concavity Theorem via Operator Interpolation
We introduce the notion of -trace and use interpolation of operators to
prove the joint concavity of the function
,
which generalizes Lieb's concavity theorem from trace to a class of homogeneous
functions . Here denotes the
elementary symmetric polynomial of the eigenvalues of . This
result gives an alternative proof for the concavity of
that was obtained and
used in a recent work to derive expectation estimates and tail bounds on
partial spectral sums of random matrices
- …