372 research outputs found
Analyzing sparse dictionaries for online learning with kernels
Many signal processing and machine learning methods share essentially the
same linear-in-the-parameter model, with as many parameters as available
samples as in kernel-based machines. Sparse approximation is essential in many
disciplines, with new challenges emerging in online learning with kernels. To
this end, several sparsity measures have been proposed in the literature to
quantify sparse dictionaries and constructing relevant ones, the most prolific
ones being the distance, the approximation, the coherence and the Babel
measures. In this paper, we analyze sparse dictionaries based on these
measures. By conducting an eigenvalue analysis, we show that these sparsity
measures share many properties, including the linear independence condition and
inducing a well-posed optimization problem. Furthermore, we prove that there
exists a quasi-isometry between the parameter (i.e., dual) space and the
dictionary's induced feature space.Comment: 10 page
Fisher and Kernel Fisher Discriminant Analysis: Tutorial
This is a detailed tutorial paper which explains the Fisher discriminant
Analysis (FDA) and kernel FDA. We start with projection and reconstruction.
Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two-
and then multi-classes are explained in FDA. Then, we discuss on the rank of
the scatters and the dimensionality of the subspace. A real-life example is
also provided for interpreting FDA. Then, possible singularity of the scatter
is discussed to introduce robust FDA. PCA and FDA directions are also compared.
We also prove that FDA and linear discriminant analysis are equivalent. Fisher
forest is also introduced as an ensemble of fisher subspaces useful for
handling data with different features and dimensionality. Afterwards, kernel
FDA is explained for both one- and multi-dimensional subspaces with both two-
and multi-classes. Finally, some simulations are performed on AT&T face dataset
to illustrate FDA and compare it with PCA
Recommended from our members
Adaptive neural control of MIMO nonlinear systems with a block-triangular pure-feedback control structure
This paper presents adaptive neural tracking control for a class of uncertain multi-input-multi-output (MIMO) nonlinear systems in block-triangular form. All subsystems within these MIMO nonlinear systems are of completely nonaffine purefeedback form and allowed to have different orders. To deal with the nonaffine appearance of the control variables, the mean value theorem (MVT) is employed to transform the systems into a block-triangular strict-feedback form with control coefficients being couplings among various inputs and outputs. A systematic procedure is proposed for the design of a new singularityfree adaptive neural tracking control strategy. Such a design procedure can remove the couplings among subsystems and hence avoids the possible circular control construction problem. As a consequence, all the signals in the closed-loop system are guaranteed to be semiglobally uniformly ultimately bounded (SGUUB). Moreover, the outputs of the systems are ensured to converge to a small neighborhood of the desired trajectories. Simulation studies verify the theoretical findings revealed in this work
- …