732 research outputs found
A Robust Zero-point Attraction LMS Algorithm on Near Sparse System Identification
The newly proposed norm constraint zero-point attraction Least Mean
Square algorithm (ZA-LMS) demonstrates excellent performance on exact sparse
system identification. However, ZA-LMS has less advantage against standard LMS
when the system is near sparse. Thus, in this paper, firstly the near sparse
system modeling by Generalized Gaussian Distribution is recommended, where the
sparsity is defined accordingly. Secondly, two modifications to the ZA-LMS
algorithm have been made. The norm penalty is replaced by a partial
norm in the cost function, enhancing robustness without increasing the
computational complexity. Moreover, the zero-point attraction item is weighted
by the magnitude of estimation error which adjusts the zero-point attraction
force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS
(DWZA-LMS) algorithm is further proposed, which shows better performance on
near sparse system identification. In addition, the mean square performance of
DWZA-LMS algorithm is analyzed. Finally, computer simulations demonstrate the
effectiveness of the proposed algorithm and verify the result of theoretical
analysis.Comment: 20 pages, 11 figure
A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation
Stochastic approximation techniques play an important role in solving many
problems encountered in machine learning or adaptive signal processing. In
these contexts, the statistics of the data are often unknown a priori or their
direct computation is too intensive, and they have thus to be estimated online
from the observed signals. For batch optimization of an objective function
being the sum of a data fidelity term and a penalization (e.g. a sparsity
promoting function), Majorize-Minimize (MM) methods have recently attracted
much interest since they are fast, highly flexible, and effective in ensuring
convergence. The goal of this paper is to show how these methods can be
successfully extended to the case when the data fidelity term corresponds to a
least squares criterion and the cost function is replaced by a sequence of
stochastic approximations of it. In this context, we propose an online version
of an MM subspace algorithm and we study its convergence by using suitable
probabilistic tools. Simulation results illustrate the good practical
performance of the proposed algorithm associated with a memory gradient
subspace, when applied to both non-adaptive and adaptive filter identification
problems
Support Recovery of Sparse Signals
We consider the problem of exact support recovery of sparse signals via noisy
measurements. The main focus is the sufficient and necessary conditions on the
number of measurements for support recovery to be reliable. By drawing an
analogy between the problem of support recovery and the problem of channel
coding over the Gaussian multiple access channel, and exploiting mathematical
tools developed for the latter problem, we obtain an information theoretic
framework for analyzing the performance limits of support recovery. Sharp
sufficient and necessary conditions on the number of measurements in terms of
the signal sparsity level and the measurement noise level are derived.
Specifically, when the number of nonzero entries is held fixed, the exact
asymptotics on the number of measurements for support recovery is developed.
When the number of nonzero entries increases in certain manners, we obtain
sufficient conditions tighter than existing results. In addition, we show that
the proposed methodology can deal with a variety of models of sparse signal
recovery, hence demonstrating its potential as an effective analytical tool.Comment: 33 page
Towards an Information Theoretic Framework for Evolutionary Learning
The vital essence of evolutionary learning consists of information flows between the environment and the entities differentially surviving and reproducing therein. Gain or loss of information in individuals and populations due to evolutionary steps should be considered in evolutionary algorithm theory and practice. Information theory has rarely been applied to evolutionary computation - a lacuna that this dissertation addresses, with an emphasis on objectively and explicitly evaluating the ensemble models implicit in evolutionary learning. Information theoretic functionals can provide objective, justifiable, general, computable, commensurate measures of fitness and diversity.
We identify information transmission channels implicit in evolutionary learning. We define information distance metrics and indices for ensembles. We extend Price\u27s Theorem to non-random mating, give it an effective fitness interpretation and decompose it to show the key factors influencing heritability and evolvability. We argue that heritability and evolvability of our information theoretic indicators are high. We illustrate use of our indices for reproductive and survival selection. We develop algorithms to estimate information theoretic quantities on mixed continuous and discrete data via the empirical copula and information dimension. We extend statistical resampling. We present experimental and real world application results: chaotic time series prediction; parity; complex continuous functions; industrial process control; and small sample social science data. We formalize conjectures regarding evolutionary learning and information geometry
Robust Sensor Fusion for Indoor Wireless Localization
Location knowledge in indoor environment using Indoor Positioning Systems
(IPS) has become very useful and popular in recent years. Indoor wireless
localization suffers from severe multi-path fading and non-line-of-sight
conditions. This paper presents a novel indoor localization framework based on
sensor fusion of Zigbee Wireless Sensor Networks (WSN) using Received Signal
Strength (RSS). The unknown position is equipped with two or more mobile nodes.
The range between two mobile nodes is fixed as priori. The attitude (roll,
pitch, and yaw) of the mobile node are measured by inertial sensors (ISs). Then
the angle and the range between any two nodes can be obtained, and thus the
path between the two nodes can be modeled as a curve. Through an efficient
cooperation between two or more mobile nodes, this framework effectively
exploits the RSS techniques. This constraint help improve the positioning
accuracy. Theoretical analysis on localization distortion and Monte Carlo
simulations shows that the proposed cooperative strategy of multiple nodes with
extended Kalman filter (EKF) achieves significantly higher positioning accuracy
than the existing systems, especially in heavily obstructed scenarios
Generalized correntropy induced metric based total least squares for sparse system identification
The total least squares (TLS) method has been successfully applied to system identification in the errors-in-variables (EIV) model, which can efficiently describe systems where input–output pairs are contaminated by noise. In this paper, we propose a new gradient-descent TLS filtering algorithm based on the generalized correntropy induced metric (GCIM), called as GCIM-TLS, for sparse system identification. By introducing GCIM as a penalty term to the TLS problem, we can achieve improved accuracy of sparse system identification. We also characterize the convergence behaviour analytically for GCIM-TLS. To reduce computational complexity, we use the first-order Taylor series expansion and further derive a simplified version of GCIM-TLS. Simulation results verify the effectiveness of our proposed algorithms in sparse system identification
A survey of cost-sensitive decision tree induction algorithms
The past decade has seen a significant interest on the problem of inducing decision trees that take account of costs of misclassification and costs of acquiring the features used for decision making. This survey identifies over 50 algorithms including approaches that are direct adaptations of accuracy based methods, use genetic algorithms, use anytime methods and utilize boosting and bagging. The survey brings together these different studies and novel approaches to cost-sensitive decision tree learning, provides a useful taxonomy, a historical timeline of how the field has developed and should provide a useful reference point for future research in this field
- …