1,042,876 research outputs found
Dimension Reduction by Mutual Information Discriminant Analysis
In the past few decades, researchers have proposed many discriminant analysis
(DA) algorithms for the study of high-dimensional data in a variety of
problems. Most DA algorithms for feature extraction are based on
transformations that simultaneously maximize the between-class scatter and
minimize the withinclass scatter matrices. This paper presents a novel DA
algorithm for feature extraction using mutual information (MI). However, it is
not always easy to obtain an accurate estimation for high-dimensional MI. In
this paper, we propose an efficient method for feature extraction that is based
on one-dimensional MI estimations. We will refer to this algorithm as mutual
information discriminant analysis (MIDA). The performance of this proposed
method was evaluated using UCI databases. The results indicate that MIDA
provides robust performance over different data sets with different
characteristics and that MIDA always performs better than, or at least
comparable to, the best performing algorithms.Comment: 13pages, 3 tables, International Journal of Artificial Intelligence &
Application
Least Dependent Component Analysis Based on Mutual Information
We propose to use precise estimators of mutual information (MI) to find least
dependent components in a linearly mixed signal. On the one hand this seems to
lead to better blind source separation than with any other presently available
algorithm. On the other hand it has the advantage, compared to other
implementations of `independent' component analysis (ICA) some of which are
based on crude approximations for MI, that the numerical values of the MI can
be used for:
(i) estimating residual dependencies between the output components;
(ii) estimating the reliability of the output, by comparing the pairwise MIs
with those of re-mixed components;
(iii) clustering the output according to the residual interdependencies.
For the MI estimator we use a recently proposed k-nearest neighbor based
algorithm. For time sequences we combine this with delay embedding, in order to
take into account non-trivial time correlations. After several tests with
artificial data, we apply the resulting MILCA (Mutual Information based Least
dependent Component Analysis) algorithm to a real-world dataset, the ECG of a
pregnant woman.
The software implementation of the MILCA algorithm is freely available at
http://www.fz-juelich.de/nic/cs/softwareComment: 18 pages, 20 figures, Phys. Rev. E (in press
BMICA-independent component analysis based on B-spline mutual information estimator
The information theoretic concept of mutual information provides a general framework to evaluate dependencies between variables. Its estimation however using B-Spline has not been used before in creating an approach for Independent Component Analysis. In this paper we present a B-Spline estimator for mutual information to find the independent components in mixed signals. Tested using electroencephalography (EEG) signals the resulting BMICA (B-Spline Mutual Information Independent Component Analysis)
exhibits better performance than the standard Independent Component Analysis algorithms of FastICA, JADE, SOBI and EFICA in similar simulations. BMICA was found to be also more reliable than the 'renown' FastICA
Mutual information between geomagnetic indices and the solar wind as seen by WIND : implications for propagation time estimates
The determination of delay times of solar wind conditions at the sunward libration point to effects on Earth is investigated using mutual information. This measures the amount of information shared between two timeseries. We consider the mutual information content of solar wind observations, from WIND, and the geomagnetic indices. The success of five commonly used schemes for estimating interplanetary propagation times is examined. Propagation assuming a fixed plane normal at 45 degrees to the GSE x-axis (i.e. the Parker Spiral estimate) is found to give optimal mutual information. The mutual information depends on the point in space chosen as the target for the propagation estimate, and we find that it is maximized by choosing a point in the nightside rather than dayside magnetosphere. In addition, we employ recurrence plot analysis to visualize contributions to the mutual information, this suggests that it appears on timescales of hours rather than minutes
Bit-Interleaved Coded Modulation Revisited: A Mismatched Decoding Perspective
We revisit the information-theoretic analysis of bit-interleaved coded
modulation (BICM) by modeling the BICM decoder as a mismatched decoder. The
mismatched decoding model is well-defined for finite, yet arbitrary, block
lengths, and naturally captures the channel memory among the bits belonging to
the same symbol. We give two independent proofs of the achievability of the
BICM capacity calculated by Caire et al. where BICM was modeled as a set of
independent parallel binary-input channels whose output is the bitwise
log-likelihood ratio. Our first achievability proof uses typical sequences, and
shows that due to the random coding construction, the interleaver is not
required. The second proof is based on the random coding error exponents with
mismatched decoding, where the largest achievable rate is the generalized
mutual information. We show that the generalized mutual information of the
mismatched decoder coincides with the infinite-interleaver BICM capacity. We
also show that the error exponent -and hence the cutoff rate- of the BICM
mismatched decoder is upper bounded by that of coded modulation and may thus be
lower than in the infinite-interleaved model. We also consider the mutual
information appearing in the analysis of iterative decoding of BICM with EXIT
charts. We show that the corresponding symbol metric has knowledge of the
transmitted symbol and the EXIT mutual information admits a representation as a
pseudo-generalized mutual information, which is in general not achievable. A
different symbol decoding metric, for which the extrinsic side information
refers to the hypothesized symbol, induces a generalized mutual information
lower than the coded modulation capacity.Comment: submitted to the IEEE Transactions on Information Theory. Conference
version in 2008 IEEE International Symposium on Information Theory, Toronto,
Canada, July 200
Mutual Information and Boson Radius in c=1 Critical Systems in One Dimension
We study the generic scaling properties of the mutual information between two
disjoint intervals, in a class of one-dimensional quantum critical systems
described by the c=1 bosonic field theory. A numerical analysis of a spin-chain
model reveals that the mutual information is scale-invariant and depends
directly on the boson radius. We interpret the results in terms of correlation
functions of branch-point twist fields. The present study provides a new way to
determine the boson radius, and furthermore demonstrates the power of the
mutual information to extract more refined information of conformal field
theory than the central charge.Comment: 4.1 pages, 5 figure
Directed Flow of Information in Chimera States
We investigated interactions within chimera states in a phase oscillator
network with two coupled subpopulations. To quantify interactions within and
between these subpopulations, we estimated the corresponding (delayed) mutual
information that -- in general -- quantifies the capacity or the maximum rate
at which information can be transferred to recover a sender's information at
the receiver with a vanishingly low error probability. After verifying their
equivalence with estimates based on the continuous phase data, we determined
the mutual information using the time points at which the individual phases
passed through their respective Poincar\'{e} sections. This stroboscopic view
on the dynamics may resemble, e.g., neural spike times, that are common
observables in the study of neuronal information transfer. This discretization
also increased processing speed significantly, rendering it particularly
suitable for a fine-grained analysis of the effects of experimental and model
parameters. In our model, the delayed mutual information within each
subpopulation peaked at zero delay, whereas between the subpopulations it was
always maximal at non-zero delay, irrespective of parameter choices. We
observed that the delayed mutual information of the desynchronized
subpopulation preceded the synchronized subpopulation. Put differently, the
oscillators of the desynchronized subpopulation were 'driving' the ones in the
synchronized subpopulation. These findings were also observed when estimating
mutual information of the full phase trajectories. We can thus conclude that
the delayed mutual information of discrete time points allows for inferring a
functional directed flow of information between subpopulations of coupled phase
oscillators
Equitability, mutual information, and the maximal information coefficient
Reshef et al. recently proposed a new statistical measure, the "maximal
information coefficient" (MIC), for quantifying arbitrary dependencies between
pairs of stochastic quantities. MIC is based on mutual information, a
fundamental quantity in information theory that is widely understood to serve
this need. MIC, however, is not an estimate of mutual information. Indeed, it
was claimed that MIC possesses a desirable mathematical property called
"equitability" that mutual information lacks. This was not proven; instead it
was argued solely through the analysis of simulated data. Here we show that
this claim, in fact, is incorrect. First we offer mathematical proof that no
(non-trivial) dependence measure satisfies the definition of equitability
proposed by Reshef et al.. We then propose a self-consistent and more general
definition of equitability that follows naturally from the Data Processing
Inequality. Mutual information satisfies this new definition of equitability
while MIC does not. Finally, we show that the simulation evidence offered by
Reshef et al. was artifactual. We conclude that estimating mutual information
is not only practical for many real-world applications, but also provides a
natural solution to the problem of quantifying associations in large data sets
- …