Information Theoretical Estimators Toolbox

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.Comment: 5 pages; ITE toolbox: https://bitbucket.org/szzoli/ite

Ecosystem Monitoring and Port Surveillance Systems

International audienceIn this project, we should build up a novel system able to perform a sustainable and long term monitoring coastal marine ecosystems and enhance port surveillance capability. The outcomes will be based on the analysis, classification and the fusion of a variety of heterogeneous data collected using different sensors (hydrophones, sonars, various camera types, etc). This manuscript introduces the identified approaches and the system structure. In addition, it focuses on developed techniques and concepts to deal with several problems related to our project. The new system will address the shortcomings of traditional approaches based on measuring environmental parameters which are expensive and fail to provide adequate large-scale monitoring. More efficient monitoring will also enable improved analysis of climate change, and provide knowledge informing the civil authority's economic relationship with its coastal marine ecosystems

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

Using state space differential geometry for nonlinear blind source separation

Given a time series of multicomponent measurements of an evolving stimulus, nonlinear blind source separation (BSS) seeks to find a "source" time series, comprised of statistically independent combinations of the measured components. In this paper, we seek a source time series with local velocity cross correlations that vanish everywhere in stimulus state space. However, in an earlier paper the local velocity correlation matrix was shown to constitute a metric on state space. Therefore, nonlinear BSS maps onto a problem of differential geometry: given the metric observed in the measurement coordinate system, find another coordinate system in which the metric is diagonal everywhere. We show how to determine if the observed data are separable in this way, and, if they are, we show how to construct the required transformation to the source coordinate system, which is essentially unique except for an unknown rotation that can be found by applying the methods of linear BSS. Thus, the proposed technique solves nonlinear BSS in many situations or, at least, reduces it to linear BSS, without the use of probabilistic, parametric, or iterative procedures. This paper also describes a generalization of this methodology that performs nonlinear independent subspace separation. In every case, the resulting decomposition of the observed data is an intrinsic property of the stimulus' evolution in the sense that it does not depend on the way the observer chooses to view it (e.g., the choice of the observing machine's sensors). In other words, the decomposition is a property of the evolution of the "real" stimulus that is "out there" broadcasting energy to the observer. The technique is illustrated with analytic and numerical examples.Comment: Contains 14 pages and 3 figures. For related papers, see http://www.geocities.com/dlevin2001/ . New version is identical to original version except for URL in the bylin

Kernel methods for measuring independence

We introduce two new functionals, the constrained covariance and the kernel mutual information, to measure the degree of independence of random variables. These quantities are both based on the covariance between functions of the random variables in reproducing kernel Hilbert spaces (RKHSs). We prove that when the RKHSs are universal, both functionals are zero if and only if the random variables are pairwise independent. We also show that the kernel mutual information is an upper bound near independence on the Parzen window estimate of the mutual information. Analogous results apply for two correlation-based dependence functionals introduced earlier: we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels, and prove the latter to be an upper bound on the mutual information near independence. The performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis

An immune-inspired, dependence-based approach to blind inversion of wiener systems

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elérica, 2016.Nas últimas décadas, o estudo de métodos para a inversão cega de sistemas de Wiener tem recebido uma atenção signi cativa, especialmente em áreas como a biologia, química, sociologia e na indústria. Um grande número de métodos tem sido desenvolvidos com diferentes abordagens e análises teóricas do problema, que incluem algoritmos de gradiente para minimizar a taxa de informação mútua do sinal extraído, algoritmos genéticos para executar a tarefa de procurar os parâmetros ótimos assim como algoritmos imuno-inspirados. Estes métodos têm como requisito fundamental que o sinal de entrada seja originalmente i.i.d., além de algumas outras condições de suavidade. Cenários de aplicação que cumprem com este requisito podem ser difíceis de ocorrer, na prática; por isso, considerar fontes não-independentes tem se tornado uma importante abordagem. Neste trabalho, propõem-se dois métodos baseados nas funções de autocorrelação e autocorrentropia para explorar a estrutura do tempo de um determinado sinal, com a nalidade de promover a inversão cega dos sistemas de Wiener usando sistemas Hammerstein. Filtros lineares com e sem realimentação são considerados e um algoritmo imuno-inspirado é usado para permitir a otimização de parâmetros sem a necessidade de manipular analiticamente a função custo, ao mesmo tempo que se aumenta a probabilidade de convergência global. Os resultados experimentais indicam que ambas as funções proporcionam meios e cazes para a inversão do sistema e também ilustram o efeito de realimentação linear sobre o desempenho global do sistema.In the last decades, the study of blind inversion of Wiener systems has received signi cant attention, in a special manner in areas such as biology, chemistry, sociology, psychology and industry. A large number of methods have been developed with di erent approaches and theoretical analysis of the problem, which include a gradient algorithm to minimize the mutual information rate of the extracted signal, genetic algorithms to perform the task of searching for the optimal parameters as well as immune-inspired algorithms. These methods have the particular requirement that the input signal must be i.i.d. and, besides some smoothness conditions. This requirement may be hard to be present in real-world problems, hence, considering non-independent sources have become an interesting approach. In this work, we propose two methods based on the autocorrelation and autocorrentropy functions for representing the time structure of a given signal, in order to cope with the unsupervised inversion of Wiener systems by Hammerstein systems. Linear lters with and without feedback are considered and an immune-inspired algorithm is used to allow parameter optimization without the need for explicitly manipulating the cost function, with the additional bene t of increasing the probability of global convergence. The experimental results indicate that both functions provide e ective means for system inversion and also illustrate the e ect of linear feedback on the overall system performance