An Immune-inspired Information-theoretic Approach To The Problem Of Ica Over A Galois Field

The problem of independent component analysis (ICA) was firstly formulated and studied in the context of real-valued signals and mixing models, but, recently, an extension of this original formulation was proposed to deal with the problem within the framework of finite fields. In this work, we propose a strategy to deal with ICA over these fields that presents two novel features: (i) it is based on the use of a cost function built directly from an estimate of the mutual information and (ii) it employs an artificial immune system to perform the search for efficient separating matrices, in contrast with the existing techniques, which are based on search schemes of an exhaustive character. The new proposal is subject to a comparative analysis based on different simulation scenarios and the work is concluded by an analysis of perspectives of practical application to digital and genomic data mining. © 2011 IEEE.618622Hyvarinen, A., Karhunen, J., Oja, E., (2001) Independent Component Analysis., , Wiley-InterscienceComon, P., Jutten, C., (2010) Handbook of Blind Source Separation: Independent Component Analysis and applications., , Academic PressComon, P., Independent component analysis, a new concept? (1994) Signal Processing, 36, pp. 287-314. , MarErdogmus, D., Principe, J.C., From linear adaptive filtering to nonlinear information processing (2006) IEEE Signal Processing Magazine, 23, pp. 14-33Adleman, L.M., Molecular computation of solutions to combinatorial problems (1994) Science, 266, pp. 1021-1024. , NovYeredor, A., ICA in boolean XOR mixtures (2007) LNCS, 4666, pp. 827-835Gutch, H.W., Gruber, P., Theis, F.J., ICA over finite fields (2010) LNCS, 6365, pp. 645-652Yeredor, A., Independent component analysis over galois fields of prime order (2011) IEEE Transactions on Information Theory, 57 (8), pp. 5342-5359. , AugustDe Castro, L.N., Von Zuben, F.J., Learning and optimization using the clonal selection principle (2002) IEEE Transactions on Evolutionary Computation, 6, pp. 239-251. , MarDelfosse, N., Loubaton, P., Adaptive blind separation of independent sources: A deflation approach (1995) Signal Processing, 45, pp. 59-83. , JanDe Castro, L.N., (2006) Fundamentals of Natural Computing: Basic Concepts, Algorithms, and applications., , Chapman & Hall/CRCLearned-Miller, E.G., John III, W.F., ICA using spacings estimates of entropy (2003) Journal of Machine Learning Research, 4, pp. 1271-1295. , DecWaterhouse, W.C., How often do determinants over finite fields vanish? (1987) Discrete Mathematics, 65, pp. 103-104. , JanDias, T.M., Attux, R., Romano, J.M.T., Suyama, R., Blind source separation of post-nonlinear mixtures using evolutionary computation and gaussianization (2009) Independent Component Analysis and Signal Separation, pp. 235-242Wada, C., Nonlinear blind source deconvolution using recurrent prediction-error filters and an artificial immune system (2009) Independent Component Analysis and Signal Separation, pp. 371-378Faria, L.C.B., Rocha, A.S.L., Kleinschmidt, J.H., Palazzo, R., Silva-Filho, M.C., DNA sequences generated by BCH codes over GF (4) (2010) Electronics Letters, 46, pp. 203-204Andrews, G.E., (1986) Q-series: Their Development and Application in Analysis, Number Theory, Combinatorics, Physics and Computer Algebra, , American Mathematical Societ

An Immune-inspired, Information-theoretic Framework For Blind Inversion Of Wiener Systems

This work proposes a new approach to the blind inversion of Wiener systems. A Wiener system is composed of a linear time-invariant (LTI) sub-system followed by a memoryless nonlinear function. The goal is to recover the input signal by knowing just the output of the Wiener system, and the straightforward scheme to achieve this is called the Hammerstein system - apply a memoryless nonlinear mapping followed by a LTI sub-system to the output signal of the Wiener system. If the input of the Wiener system is originally iid and some mild conditions are satisfied, the inversion is possible. Based on this statement and the limitations of relevant previous works, a solution is proposed combining (i) immune-inspired optimization algorithms, (ii) information theory and (iii) IIR filters that yield a robust scheme with a relatively reduced risk of local convergence. Experimental results indicated a similar or superior performance of the new approach, in comparison with two other blind methodologies.1131831Haykin, S., (2001) Adaptive Filter Theory, , 4th ed. Prentice HallJutten, C., Karhunen, J., Advances in blind source separation (BSS) and independent component analysis (ICA) for nonlinear mixtures (2004) Int.J. Neural Syst., 14 (5), pp. 267-292Romano, J.M.T., Attux, R., Cavalcante, C., Suyama, R., (2011) Unsupervised Signal Processing: Channel Equalization and Source Separation, , CRC PressTaleb, A., Solé-Casals, J., Jutten, C., Quasi-nonparametric blind inversion of Wiener systems (2001) IEEE Trans. Signal Process., 49 (5), pp. 917-924Taleb, A., Jutten, C., Source separation in post-nonlinear mixtures (1999) IEEE Trans. Signal Process., 47 (10), pp. 2807-2820Hosseini, S., Deville, Y., Blind separation of linear-quadratic mixtures of real sources using a recurrent structure (2003) Artificial Neural Nets Problem Solving Methods, Lecture Notes in Computer Science, 2687, pp. 241-248Duarte, L., Jutten, C., Moussaoui, S., A Bayesian nonlinear source separation method for smart ion-selective electrode arrays (2009) IEEE Sens. J., 9 (12), pp. 1763-1771Hunter, I.W., Korenberg, M.J., The identification of nonlinear biological systems Wiener and Hammerstein cascade models (1986) Biol. Cybern., 55 (23), pp. 135-144Chatterjee, S.K., Ghosh, S., Das, S., Manzella, V., Vitaletti, A., Masi, E., Santopolo, L., Maharatna, K., Forward and inverse modelling approaches for prediction of light stimulus from electrophysiological response in plants (2014) Measurement, 53, pp. 101-116Bars, R., Nonlinear and long range control of a distillation pilot plant (1990) Proceedings of the 9th IFAC/IFORS Symposium on Identification and System Parameter Estimation, pp. 848-853Das, S., Mukherjee, S., Pan, I., Gupta, A., Identification of the core temperature in a fractional order noisy environment for thermal feedback in nuclear reactors (2011) IEEE Technology Students' Symposium, pp. 180-186Ma, W., Lim, H., Sznaier, M., Camps, O., Risk Adjusted Identification of Wiener Systems (2006) 45th IEEE Conference on Decision and Control, pp. 2512-2517Príncipe, J., (2010) Information Theoretic Learning Renyi's Entropy and Kernel Perspectives, , Springer VerlagCover, T., Thomas, J., (2006) Elements of Information Theory, , 2nd ed. Wiley-InterscienceSolé-Casals, J., Jutten, C., Taleb, A., Parametric approach to blind deconvolution of nonlinear channels (2002) Neurocomputing, 48 (14), pp. 339-355Rojas, F., An evolutionary approach for blind inversion of Wiener systems (2007) Independent Component Analysis and Signal Separation, Lecture Notes in Computer Science, 4666, pp. 260-267Babaie-Zadeh, M., Blind inversion of Wiener system using a minimization-projection (MP) approach (2003) ICA 2003, pp. 681-686Parzen, E., On estimation of a probability density function and mode (1962) Ann. Math. Stat., 33 (3), pp. 1065-1076Solé-Casals, J., Faundez-Zanuy, M., Application of the mutual information minimization to speaker recognition/identification improvement (2006) Neurocomputing, 69 (1315), pp. 1467-1474Zhang, K., Practical method for blind inversion of Wiener systems (2004) 2004 International Joint Conference on Neural Networks, pp. 2163-2168Solé-Casals, J., Jutten, C., Pham, D., Fast approximation of nonlinearities for improving inversion algorithms of pnl mixtures and Wiener systems (2005) Signal Process., 85 (9), pp. 1780-1786Solé-Casals, J., Caiafa, C., A fast gradient approximation for nonlinear blind signal processing (2012) Cogn. Comput., 4 (1063), pp. 1-10Rojas, F., A canonical genetic algorithm for blind inversion of linear channels (2006) Independent Component Analysis and Signal Separation, Lecture Notes in Computer Science, 3889, pp. 238-245Bäck, T., Schwefel, H., An overview of evolutionary algorithms for parameter optimization (1993) Evol. Comput., 1 (1), pp. 1-23Comon, P., Jutten, C., (2010) Handbook of Blind Source Separation, , Academic PressDe Castro, L.N., Von Zuben, F.J., Learning and optimization using the clonal selection principle (2002) IEEE Trans. Evol. Comput., 6 (13), pp. 239-251Darbellay, G.A., Vajda, I., Estimation of the information by an adaptive partitioning of the observation space (1999) IEEE Trans. Inf. Theory, 45 (4), pp. 1315-1321Pham, D.-T., Blind separation of instantaneous mixture of sources based on order statistics (2000) IEEE Trans. Signal Process., 48 (2), pp. 363-375Leon-Garcia, A., (1994) Probability and Random Processes for Electrical Engineering, , Addison WesleyLarue, A., Mars, J., Jutten, C., Frequency-domain blind deconvolution based on mutual information rate (2006) IEEE Trans. Signal Process., 54 (5), pp. 1771-1781Burnet, F.M., Clonal selection and after (1978) Theoretical Immunology, pp. 63-85De Castro, L.N., Timmis, J., (2002) Artificial Immune Systems A New Computational Intelligence Approach, , SpringerDe França, F.O., On the diversity mechanisms of opt-aiNet: A comparative study with fitness sharing (2010) 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1-8Dias, T., Attux, R., Romano, J., Suyama, R., Blind source separation of post-nonlinear mixtures using evolutionary computation and Gaussianization (2009) Independent Component Analysis and Signal Separation, Lecture Notes in Computer Science, 5441, pp. 235-242Wada, C., Consolaro, D., Ferrari, R., Suyama, R., Attux, R., Zuben, F., Nonlinear blind source deconvolution using recurrent prediction-error filters and an artificial immune system (2009) Independent Component Analysis and Signal Separation, Lecture Notes in Computer Science, 5441, pp. 371-378Wolpert, D., Macready, W., No free lunch theorems for optimization (1997) IEEE Trans. Evol. Comput., 1 (1), pp. 67-82Darbellay, G.A., Tichavsky, P., Independent component analysis through direct estimation of the mutual information (2000) ICA 2000, pp. 69-75Costa, J.P., Lagrange, A., Arliaud, A., Acoustic echo cancellation using nonlinear cascade filters (2003) Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP'03, pp. 389-392Sill, J., Monotonic networks (1998) Advances in Neural Information Processing Systems, 10, pp. 661-667Lang, B., Monotonic multi-layer perceptron networks as universal approximators (2005) Artificial Neural Networks: Formal Models and Their Applications, ICANN 2005, pp. 31-3