Reinforcement Learning on Slow Features of High-Dimensional Input Streams

Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA) network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture is able to learn quite demanding reinforcement learning tasks on high-dimensional visual input streams in a time that is comparable to the time needed when an explicit highly informative low-dimensional state-space representation is given instead of the high-dimensional visual input. The learning speed of the proposed architecture in a task similar to the Morris water maze task is comparable to that found in experimental studies with rats. This study thus supports the hypothesis that slowness learning is one important unsupervised learning principle utilized in the brain to form efficient state representations for behavioral learning

Tensor decompositions of higher-order correlations by nonlinear Hebbian learning

Biological synaptic plasticity exhibits nonlinearities that are not accounted for by classic Hebbian learning rules. Here, we introduce a simple family of generalized nonlinear Hebbian learning rules. We study the computations implemented by their dynamics in the simple setting of a neuron receiving feedforward inputs. These nonlinear Hebbian rules allow a neuron to learn tensor decompositions of its higher-order input correlations. The particular input correlation decomposed and the form of the decomposition depend on the location of nonlinearities in the plasticity rule. For simple, biologically motivated parameters, the neuron learns eigenvectors of higher-order input correlation tensors. We prove that tensor eigenvectors are attractors and determine their basins of attraction. We calculate the volume of those basins, showing that the dominant eigenvector has the largest basin of attraction. We then study arbitrary learning rules and find that any learning rule that admits a finite Taylor expansion into the neural input and output also has stable equilibria at generalized eigenvectors of higher-order input correlation tensors. Nonlinearities in synaptic plasticity thus allow a neuron to encode higher-order input correlations in a simple fashion.https://proceedings.neurips.cc/paper/2021/hash/5e34a2b4c23f4de585fb09a7f546f527-Abstract.htm

Une approche géométrique à l'analyse en composantes indépendantes

Représentation du signal dans une base appropriée -- Une approche basée sur les facteurs indépendants -- Détermination des facteurs indépendants -- Cas de deux dimensions (p = 2) : détermination des composantes indépendantes en utilisant la colinéarité des éléments de Z(X) -- La forme particulière de Zn(X) -- Analyse des conditions d'orthogonalité -- Expériences numériques -- Le cas 2D : détermination des composantes indépendantes par la minimisation d'une fonction objective -- Définition de la méthode II -- Analyse de la méthode II -- Comparaison des deux méthodes -- Comparaison de la méthode I avec la méthode FastICA -- Cas de trois composantes (p = 3) -- Description de la méthode

Adaptive blind signal separation.

by Chi-Chiu Cheung.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.Includes bibliographical references (leaves 124-131).Abstract --- p.iAcknowledgments --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- The Blind Signal Separation Problem --- p.1Chapter 1.2 --- Contributions of this Thesis --- p.3Chapter 1.3 --- Applications of the Problem --- p.4Chapter 1.4 --- Organization of the Thesis --- p.5Chapter 2 --- The Blind Signal Separation Problem --- p.7Chapter 2.1 --- The General Blind Signal Separation Problem --- p.7Chapter 2.2 --- Convolutive Linear Mixing Process --- p.8Chapter 2.3 --- Instantaneous Linear Mixing Process --- p.9Chapter 2.4 --- Problem Definition and Assumptions in this Thesis --- p.9Chapter 3 --- Literature Review --- p.13Chapter 3.1 --- Previous Works on Blind Signal Separation with Instantaneous Mixture --- p.13Chapter 3.1.1 --- Algebraic Approaches --- p.14Chapter 3.1.2 --- Neural approaches --- p.15Chapter 3.2 --- Previous Works on Blind Signal Separation with Convolutive Mixture --- p.20Chapter 4 --- The Information-theoretic ICA Scheme --- p.22Chapter 4.1 --- The Bayesian YING-YANG Learning Scheme --- p.22Chapter 4.2 --- The Information-theoretic ICA Scheme --- p.25Chapter 4.2.1 --- Derivation of the cost function from YING-YANG Machine --- p.25Chapter 4.2.2 --- Connections to previous information-theoretic approaches --- p.26Chapter 4.2.3 --- Derivation of the Algorithms --- p.27Chapter 4.2.4 --- Roles and Constraints on the Nonlinearities --- p.30Chapter 4.3 --- Direction and Motivation for the Analysis of the Nonlinearity --- p.30Chapter 5 --- Properties of the Cost Function and the Algorithms --- p.32Chapter 5.1 --- Lemmas and Corollaries --- p.32Chapter 5.1.1 --- Singularity of J(V) --- p.33Chapter 5.1.2 --- Continuity of J(V) --- p.34Chapter 5.1.3 --- Behavior of J(V) along a radially outward line --- p.35Chapter 5.1.4 --- Impossibility of divergence of the information-theoretic ICA al- gorithms with a large class of nonlinearities --- p.36Chapter 5.1.5 --- Number and stability of correct solutions in the 2-channel case --- p.37Chapter 5.1.6 --- Scale for the equilibrium points --- p.39Chapter 5.1.7 --- Absence of local maximum of J(V) --- p.43Chapter 6 --- The Algorithms with Cubic Nonlinearity --- p.44Chapter 6.1 --- The Cubic Nonlinearity --- p.44Chapter 6.2 --- Theoretical Results on the 2-Channel Case --- p.46Chapter 6.2.1 --- Equilibrium points --- p.46Chapter 6.2.2 --- Stability of the equilibrium points --- p.49Chapter 6.2.3 --- An alternative proof for the stability of the equilibrium points --- p.50Chapter 6.2.4 --- Convergence Analysis --- p.52Chapter 6.3 --- Experiments on the 2-Channel Case --- p.53Chapter 6.3.1 --- Experiments on two sub-Gaussian sources --- p.54Chapter 6.3.2 --- Experiments on two super-Gaussian sources --- p.55Chapter 6.3.3 --- Experiments on one super-Gaussian source and one sub-Gaussian source which are globally sub-Gaussian --- p.57Chapter 6.3.4 --- Experiments on one super-Gaussian source and one sub-Gaussian source which are globally super-Gaussian --- p.59Chapter 6.3.5 --- Experiments on asymmetric exponentially distributed signals .。 --- p.60Chapter 6.3.6 --- Demonstration on exactly and nearly singular initial points --- p.61Chapter 6.4 --- Theoretical Results on the 3-Channel Case --- p.63Chapter 6.4.1 --- Equilibrium points --- p.63Chapter 6.4.2 --- Stability --- p.66Chapter 6.5 --- Experiments on the 3-Channel Case --- p.66Chapter 6.5.1 --- Experiments on three pairwise globally sub-Gaussian sources --- p.67Chapter 6.5.2 --- Experiments on three sources consisting of globally sub-Gaussian and globally super-Gaussian pairs --- p.67Chapter 6.5.3 --- Experiments on three pairwise globally super-Gaussian sources --- p.69Chapter 7 --- Nonlinearity and Separation Capability --- p.71Chapter 7.1 --- Theoretical Argument --- p.71Chapter 7.1.1 --- Nonlinearities that strictly match the source distribution --- p.72Chapter 7.1.2 --- Nonlinearities that loosely match the source distribution --- p.72Chapter 7.2 --- Experiment Verification --- p.76Chapter 7.2.1 --- Experiments on reversed sigmoid --- p.76Chapter 7.2.2 --- Experiments on the cubic root nonlinearity --- p.77Chapter 7.2.3 --- Experimental verification of Theorem 2 --- p.77Chapter 7.2.4 --- Experiments on the MMI algorithm --- p.78Chapter 8 --- Implementation with Mixture of Densities --- p.80Chapter 8.1 --- Implementation of the Information-theoretic ICA scheme with Mixture of Densities --- p.80Chapter 8.1.1 --- The mixture of densities --- p.81Chapter 8.1.2 --- Derivation of the algorithms --- p.82Chapter 8.2 --- Experimental Verification on the Nonlinearity Adaptation --- p.84Chapter 8.2.1 --- Experiment 1: Two channels of sub-Gaussian sources --- p.84Chapter 8.2.2 --- Experiment 2: Two channels of super-Gaussian sources --- p.85Chapter 8.2.3 --- Experiment 3: Three channels of different signals --- p.89Chapter 8.3 --- Seeking the Simplest Workable Mixtures of Densities ......... .。 --- p.91Chapter 8.3.1 --- Number of components --- p.91Chapter 8.3.2 --- Mixture of two densities with only biases changeable --- p.93Chapter 9 --- ICA with Non-Kullback Cost Function --- p.97Chapter 9.1 --- Derivation of ICA Algorithms from Non-Kullback Separation Functionals --- p.97Chapter 9.1.1 --- Positive Convex Divergence --- p.97Chapter 9.1.2 --- Lp Divergence --- p.100Chapter 9.1.3 --- De-correlation Index --- p.102Chapter 9.2 --- Experiments on the ICA Algorithm Based on Positive Convex Divergence --- p.103Chapter 9.2.1 --- Experiments on the algorithm with fixed nonlinearities --- p.103Chapter 9.2.2 --- Experiments on the algorithm with mixture of densities --- p.106Chapter 10 --- Conclusions --- p.107Chapter A --- Proof for Stability of the Equilibrium Points of the Algorithm with Cubic Nonlinearity on Two Channels of Signals --- p.110Chapter A.1 --- Stability of Solution Group A --- p.110Chapter A.2 --- Stability of Solution Group B --- p.111Chapter B --- Proof for Stability of the Equilibrium Points of the Algorithm with Cubic Nonlinearity on Three Channels of Signals --- p.119Chapter C --- Proof for Theorem2 --- p.122Bibliography --- p.12

Técnicas híbridas de processamento de sinais biomédicos implementadas com redes neurais artificiais

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico.O presente trabalho aborda o desenvolvimento e avaliação de técnicas híbridas de processamento de sinais voltadas para sinais biomédicos com ênfase na implementação usando redes neurais artificiais (RNAs). Quatro formas básicas de hibridação diferenciadas pelo grau de interação entre as características e propriedades das técnicas constituintes são abordadas: a hibridação seqüencial, paralela, auxiliar e encastoada. A hibridação seqüencial da análise em componentes independentes (ACI) com a promediação e a hibridação auxiliar e seqüencial da transformada wavelet com redes neurais artificiais são propostas e investigadas para o processamento de registros eletrocardiográficos de alta resolução (ECGAR). A primeira técnica objetiva atenuar as interferências no ECGAR e a segunda extrair características espectro-temporais do ECGAR e classificar ECGARs como de indivíduos com ou sem potenciais tardios ventriculares. Na avaliação da primeira técnica os resultados são comparados com o uso isolado da promediação, resultando em uma melhora de 4 dB na relação sinal-ruído. Na segunda técnica obteve-se 91% de acerto na classificação, comparável a outros trabalhos envolvendo RNAs, acrescentando-se a possibilidade de interpretação do processamento efetuado pela RNA

Estudio de los circuitos funcionales de hipocampo mediante generadores multicelulares del potencial extracelular

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Facultad de Ciencias. Fecha de lectura: 06-09-201

Analyse en composantes indépendantes pour la caractérisation d'images hyperspectrales en télédétection

En réponse partielle aux problèmes écologiques actuels, l'imagerie hyperspectrale ambitionne de connaître la composition locale d'une parcelle agraire en recherchant sa signature spectrale car celle-ci caractérise de façon unique un élément. Cependant, cette signature s'avère être un mélange pondéré de celles des éléments s'y trouvant. Afin de récupérer leurs signatures à partir du mélange, l'analyse en composantes indépendantes (ACI) est légitimement utilisée! Malgré le nombre restreint de travaux sur l'ACI en hyperspectral, devant sa popularité en traitement de signal, nous l'avons appliquée en utilisant l'algorithme FastICA, méthode la plus récente et efficace, d'abord sur des images et des signaux classiques (pour constater son efficacité), puis sur une base de signatures étalons. Le but est de comparer les composantes indépendantes à une base référencée pour former les paires les plus ressemblantes. Cependant, du fait des ambiguïtés et d'absence de critère de validation de l'ACI, il est impossible de prédire ni vérifier les paires. Pour y remédier, notre protocole expérimental est divisé entre comparaisons «théorique» et «pratique», basées sur des niveaux de confiance, permettant de former les paires considérées justes d'une part (base partielle) et expérimentales d'autre part (base totale) qui, comparées, déterminent le succès d'association. Les résultats, assujettis à deux seuils de confiance relatifs, sont excellents pour les signaux, bons pour les images mais globalement médiocres pour les signatures. La raison principale est un effet beaucoup plus visible en ce cas de la subjectivité de la prise de décision et de la décorrélation inévitable entraînant déformations et trop grande dépendance à la base. Cependant, les résultats deviennent très satisfaisants pour une sélection adéquate (cultures, arbres et minéraux). Pour tenter encore de les améliorer, des recommandations constructives ont été proposées, afin de poser le deuxième échelon de ce travail, qui se voulait novateur

Independent component analysis techniques and their performance evaluation for electroencephalography.

The ongoing electrical activity of the brain is known as the electroencephalogram (EEG). Evoked potentials (EPs) are voltage deviations in the EEG elicited in association with stimuli. EPs provide clinical information by allowing an insight into neurological processes. The amplitude of EPs is typically several times less than the background EEG. The background EEG has the effect of obscuring the EPs and therefore appropriate signal processing is required for their recovery. The EEG waveforms recorded from electrodes placed on the scalp contains the ongoing background EEG, EPs from various brain sources as well as signal components with sources external to the brain. An example of externally generated signal which is picked up by the electrodes on the scalp is the electrooculogram (EOG). This signal is generated by the eyes when eye movements or blinks are performed. Saccade-related EEG waveforms were recorded from 7 normal subjects. A signal source separation technique, namely the independent component analysis (ICA) algorithm of Bell and Sejnowski (hereafter refereed to as BS_ICA), was employed to analyse the recorded waveforms. The effectiveness of the BS_ICA algorithm as well as that of the ICA algorithm of Cardoso, was investigated for removing ocular artefact (OA) from the EEG. It was quantitavely demonstrated that both ICA algorithms were more effective than the conventional correlation-based techniques for removing the OA from the EEG.A novel iterative synchronised averaging method for EPs was devised. The method optimally synchronised the waveforms from successive trials with respect to the event of interest prior to averaging and thus preserved the features of the signals components that were time-locked to the event. The recorded EEG waveforms were analysed using BS_ICA and saccade-related components (frontal and occipital pre-saccadic potentials, and the lambda wave) were extracted and their scalp topographies were obtained. This initial study highlighted some limitations of the conventional ICA approach of Bell and Sejnowski for analysing saccade-related EEG waveforms.Novel techniques were devised in order to improve the performance of the ICA algorithm of Bell and Sejnowski for extracting the lambda wave EP component. One approach involved designing a template-model that represented the temporal characteristics of a lambda wave. Its incorporation into the BS_ICA algorithm improved the signal source separation ability of the algorithm for extracting the lambda wave from the EEG waveforms. The second approach increased the effective length of the recorded EEG traces prior to their processing by the BS_ICA algorithm. This involved abutting EEG traces from an appropriate number of successive trials (a trial was a set of waveforms recorded from 64 electrode locations in a experiment involving a saccade performance). It was quantitatively demonstrated that the process of abutting EEG waveforms was a valuable pre-processing operation for the ICA algorithm of Bell and Sejnowski when extracting the lambda wave.A Fuzzy logic method was implemented to identify BS_ICA-extracted single-trial saccade-related lambda waves. The method provided an effective means to automate the identification of the lambda waves extracted by BS_ICA. The approach correctly identified the single-trial lambda waves with an Accuracy of 97.4%