Learning as a Nonlinear Line of Attraction for Pattern Association, Classification and Recognition

Development of a mathematical model for learning a nonlinear line of attraction is presented in this dissertation, in contrast to the conventional recurrent neural network model in which the memory is stored in an attractive fixed point at discrete location in state space. A nonlinear line of attraction is the encapsulation of attractive fixed points scattered in state space as an attractive nonlinear line, describing patterns with similar characteristics as a family of patterns. It is usually of prime imperative to guarantee the convergence of the dynamics of the recurrent network for associative learning and recall. We propose to alter this picture. That is, if the brain remembers by converging to the state representing familiar patterns, it should also diverge from such states when presented by an unknown encoded representation of a visual image. The conception of the dynamics of the nonlinear line attractor network to operate between stable and unstable states is the second contribution in this dissertation research. These criteria can be used to circumvent the plasticity-stability dilemma by using the unstable state as an indicator to create a new line for an unfamiliar pattern. This novel learning strategy utilizes stability (convergence) and instability (divergence) criteria of the designed dynamics to induce self-organizing behavior. The self-organizing behavior of the nonlinear line attractor model can manifest complex dynamics in an unsupervised manner. The third contribution of this dissertation is the introduction of the concept of manifold of color perception. The fourth contribution of this dissertation is the development of a nonlinear dimensionality reduction technique by embedding a set of related observations into a low-dimensional space utilizing the result attained by the learned memory matrices of the nonlinear line attractor network. Development of a system for affective states computation is also presented in this dissertation. This system is capable of extracting the user\u27s mental state in real time using a low cost computer. It is successfully interfaced with an advanced learning environment for human-computer interaction

Gaussian Weighted Neighborhood Connectivity of Nonlinear Line Attractor for Learning Complex Manifolds

The human brain has the capability to process high quantities of data quickly for detection and recognition tasks. These tasks are made simpler by the understanding of data, which intentionally removes redundancies found in higher dimensional data and maps the data onto a lower dimensional space. The brain then encodes manifolds created in these spaces, which reveal a specific state of the system. We propose to use a recurrent neural network, the nonlinear line attractor (NLA) network, for the encoding of these manifolds as specific states, which will draw untrained data towards one of the specific states that the NLA network has encoded. We propose a Gaussian-weighted modular architecture for reducing the computational complexity of the conventional NLA network. The proposed architecture uses a neighborhood approach for establishing the interconnectivity of neurons to obtain the manifolds. The modified NLA network has been implemented and tested on the Electro-Optic Synthetic Vehicle Model Database created by the Air Force Research Laboratory (AFRL), which contains a vast array of high resolution imagery with several different lighting conditions and camera views. It is observed that the NLA network has the capability for representing high dimensional data for the recognition of the objects of interest through its new learning strategy. A nonlinear dimensionality reduction scheme based on singular value decomposition has found to be very effective in providing a low dimensional representation of the dataset. Application of the reduced dimensional space on the modified NLA algorithm would provide fast and more accurate recognition performance for real time applications

Gaussian Nonlinear Line Attractor for Learning Multidimensional Data

The human brain’s ability to extract information from multidimensional data modeled by the Nonlinear Line Attractor (NLA), where nodes are connected by polynomial weight sets. Neuron connections in this architecture assumes complete connectivity with all other neurons, thus creating a huge web of connections. We envision that each neuron should be connected to a group of surrounding neurons with weighted connection strengths that reduces with proximity to the neuron. To develop the weighted NLA architecture, we use a Gaussian weighting strategy to model the proximity, which will also reduce the computation times significantly. Once all data has been trained in the NLA network, the weight set can be reduced using a locality preserving nonlinear dimensionality reduction technique. By reducing the weight sets using this technique, we can reduce the amount of outputs for recognition tasks. An appropriate distance measure can then be used for comparing testing data and the trained data when processed through the NLA architecture. It is observed that the proposed GNLA algorithm reduces training time significantly and is able to provide even better recognition using fewer dimensions than the original NLA algorithm. We have tested this algorithm and showed that it works well in different datasets, including the EO Synthetic Vehicle database and the Sheffield face database

ARTIFICIAL INTELLIGENCE APPLIED TO THE STUDY OF CONSCIOUS PERCEPTIVE STATES

My PhD research consists of the processing of signals from a 14-electrode EEG system, connected to immersive glasses that allow for a realistic visual experience and for the investigation of the brain network in order to identify signal features corresponding to different perceptive and cognitive stimuli. The aim of the research is to implement a procedure that identifies correspondences among EEG signals and chaotic attractors. The chaotic attractors can be defined as a trajectory of a dynamical system, contained in a defined volume of phase space. A dynamical system can have chaotic behavior, i.e. an organized (but not periodic) behavior sensitive to the initial conditions. EEG signals can be considered dynamical systems. In this work a custom Artificial Neural Network (ITSOM) processes individual signals or many signals simultaneously. The sequence of the ITSOM winning nodes tends to repeat itself creating a time series of chaotic attractors. The ITSOM attributes similar codes to attractors emerging from similar brain states, perceptions and emotions. These attractors are isomorphic to the attractors in which the corresponding dynamical system (the signal time series) is evolving and univocally characterize the input element that produces them. If the attractors are chaotic, this means that the signals are individually self-organized or, by examining more signals together, there is a form of coherence among signals. The ITSOM network memorizes the time series of the winning nodes. The cumulative scores for each input are normalized following the z standardized variable distribution. Attractors are labeled with a binary code that univocally identifies them, and the flexibility of the Artificial Neural Network allows attributing the same codes to similar dynamical events. During the experiment, the subject is looking at the screen while different shades of colors, yellow, red and blue are displayed. Each stimulation lasts five seconds, between stimuli there is a black screen, used to reset the previous color stimuli. The collected results show, as forecast, many correspondences among binary codes coming from similar stimuli. The thesis provides a detailed description of these results

A Subspace Projection Methodology for Nonlinear Manifold Based Face Recognition

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this dissertation. Feature extraction methods aim to find compact representations of data that are easy to classify. Measurements with similar values are grouped to same category, while those with differing values are deemed to be of separate categories. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature space is developed. Least squares estimation approach that utilizes interdependency between points in training patterns is used to form the nonlinear region. The proposed feature extraction strategy is employed to improve face recognition accuracy under varying illumination conditions and facial expressions. Though the face features show variations under these conditions, the features of one individual tend to cluster together and can be considered as a neighborhood. Low dimensional representations of face patterns in the feature space may lie in a nonlinear constraint region, which when modeled leads to efficient pattern classification. A feature space encompassing multiple pattern classes can be trained by modeling a separate constraint region for each pattern class and obtaining a mean constraint region by averaging all the individual regions. Unlike most other nonlinear techniques, the proposed method provides an easy intuitive way to place new points onto a nonlinear region in the feature space. The proposed feature extraction and classification method results in improved accuracy when compared to the classical linear representations. Face recognition accuracy is further improved by introducing the concepts of modularity, discriminant analysis and phase congruency into the proposed method. In the modular approach, feature components are extracted from different sub-modules of the images and concatenated to make a single vector to represent a face region. By doing this we are able to extract features that are more representative of the local features of the face. When projected onto an arbitrary line, samples from well formed clusters could produce a confused mixture of samples from all the classes leading to poor recognition. Discriminant analysis aims to find an optimal line orientation for which the data classes are well separated. Experiments performed on various databases to evaluate the performance of the proposed face recognition technique have shown improvement in recognition accuracy, especially under varying illumination conditions and facial expressions. This shows that the integration of multiple subspaces, each representing a part of a higher order nonlinear function, could represent a pattern with variability. Research work is progressing to investigate the effectiveness of subspace projection methodology for building manifolds with other nonlinear functions and to identify the optimum nonlinear function from an object classification perspective

Complexity Science in Human Change

This reprint encompasses fourteen contributions that offer avenues towards a better understanding of complex systems in human behavior. The phenomena studied here are generally pattern formation processes that originate in social interaction and psychotherapy. Several accounts are also given of the coordination in body movements and in physiological, neuronal and linguistic processes. A common denominator of such pattern formation is that complexity and entropy of the respective systems become reduced spontaneously, which is the hallmark of self-organization. The various methodological approaches of how to model such processes are presented in some detail. Results from the various methods are systematically compared and discussed. Among these approaches are algorithms for the quantification of synchrony by cross-correlational statistics, surrogate control procedures, recurrence mapping and network models.This volume offers an informative and sophisticated resource for scholars of human change, and as well for students at advanced levels, from graduate to post-doctoral. The reprint is multidisciplinary in nature, binding together the fields of medicine, psychology, physics, and neuroscience

Nonlinear Stochastic Modeling and Analysis of Cardiovascular System Dynamics - Diagnostic and Prognostic Applications

The purpose of this investigation is to develop monitoring, diagnostic and prognostic schemes for cardiovascular diseases by studying the nonlinear stochastic dynamics underlying complex heart system. The employment of a nonlinear stochastic analysis combined with wavelet representations can extract effective cardiovascular features, which will be more sensitive to the pathological dynamics instead of the extraneous noises. While conventional statistical and linear systemic approaches have limitations for capturing signal variations resulting from changes in the cardiovascular system states. The research methodology includes signal representation using optimal wavelet function design, feature extraction using nonlinear recurrence analysis, and local recurrence modeling for state prediction.Industrial Engineering & Managemen

Self-Organized Criticality as a Neurodynamical Correlate of Consciousness: A neurophysiological approach to measure states of consciousness based on EEG-complexity features

Background and Objectives This thesis was based on the hypothesis that the physics-derived theoretical framework of self-organized criticality can be applied to the neuronal dynamics of the human brain. From a consciousness science perspective, this is especially appealing as critical brain dynamics imply a vicinity a phase transition, which is associated with optimized information processing functions as well as the largest repertoire of configurations that a system explores throughout its temporal evolution. Hence, self-organised criticality could serve as a neurodynamical correlate for consciousness, which provides the possibility of deriving empirically testable neurophysiological indices suitable to characterise and quantify states of consciousness. The purpose of this work was to experimentally examine the feasibility of the self-organized criticality theory as a correlate for states of consciousness. Therefore, it was aimed at answering the following research questions based on the analysis of three 64 channel EEG datasets: (i) Can signatures of self-organized criticality be found on the level of the EEG in terms of scale-free distribution of neuronal avalanches and the presence of long-range temporal correlations (LRTC) in neuronal oscillations? (ii) Are criticality features suitable to differentiate state of consciousness in the spectrum of wakefulness? (iii) Can the neuronal dynamics be shifted towards the critical point of a phase transition associated with optimized information processing function by mind-body interventions? (iv) Can an explicit relationship to other nonlinear complexity features and power spectral density parameter be identified? (v) Do EEG-based criticality features reflect individual temperament traits? Material and Methods (1): Re-analysis: Thirty participants highly proficient in meditation (mean age 47 years, 11 females/19 males, meditation experience of at least 5 years practice or more than 1000 h of total meditation time) were measured with 64-channel EEG during one session consisting of a task-free baseline resting, a reading condition and three meditation conditions, namely thoughtless emptiness, presence monitoring and focused attention. (2): 64-channel EEG was recorded from 34 participants (mean age 36.0 ±13.4 years, 24 females/ 10 males) before, during and after a professional singing bowl massage. Further, psychometric data was assessed including absorption capacity defined as the individual’s capacity for engaging attentional resources in sensory and imaginative experiences measured by the Tellegen-Absorption Scale (TAS-D), subjective changes in in body sensation, emotional state, and mental state (CSP-14) as well as the phenomenology of consciousness (PCI-K). (3): Electrophysiological data (64 channels of EEG, EOG, ECG, skin conductance, and respiration) was recorded from 116 participants (mean age 40.0 ±13.4 years, 83 females/ 33 males) – in collaboration with the Institute of Psychology, Bundeswehr University Munich - during a task-free baseline resting state. The individual level of sensory processing sensitivity was assessed using the High Sensitive Person Scale (HSPS-G). The datasets were analysed applying analytical tools from self-organized criticality theory (detrended fluctuation analysis, neuronal avalanche analysis), nonlinear complexity algorithms (multiscale entropy, Higuchi’s fractal dimension) and power spectral density. In study 1 and 2, task conditions were contrasted, and effect sizes were compared using a paired two-tailed t-test calculated across participants, and features. T-values were corrected for multiple testing using false discovery rate. To calculate correlations between the EEG features, Spearman’s rank correlation was applied after determining that the distribution was not appropriate for parametric testing by the Shapiro-Wilk test. In addition, in study 1, a discrimination analysis was carried out to determine the classification performance of the EEG features. Here, partial least squares regression and receiver operating characteristics analysis was applied. To determine whether the EEG features reflect individual temperament traits, the individual level of absorption capacity (study 2) and sensory processing sensitivity (study 3) was correlated with the EEG features using Spearman’s rank correlation. Results Signatures of self-organized criticality in the form of scale-free distribution of neuronal avalanches and long-range temporal correlations (LRTCs) in the amplitude of neural oscillations were observed in three distinct EEG-datasets. EEG criticality as well as complexity features were suitable to characterise distinct states of consciousness. In study 1, compared to the task-free resting condition, all three meditative states revealed significantly reduced long-range temporal correlation with moderate effect sizes (presence monitoring: d= -0.49, p<.001; thoughtless emptiness: d= -0.37, p<.001; and focused attention: d= -0.28, p=.003). The critical exponent was suitable to differentiate between focused attention and presence monitoring (d= -0.32, p=.02). Further, in study 2, the criticality features significantly changed during the course of the experiment, whereby values indicated a shift towards the critical regime during the sound condition. Both analyses of the first and second dataset revealed that the critical exponent was significantly negatively correlated with the sample entropy, the scaling exponent resulting from the DFA denoting the amount of long-range temporal correlations as well as Higuchi’s fractal dimension in each condition, respectively. In addition, the critical scaling exponent was found to be significantly negatively correlated with the trait absorption (Spearman's ρ= -0.39, p= .007), whereas an association between critical dynamics and the level of sensory processing sensitivity could not be verified (study 3). Conclusion The findings of this thesis suggest that neuronal dynamics are governed by the phenomena of self-organized criticality. EEG-based criticality features were shown to be sensitive to detect experimentally induced alterations in the state of consciousness. Further, an explicit relationship with nonlinear measures determining the degree of neuronal complexity was identified. Thus, self-organized criticality seems feasible as a neurodynamical correlate for consciousness with the potential to quantify and characterize states of consciousness. Its agreement with the current most influencing theories in the field of consciousness research is discussed

The neural circuit basis of learning

The astounding capacity for learning ranks among the nervous system’s most impressive features. This thesis comprises studies employing varied approaches to improve understanding, at the level of neural circuits, of the brain’s capacity for learning. The first part of the thesis contains investigations of hippocampal circuitry – both theoretical work and experimental work in the mouse Mus musculus – as a model system for declarative memory. To begin, Chapter 2 presents a theory of hippocampal memory storage and retrieval that reflects nonlinear dendritic processing within hippocampal pyramidal neurons. As a prelude to the experimental work that comprises the remainder of this part, Chapter 3 describes an open source software platform that we have developed for analysis of data acquired with in vivo Ca2+ imaging, the main experimental technique used throughout the remainder of this part of the thesis. As a first application of this technique, Chapter 4 characterizes the content of signaling at synapses between GABAergic neurons of the medial septum and interneurons in stratum oriens of hippocampal area CA1. Chapter 5 then combines these techniques with optogenetic, pharmacogenetic, and pharmacological manipulations to uncover inhibitory circuit mechanisms underlying fear learning. The second part of this thesis focuses on the cerebellum-like electrosensory lobe in the weakly electric mormyrid fish Gnathonemus petersii, as a model system for non-declarative memory. In Chapter 6, we study how short-duration EOD motor commands are recoded into a complex temporal basis in the granule cell layer, which can be used to cancel Purkinje-like cell firing to the longer duration and temporally varying EOD-driven sensory responses. In Chapter 7, we consider not only the temporal aspects of the granule cell code, but also the encoding of body position provided from proprioceptive and efference copy sources. Together these studies clarify how the cerebellum-like circuitry of the electrosensory lobe combines information of different forms and then uses this combined information to predict the complex dependence of sensory responses on body position and timing relative to electric organ discharge