Variations in associative memory design

Ankara : The Department of Electrical and Electronics Engineering and the Institute of Engineering and Sciences of Bilkent University, 1996.Thesis (Master's) -- Bilkent University, 1996.Includes bibliographical references leaves 66-68.This thesis is concerned with the anaiysis and synthesis of neurai networks to be used as associative memories. First considering a discrete-time neurai network modei which uses a quantizer-type muitiievei activation function, a way of seiecting the connection weights is proposed. In addition to this, the idea of overiapping decompositions, which is extensiveiy used in the soiution of iarge-scaie probiems, is appiied to discrete-time neurai networks with binary neurons. 'I’lie necesscuy toois for expansions and contractions are derived, and algorithms for decomposition of a set equiiibria into smaiier dimensionai equiiibria sets and for designing neurai networks for these smaiier ciimensionai equiiibria sets are given. The concept is iiiustrated with various exarnpies.Akar, MehmetM.S

A wavelet-based CMAC for enhanced multidimensional learning

The CMAC (Cerebellar Model Articulation Controller) neural network has been successfully used in control systems and other applications for many years. The network structure is modular and associative, allowing for rapid learning convergence with an ease of implementation in either hardware or software. The rate of convergence of the network is determined largely by the choice of the receptive field shape and the generalization parameter. This research contains a rigorous analysis of the rate of convergence with the standard CMAC, as well as the rate of convergence of networks using other receptive field shape. The effects of decimation from state-space to weight space are examined in detail. This analysis shows CMAC to be an adaptive lowpass filter, where the filter dynamics are governed by the generalization parameter. A more general CMAC is derived using wavelet-based receptive fields and a controllable decimation scheme, that is capable of convergence at any frequency within the Nyquist limits. The flexible decimation structure facilitates the optimization of computation for complex multidimensional problems. The stability of the wavelet-based CMAC is also examined

Medical imaging analysis with artificial neural networks

Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging

Characterising brain connectivity along the lifespan in a rodent model of healthy ageing

The brain parenchyma undergoes several structural changes throughout life, which have a ma- jor impact on its physiological evolution, and which are behaviorally reflected as changes in cognition and ability. A key question is how age-related structural alterations impact the func- tion of the different areas. Functional connectivity, measured as correlation between brain re- gions during the resting state Magnetic Resonance Imaging (MRI), is a quantitative measure of function that can be reliably used to characterize the evolution of the communication between regions across the lifespan. However, most of the works so far have done it with a hypothesis driven approach. The present work aims to identify the functional connectivity patterns of the whole brain during resting state in a rodent model of healthy ageing. For this purpose, we have followed the standard workflow recently proposed in a consensus paper on functional imag- ing processing in preclinical MRI. We have set up a longitudinal functional MRI experiment to measure functional connectivity in rats at different times. Independent component analysis has been used to identify characteristic resting-state networks and compare them between three different ages, corresponding to adulthood to early senescence. The goal is to highlight region- , sex-, and age-specific patterns that drive the physiological decline in cognition observed in senescence, with potential to identify vulnerable regions in and define targets for intervention. Our results uncovered patterns of increased functional connectivity between adulthood and senescence in several key regions controlling the functions known to be affected by age. Such increase in connectivity can be explained as a compensatory mechanism that allows the brain to cope with reduced microstructural integrity. The study of healthy ageing in absence of disease sets the baseline for the identification of pathological conditionsEl parénquima cerebral experimenta varios cambios estructurales a lo largo de la vida, que tienen un gran impacto en su evolución fisiológica, y que se reflejan conductualmente como cambios en la cognición y la capacidad. Una cuestión clave es cómo repercuten las alteraciones estructurales relacionadas con la edad en la función de las distintas áreas. La conectividad fun- cional, medida como correlación entre regiones cerebrales durante la Resonancia Magnética (RM) en estado de reposo, es una medida cuantitativa de la función que puede utilizarse de forma fiable para caracterizar la evolución de la comunicación entre regiones a lo largo de la vida. Sin embargo, la mayoría de los trabajos realizados hasta ahora lo han hecho con un en- foque basado en hipótesis. El presente trabajo pretende identificar los patrones de conectividad funcional de todo el cerebro durante el estado de reposo en un modelo de roedor de envejec- imiento sano. Para ello, hemos seguido el flujo de trabajo estándar propuesto recientemente en un documento de consenso sobre el procesamiento de imágenes funcionales en RM preclínica. Hemos establecido un experimento de RM funcional longitudinal para medir la conectividad funcional en ratas en diferentes momentos. Se ha utilizado el análisis de componentes indepen- dientes para identificar redes características en estado de reposo y compararlas entre tres edades diferentes, correspondientes a la edad adulta y a la senescencia temprana. El objetivo es destacar los patrones específicos de región, sexo y edad que impulsan el declive fisiológico de la cogni- ción observado en la senescencia, con potencial para identificar regiones vulnerables y definir objetivos de intervención. Nuestros resultados descubrieron patrones de aumento de la conec- tividad funcional entre la edad adulta y la senescencia en varias regiones clave que controlan las funciones que se sabe que se ven afectadas por la edad. Este aumento de la conectividad puede explicarse como un mecanismo compensatorio que permite al cerebro hacer frente a la reducción de la integridad microestructural. El estudio del envejecimiento sano en ausencia de enfermedad sienta las bases para la identificación de condiciones patológicasEl parènquima cerebral experimenta diversos canvis estructurals al llarg de la vida, que tenen un gran impacte en la seua evolució fisiològica, i que es reflecteixen conductualment com a canvis en la cognició i la capacitat. Una qüestió clau és com repercuteixen les alteracions estructurals relacionades amb l’edat en la funció de les diferents àrees. La connectivitat funcional, mesurada com a correlació entre regions cerebrals durant la Ressonància Magnètica (RM) en estat de repòs, és una mesura quantitativa de la funció que pot utilitzar-se de manera fiable per a carac- teritzar l’evolució de la comunicació entre regions al llarg de la vida. No obstant això, la majoria dels treballs realitzats fins ara ho han fet amb un enfocament basat en hipòtesi. El present tre- ball pretén identificar els patrons de connectivitat funcional de tot el cervell durant l’estat de repòs en un model de rosegador d’envelliment sa. Per a això, hem seguit el flux de treball estàndard proposat recentment en un document de consens sobre el processament d’imatges funcionals en RM preclínica. Hem establit un experiment de RM funcional longitudinal per a mesurar la connectivitat funcional en rates en diferents moments. S’ha utilitzat l’anàlisi de com- ponents independents per a identificar xarxes característiques en estat de repòs i comparar-les entre tres edats diferents, corresponents a l’edat adulta i a la senescència primerenca. L’objectiu és destacar els patrons específics de regió, sexe i edat que impulsen el declivi fisiològic de la cognició observat en la senescència, amb potencial per a identificar regions vulnerables i definir objectius d’intervenció. Els nostres resultats van descobrir patrons d’augment de la connec- tivitat funcional entre l’edat adulta i la senescència en diverses regions clau que controlen les funcions que se sap que es veuen afectades per l’edat. Aquest augment de la connectivitat pot explicar-se com un mecanisme compensatori que permet al cervell fer front a la reducció de la integritat microestructural. L’estudi de l’envelliment sa en absència de malaltia estableix les bases per a la identificació de condicions patològique

Distributed Bayesian Computation and Self-Organized Learning in Sheets of Spiking Neurons with Local Lateral Inhibition

During the last decade, Bayesian probability theory has emerged as a framework in cognitive science and neuroscience for describing perception, reasoning and learning of mammals. However, our understanding of how probabilistic computations could be organized in the brain, and how the observed connectivity structure of cortical microcircuits supports these calculations, is rudimentary at best. In this study, we investigate statistical inference and self-organized learning in a spatially extended spiking network model, that accommodates both local competitive and large-scale associative aspects of neural information processing, under a unified Bayesian account. Specifically, we show how the spiking dynamics of a recurrent network with lateral excitation and local inhibition in response to distributed spiking input, can be understood as sampling from a variational posterior distribution of a well-defined implicit probabilistic model. This interpretation further permits a rigorous analytical treatment of experience-dependent plasticity on the network level. Using machine learning theory, we derive update rules for neuron and synapse parameters which equate with Hebbian synaptic and homeostatic intrinsic plasticity rules in a neural implementation. In computer simulations, we demonstrate that the interplay of these plasticity rules leads to the emergence of probabilistic local experts that form distributed assemblies of similarly tuned cells communicating through lateral excitatory connections. The resulting sparse distributed spike code of a well-adapted network carries compressed information on salient input features combined with prior experience on correlations among them. Our theory predicts that the emergence of such efficient representations benefits from network architectures in which the range of local inhibition matches the spatial extent of pyramidal cells that share common afferent input

Méthodes géométriques pour la mémoire et l'apprentissage

This thesis is devoted to geometric methods in optimization, learning and neural networks. In many problems of (supervised and unsupervised) learning, pattern recognition, and clustering there is a need to take into account the internal (intrinsic) structure of the underlying space, which is not necessary Euclidean. For Riemannian manifolds we construct computational algorithms for Newton method, conjugate-gradient methods, and some non-smooth optimization methods like the r-algorithm. For this purpose we develop methods for geodesic calculation in submanifolds based on Hamilton equations and symplectic integration. Then we construct a new type of neural associative memory capable of unsupervised learning and clustering. Its learning is based on generalized averaging over Grassmann manifolds. Further extension of this memory involves implicit space transformation and kernel machines. Also we consider geometric algorithms for signal processing and adaptive filtering. Proposed methods are tested for academic examples as well as real-life problems of image recognition and signal processing. Application of proposed neural networks is demonstrated for a complete real-life project of chemical image recognition (electronic nose).Cette these est consacree aux methodes geometriques dans l'optimisation, l'apprentissage et les reseaux neuronaux. Dans beaucoup de problemes de l'apprentissage (supervises et non supervises), de la reconnaissance des formes, et du groupage, il y a un besoin de tenir en compte de la structure interne (intrinseque) de l'espace fondamental, qui n'est pas toujours euclidien. Pour les varietes Riemanniennes nous construisons des algorithmes pour la methode de Newton, les methodes de gradients conjugues, et certaines methodes non-lisses d'optimisation comme r-algorithme. A cette fin nous developpons des methodes pour le calcul des geodesiques dans les sous-varietes bases sur des equations de Hamilton et l'integration symplectique. Apres nous construisons un nouveau type avec de la memoire associative neuronale capable de l'apprentissage non supervise et du groupage (clustering). Son apprentissage est base sur moyennage generalise dans les varietes de Grassmann. Future extension de cette memoire implique les machines a noyaux et transformations de l'espace implicites. Aussi nous considerons des algorithmes geometriques pour le traitement des signaux et le filtrage adaptatif. Les methodes proposees sont testees avec des exemples standard et avec des problemes reels de reconnaissance des images et du traitement des signaux. L'application des reseaux neurologiques proposes est demontree pour un projet reel complet de la reconnaissance des images chimiques (nez electronique)

Signal Processing Combined with Machine Learning for Biomedical Applications

The Master’s thesis is comprised of four projects in the realm of machine learning and signal processing. The abstract of the thesis is divided into four parts and presented as follows, Abstract 1: A Kullback-Leibler Divergence-Based Predictor for Inter-Subject Associative BCI. Inherent inter-subject variability in sensorimotor brain dynamics hinders the transferability of brain-computer interface (BCI) model parameters across subjects. An individual training session is essential for effective BCI control to compensate for variability. We report a Kullback-Leibler Divergence (KLD)-based predictor for inter-subject associative BCI. An online dataset comprising left/right hand, both feet, and tongue motor imagery tasks was used to show correlation between the proposed inter-subject predictor and BCI performance. Linear regression between the KLD predictor and BCI performance showed a strong inverse correlation (r = -0.62). The KLD predictor can act as an indicator for generalized inter-subject associative BCI designs. Abstract 2: Multiclass Sensorimotor BCI Based on Simultaneous EEG and fNIRS. Hybrid BCI (hBCI) utilizes multiple data modalities to acquire brain signals during motor execution (ME) tasks. Studies have shown significant enhancements in the classification of binary class ME-hBCIs; however, four-class ME-hBCI classification is yet to be done using multiclass algorithms. We present a quad-class classification of ME-hBCI tasks from simultaneous EEG-fNIRS recordings. Appropriate features were extracted from EEG-fNIRS signals and combined for hybrid features and classified with support vector machine. Results showed a significant increase in hybrid accuracy over single modalities and show hybrid method’s performance enhancement capability. Abstract 3: Deep Learning for Improved Inter-Subject EEG-fNIRS Hybrid BCI Performance. Multimodality based hybrid BCI has become famous for performance improvement; however, the inherent inter-subject and inter-session variation between participants brain dynamics poses obstacles in achieving high performance. This work presents an inter-subject hBCI to classify right/left-hand MI tasks from simultaneous EEG-fNIRS recordings of 29 healthy subjects. State-of-art features were extracted from EEG-fNIRS signals and combined for hybrid features, and finally, classified using deep Long short-term memory classifier. Results showed an increase in the inter-subject performance for the hybrid system while making the system more robust to brain dynamics change and hints to the feasibility of EEG-fNIRS based inter-subject hBCI. Abstract 4: Microwave Based Glucose Concentration Classification by Machine Learning. Non-invasive blood sugar measurement attracts increased attention in recent years, given the increase in diabetes-related complications and inconvenience in the traditional ways using blood. This work utilized machine learning (ML) algorithms to classify glucose concentration (GC) from the measured broadband microwave scattering signals (S11). An N-type microwave adapter pair was utilized to measure the sweeping frequency scattering-parameter (S-parameter) of the glucose solutions with GC varying from 50-10,000 dg/dL. Dielectric parameters were retrieved from the measured wideband complex S-parameters based on the modified Debye dielectric dispersion model. Results indicate that the best algorithm can achieve a perfect classification accuracy and suggests an alternate way to develop a GC detection method using ML algorithms

Latent Factor Analysis of High-Dimensional Brain Imaging Data

Recent advances in neuroimaging study, especially functional magnetic resonance imaging (fMRI), has become an important tool in understanding the human brain. Human cognitive functions can be mapped with the brain functional organization through the high-resolution fMRI scans. However, the high-dimensional data with the increasing number of scanning tasks and subjects pose a challenge to existing methods that wasn’t optimized for high-dimensional imaging data. In this thesis, I develop advanced data-driven methods to help utilize more available sources of information in order to reveal more robust brain-behavior relationship. In the first chapter, I provide an overview of the current related research in fMRI and my contributions to the field. In the second chapter, I propose two extensions to the connectome-based predictive modeling (CPM) method that is able to combine multiple connectomes when building predictive models. The two extensions are both able to generate higher prediction accuracy than using the single connectome or the average of multiple connectomes, suggesting the advantage of incorporating multiple sources of information in predictive modeling. In the third chapter, I improve CPM from the target behavioral measure’s perspective. I propose another two extensions for CPM that are able to combine multiple available behavioral measures into a composite measure for CPM to predict. The derived composite measures are shown to be predicted more accurately than any other single behavioral measure, suggesting a more robust brainbehavior relationship. In the fourth chapter, I propose a nonlinear dimensionality reduction framework to embed fMRI data from multiple tasks into a low-dimensional space. This framework helps reveal the common brain state in the multiple available tasks while also help discover the differences among these tasks. The results also provide valuable insights into the various prediction performance based on connectomes from different tasks. In the fifth chapter, I propose an another hyerbolic geometry-based brain graph edge embedding framework. The framework is based on Poincar´e embedding and is able to more accurately represent edges in the brain graph in a low-dimensional space than traditional Euclidean geometry-based embedding. Utilizing the embedding, we are able to cluster edges of the brain graph into disjoint clusters. The edge clusters can then be used to define overlapping brain networks and the derived metrics like network overlapping number can be used to investigate functional flexibility of each brain region. Overall, these work provide rich data-driven methods that help understand the brain-behavioral relationship through predictive modeling and low-dimensional data representation

Task switching in the prefrontal cortex

The overall goal of this dissertation is to elucidate the cellular and circuit mechanisms underlying flexible behavior in the prefrontal cortex. We are often faced with situations in which the appropriate behavior in one context is inappropriate in others. If these situations are familiar, we can perform the appropriate behavior without relearning how the context relates to the behavior — an important hallmark of intelligence. Neuroimaging and lesion studies have shown that this dynamic, flexible process of remapping context to behavior (task switching) is dependent on prefrontal cortex, but the precise contributions and interactions of prefrontal subdivisions are still unknown. This dissertation investigates two prefrontal areas that are thought to be involved in distinct, but complementary executive roles in task switching — the dorsolateral prefrontal cortex (dlPFC) and the anterior cingulate cortex (ACC). Using electrophysiological recordings from macaque monkeys, I show that synchronous network oscillations in the dlPFC provide a mechanism to flexibly coordinate context representations (rules) between groups of neurons during task switching. Then, I show that, wheras the ACC neurons can represent rules at the cellular level, they do not play a significant role in switching between contexts — rather they seem to be more related to errors and motivational drive. Finally, I develop a set of web-enabled interactive visualization tools designed to provide a multi-dimensional integrated view of electrophysiological datasets. Taken together, these results contribute to our understanding of task switching by investigating new mechanisms for coordination of neurons in prefrontal cortex, clarifying the roles of prefrontal subdivisions during task switching, and providing visualization tools that enhance exploration and understanding of large, complex and multi-scale electrophysiological data