12 research outputs found
Analysis of Bidirectional Associative Memory using SCSNA and Statistical Neurodynamics
Bidirectional associative memory (BAM) is a kind of an artificial neural
network used to memorize and retrieve heterogeneous pattern pairs. Many efforts
have been made to improve BAM from the the viewpoint of computer application,
and few theoretical studies have been done. We investigated the theoretical
characteristics of BAM using a framework of statistical-mechanical analysis. To
investigate the equilibrium state of BAM, we applied self-consistent signal to
noise analysis (SCSNA) and obtained a macroscopic parameter equations and
relative capacity. Moreover, to investigate not only the equilibrium state but
also the retrieval process of reaching the equilibrium state, we applied
statistical neurodynamics to the update rule of BAM and obtained evolution
equations for the macroscopic parameters. These evolution equations are
consistent with the results of SCSNA in the equilibrium state.Comment: 13 pages, 4 figure
The stability and attractivity of neural associative memories.
Han-bing Ji.Thesis (Ph.D.)--Chinese University of Hong Kong, 1996.Includes bibliographical references (p. 160-163).Microfiche. Ann Arbor, Mich.: UMI, 1998. 2 microfiches ; 11 x 15 cm
New Results for Periodic Solution of High-Order BAM Neural Networks with Continuously Distributed Delays and Impulses
By M-matrix theory, inequality techniques, and Lyapunov functional method, certain sufficient conditions are obtained to ensure the existence, uniqueness, and global exponential stability of periodic solution for a new type of high-order BAM neural networks with continuously distributed delays and impulses. These novel conditions extend and improve some previously known results in the literature. Finally, an illustrative example and its numerical simulation are given to show the feasibility and correctness of the derived criteria
Associative neural networks: properties, learning, and applications.
by Chi-sing Leung.Thesis (Ph.D.)--Chinese University of Hong Kong, 1994.Includes bibliographical references (leaves 236-244).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Background of Associative Neural Networks --- p.1Chapter 1.2 --- A Distributed Encoding Model: Bidirectional Associative Memory --- p.3Chapter 1.3 --- A Direct Encoding Model: Kohonen Map --- p.6Chapter 1.4 --- Scope and Organization --- p.9Chapter 1.5 --- Summary of Publications --- p.13Chapter I --- Bidirectional Associative Memory: Statistical Proper- ties and Learning --- p.17Chapter 2 --- Introduction to Bidirectional Associative Memory --- p.18Chapter 2.1 --- Bidirectional Associative Memory and its Encoding Method --- p.18Chapter 2.2 --- Recall Process of BAM --- p.20Chapter 2.3 --- Stability of BAM --- p.22Chapter 2.4 --- Memory Capacity of BAM --- p.24Chapter 2.5 --- Error Correction Capability of BAM --- p.28Chapter 2.6 --- Chapter Summary --- p.29Chapter 3 --- Memory Capacity and Statistical Dynamics of First Order BAM --- p.31Chapter 3.1 --- Introduction --- p.31Chapter 3.2 --- Existence of Energy Barrier --- p.34Chapter 3.3 --- Memory Capacity from Energy Barrier --- p.44Chapter 3.4 --- Confidence Dynamics --- p.49Chapter 3.5 --- Numerical Results from the Dynamics --- p.63Chapter 3.6 --- Chapter Summary --- p.68Chapter 4 --- Stability and Statistical Dynamics of Second order BAM --- p.70Chapter 4.1 --- Introduction --- p.70Chapter 4.2 --- Second order BAM and its Stability --- p.71Chapter 4.3 --- Confidence Dynamics of Second Order BAM --- p.75Chapter 4.4 --- Numerical Results --- p.82Chapter 4.5 --- Extension to higher order BAM --- p.90Chapter 4.6 --- Verification of the conditions of Newman's Lemma --- p.94Chapter 4.7 --- Chapter Summary --- p.95Chapter 5 --- Enhancement of BAM --- p.97Chapter 5.1 --- Background --- p.97Chapter 5.2 --- Review on Modifications of BAM --- p.101Chapter 5.2.1 --- Change of the encoding method --- p.101Chapter 5.2.2 --- Change of the topology --- p.105Chapter 5.3 --- Householder Encoding Algorithm --- p.107Chapter 5.3.1 --- Construction from Householder Transforms --- p.107Chapter 5.3.2 --- Construction from iterative method --- p.109Chapter 5.3.3 --- Remarks on HCA --- p.111Chapter 5.4 --- Enhanced Householder Encoding Algorithm --- p.112Chapter 5.4.1 --- Construction of EHCA --- p.112Chapter 5.4.2 --- Remarks on EHCA --- p.114Chapter 5.5 --- Bidirectional Learning --- p.115Chapter 5.5.1 --- Construction of BL --- p.115Chapter 5.5.2 --- The Convergence of BL and the memory capacity of BL --- p.116Chapter 5.5.3 --- Remarks on BL --- p.120Chapter 5.6 --- Adaptive Ho-Kashyap Bidirectional Learning --- p.121Chapter 5.6.1 --- Construction of AHKBL --- p.121Chapter 5.6.2 --- Convergent Conditions for AHKBL --- p.124Chapter 5.6.3 --- Remarks on AHKBL --- p.125Chapter 5.7 --- Computer Simulations --- p.126Chapter 5.7.1 --- Memory Capacity --- p.126Chapter 5.7.2 --- Error Correction Capability --- p.130Chapter 5.7.3 --- Learning Speed --- p.157Chapter 5.8 --- Chapter Summary --- p.158Chapter 6 --- BAM under Forgetting Learning --- p.160Chapter 6.1 --- Introduction --- p.160Chapter 6.2 --- Properties of Forgetting Learning --- p.162Chapter 6.3 --- Computer Simulations --- p.168Chapter 6.4 --- Chapter Summary --- p.168Chapter II --- Kohonen Map: Applications in Data compression and Communications --- p.170Chapter 7 --- Introduction to Vector Quantization and Kohonen Map --- p.171Chapter 7.1 --- Background on Vector quantization --- p.171Chapter 7.2 --- Introduction to LBG algorithm --- p.173Chapter 7.3 --- Introduction to Kohonen Map --- p.174Chapter 7.4 --- Chapter Summary --- p.179Chapter 8 --- Applications of Kohonen Map in Data Compression and Communi- cations --- p.181Chapter 8.1 --- Use Kohonen Map to design Trellis Coded Vector Quantizer --- p.182Chapter 8.1.1 --- Trellis Coded Vector Quantizer --- p.182Chapter 8.1.2 --- Trellis Coded Kohonen Map --- p.188Chapter 8.1.3 --- Computer Simulations --- p.191Chapter 8.2 --- Kohonen MapiCombined Vector Quantization and Modulation --- p.195Chapter 8.2.1 --- Impulsive Noise in the received data --- p.195Chapter 8.2.2 --- Combined Kohonen Map and Modulation --- p.198Chapter 8.2.3 --- Computer Simulations --- p.200Chapter 8.3 --- Error Control Scheme for the Transmission of Vector Quantized Data --- p.213Chapter 8.3.1 --- Motivation and Background --- p.214Chapter 8.3.2 --- Trellis Coded Modulation --- p.216Chapter 8.3.3 --- "Combined Vector Quantization, Error Control, and Modulation" --- p.220Chapter 8.3.4 --- Computer Simulations --- p.223Chapter 8.4 --- Chapter Summary --- p.226Chapter 9 --- Conclusion --- p.232Bibliography --- p.23
The (Diverse) Company You Keep: Content and Structure of Immigrants' Social Networks as a Window Into Intercultural Relations in Catalonia
This research examines how the social networks of immigrants residing in a European bicultural and bilingual context (Catalonia) relate to levels of adjustment (both psychological and sociocultural) and to bicultural identity integration (BII). Moroccan, Pakistani, Ecuadorian, and Romanian immigrants residing in Barcelona nominated 25 individuals (i.e., alters) from their habitual social networks and provided demographic (e.g., ethnicity), relationship type (e.g., family, friend, neighbor), and structural (who knew whom) information for each of these alters. Even after controlling for individual-level demographic and acculturation variables, the content and structure of immigrants’ personal social networks had unique associations with both types of adjustment and with BII. Specifically, the overall degree of cultural diversity in the network and the amount of Catalan (but not Spanish) "weak" ties (i.e., acquaintances, colleagues, neighbors) positively predicted these outcomes. Amount of interconnectedness between local coethnic and Catalan/Spanish alters also predicted sociocultural adjustment and BII positively. Finally, against a "culture and language similarity" hypothesis, Moroccan and Pakistani participants had social networks that were more culturally integrated, relative to Ecuadorians and Romanians. Results from this study attest to the importance of examining actual intercultural relations and going beyond individuals’ reported acculturation preferences to understand immigrants’ overall adaptation and cultural identity dynamics. Furthermore, results highlight the interplay between interculturalism experienced at the intrapersonal, subjective level (i.e., BII), and at the meso-level (i.e., having culturally diverse networks that also include interethnic ties among alters)
A Review of Findings from Neuroscience and Cognitive Psychology as Possible Inspiration for the Path to Artificial General Intelligence
This review aims to contribute to the quest for artificial general
intelligence by examining neuroscience and cognitive psychology methods for
potential inspiration. Despite the impressive advancements achieved by deep
learning models in various domains, they still have shortcomings in abstract
reasoning and causal understanding. Such capabilities should be ultimately
integrated into artificial intelligence systems in order to surpass data-driven
limitations and support decision making in a way more similar to human
intelligence. This work is a vertical review that attempts a wide-ranging
exploration of brain function, spanning from lower-level biological neurons,
spiking neural networks, and neuronal ensembles to higher-level concepts such
as brain anatomy, vector symbolic architectures, cognitive and categorization
models, and cognitive architectures. The hope is that these concepts may offer
insights for solutions in artificial general intelligence.Comment: 143 pages, 49 figures, 244 reference
Self Organisation and Hierarchical Concept Representation in Networks of Spiking Neurons
The aim of this work is to introduce modular processing mechanisms for cortical functions implemented
in networks of spiking neurons. Neural maps are a feature of cortical processing
found to be generic throughout sensory cortical areas, and self-organisation to the fundamental
properties of input spike trains has been shown to be an important property of cortical organisation.
Additionally, oscillatory behaviour, temporal coding of information, and learning through
spike timing dependent plasticity are all frequently observed in the cortex. The traditional
self-organising map (SOM) algorithm attempts to capture the computational properties of this
cortical self-organisation in a neural network. As such, a cognitive module for a spiking SOM
using oscillations, phasic coding and STDP has been implemented. This model is capable of
mapping to distributions of input data in a manner consistent with the traditional SOM algorithm,
and of categorising generic input data sets. Higher-level cortical processing areas appear
to feature a hierarchical category structure that is founded on a feature-based object representation.
The spiking SOM model is therefore extended to facilitate input patterns in the form of
sets of binary feature-object relations, such as those seen in the field of formal concept analysis.
It is demonstrated that this extended model is capable of learning to represent the hierarchical
conceptual structure of an input data set using the existing learning scheme. Furthermore,
manipulations of network parameters allow the level of hierarchy used for either learning or
recall to be adjusted, and the network is capable of learning comparable representations when
trained with incomplete input patterns. Together these two modules provide related approaches
to the generation of both topographic mapping and hierarchical representation of input spaces
that can be potentially combined and used as the basis for advanced spiking neuron models of
the learning of complex representations
Information processing in biological complex systems: a view to bacterial and neural complexity
This thesis is a study of information processing of biological complex systems seen from the perspective of dynamical complexity (the degree of statistical independence of a system as a whole with respect to its components due to its causal structure). In particular, we investigate the influence of signaling functions in cell-to-cell communication in bacterial and neural systems. For each case, we determine the spatial and causal dependencies in the system dynamics from an information-theoretic point of view and we relate it with their physiological capabilities. The main research content is presented into three main chapters. First, we study a previous theoretical work on synchronization, multi-stability, and clustering of a population of coupled synthetic genetic oscillators via quorum sensing. We provide an extensive numerical analysis of the spatio-temporal interactions, and determine conditions in which the causal structure of the system leads to high dynamical complexity in terms of associated metrics. Our results indicate that this complexity is maximally receptive at transitions between dynamical regimes, and maximized for transient multi-cluster oscillations associated with chaotic behaviour. Next, we introduce a model of a neuron-astrocyte network with bidirectional coupling using glutamate-induced calcium signaling. This study is focused on the impact of the astrocyte-mediated potentiation on synaptic transmission. Our findings suggest that the information generated by the joint activity of the population of neurons is irreducible to its independent contribution due to the role of astrocytes. We relate these results with the shared information modulated by the spike synchronization imposed by the bidirectional feedback between neurons and astrocytes. It is shown that the dynamical complexity is maximized when there is a balance between the spike correlation and spontaneous spiking activity. Finally, the previous observations on neuron-glial signaling are extended to a large-scale system with community structure. Here we use a multi-scale approach to account for spatiotemporal features of astrocytic signaling coupled with clusters of neurons. We investigate the interplay of astrocytes and spiking-time-dependent-plasticity at local and global scales in the emergence of complexity and neuronal synchronization. We demonstrate the utility of astrocytes and learning in improving the encoding of external stimuli as well as its ability to favour the integration of information at synaptic timescales to exhibit a high intrinsic causal structure at the system level. Our proposed approach and observations point to potential effects of the astrocytes for sustaining more complex information processing in the neural circuitry
On the application of neural networks to symbol systems.
While for many years two alternative approaches to building intelligent systems, symbolic
AI and neural networks, have each demonstrated specific advantages and also revealed
specific weaknesses, in recent years a number of researchers have sought methods of combining
the two into a unified methodology which embodies the benefits of each while attenuating the
disadvantages.
This work sets out to identify the key ideas from each discipline and combine them
into an architecture which would be practically scalable for very large network applications.
The architecture is based on a relational database structure and forms the environment for an
investigation into the necessary properties of a symbol encoding which will permit the singlepresentation
learning of patterns and associations, the development of categories and features
leading to robust generalisation and the seamless integration of a range of memory persistencies
from short to long term.
It is argued that if, as proposed by many proponents of symbolic AI, the symbol encoding
must be causally related to its syntactic meaning, then it must also be mutable as the network
learns and grows, adapting to the growing complexity of the relationships in which it is
instantiated. Furthermore, it is argued that in order to create an efficient and coherent memory
structure, the symbolic encoding itself must have an underlying structure which is not accessible
symbolically; this structure would provide the framework permitting structurally sensitive processes
to act upon symbols without explicit reference to their content. Such a structure must dictate
how new symbols are created during normal operation.
The network implementation proposed is based on K-from-N codes, which are shown
to possess a number of desirable qualities and are well matched to the requirements of the symbol
encoding. Several networks are developed and analysed to exploit these codes, based around
a recurrent version of the non-holographic associati ve memory of Willshaw, et al. The simplest
network is shown to have properties similar to those of a Hopfield network, but the storage capacity
is shown to be greater, though at a cost of lower signal to noise ratio.
Subsequent network additions break each K-from-N pattern into L subsets, each using
D-from-N coding, creating cyclic patterns of period L. This step increases the capacity still further
but at a cost of lower signal to noise ratio. The use of the network in associating pairs of
input patterns with any given output pattern, an architectural requirement, is verified.
The use of complex synaptic junctions is investigated as a means to increase storage
capacity, to address the stability-plasticity dilemma and to implement the hierarchical aspects
of the symbol encoding defined in the architecture. A wide range of options is developed which
allow a number of key global parameters to be traded-off. One scheme is analysed and simulated.
A final section examines some of the elements that need to be added to our current understanding
of neural network-based reasoning systems to make general purpose intelligent systems
possible. It is argued that the sections of this work represent pieces of the whole in this
regard and that their integration will provide a sound basis for making such systems a reality
Musified Togetherness; Co-Singing in Families Living With Dementia
Dette ph.d.-prosjektet har som mål å utforske muligheter for og implikasjoner av lavterskel hverdagssang for mennesker med demens og deres nærmeste. En sentral problemstilling er hvordan mennesker med demens og deres pårørende kan bruke og oppleve sang som en integrert del av kommunikasjon og samhandling i dagliglivet, utenfor en profesjonell eller terapeutisk ramme, basert på deres egne erfaringer med sang gjennom livsløpet.
Prosjektets primære empiriske materiale består av en undersøkende forsknings-intraaksjon inspirert av deltakende aksjonsforskning. Deltakerne var en eldre kvinne med demens og hennes voksne datter. Sammen utforsket vi enkle sangaktiviteter som de kunne integrere i dagliglivet, basert på deres preferanser, interesser og tidligere erfaringer med sang. Teoretisk bygger prosjektet på Karen Barads agensiale realisme samt teorier knyttet til affirmativ filosofi, nevropsykologi og nevrofysiologi. Forskningsspørsmålene dreier seg om (1) ulike aspekter ved relasjonell sang som praksis og erfaring, (2) sangens underliggende prosesser og mekanismer, og (3) begrepsmessige og diskursive implikasjoner. Forskningsprosessen har dermed undersøkt ulike aspekter av sang (i familier som lever med demens) som en materielldiskursiv praksis (Barad, 2007).
Gjennom teoretisk og empirisk utforskning og diffraksjon introduserer jeg flere perspektiver på hva hverdagssang i familier som lever med demens kan være. Avhandlingen bidrar til ny kunnskap ved å utforske og veve sammen eksisterende forskning, ulike teorier og diskurser, forskningsintraaksjonen og autoetnografisk materiale. Slik belyser den affirmative og relasjonelle sider ved hverdagssang for mennesker med demens og deres nærmeste. Videre foreslår avhandlingen «samsang» og ulike former for «musisk samvær» som passende begreper og konsepter –og eksempler på hverdagspraksiser – for å formidle implikasjonene av en slik tilnærming til sang og demens. Disse begrepene fremhever sang som en relasjonell aktivitet – en form for samvær og samhørighet. Gjennom utforskning med diffraktiv analyse i flere lag gir avhandlingen dessuten et metodologisk bidrag til performativ og post-kvalitativ forskning.publishedVersio