Adaptive Networks as a Model for Human Speech Development

Unrestricted English text can be converted to speech through the use of a look up table, or through a parallel feedforward network of deterministic processing units. Here, we reproduce the network structure used in NETtalk. Several experiments are carried out to determine which characteristics of the network are responsible for which learning behavior, and how closely that maps human speech development. The network is also trained with different levels of speech complexity, and with a second language. The results are shown to be highly dependent on statistical characteristics of the input

Neuron as a reward-modulated combinatorial switch and a model of learning behavior

This paper proposes a neuronal circuitry layout and synaptic plasticity principles that allow the (pyramidal) neuron to act as a "combinatorial switch". Namely, the neuron learns to be more prone to generate spikes given those combinations of firing input neurons for which a previous spiking of the neuron had been followed by a positive global reward signal. The reward signal may be mediated by certain modulatory hormones or neurotransmitters, e.g., the dopamine. More generally, a trial-and-error learning paradigm is suggested in which a global reward signal triggers long-term enhancement or weakening of a neuron's spiking response to the preceding neuronal input firing pattern. Thus, rewards provide a feedback pathway that informs neurons whether their spiking was beneficial or detrimental for a particular input combination. The neuron's ability to discern specific combinations of firing input neurons is achieved through a random or predetermined spatial distribution of input synapses on dendrites that creates synaptic clusters that represent various permutations of input neurons. The corresponding dendritic segments, or the enclosed individual spines, are capable of being particularly excited, due to local sigmoidal thresholding involving voltage-gated channel conductances, if the segment's excitatory and absence of inhibitory inputs are temporally coincident. Such nonlinear excitation corresponds to a particular firing combination of input neurons, and it is posited that the excitation strength encodes the combinatorial memory and is regulated by long-term plasticity mechanisms. It is also suggested that the spine calcium influx that may result from the spatiotemporal synaptic input coincidence may cause the spine head actin filaments to undergo mechanical (muscle-like) contraction, with the ensuing cytoskeletal deformation transmitted to the axon initial segment where it may...Comment: Version 5: added computer code in the ancillary files sectio

Learning to read aloud: A neural network approach using sparse distributed memory

An attempt to solve a problem of text-to-phoneme mapping is described which does not appear amenable to solution by use of standard algorithmic procedures. Experiments based on a model of distributed processing are also described. This model (sparse distributed memory (SDM)) can be used in an iterative supervised learning mode to solve the problem. Additional improvements aimed at obtaining better performance are suggested

MIKSI KOGNITIOTIETEELLÄ EI OLE TEORIAA KESKUSPROSESSEISTA?

Aivojen ei-modulaaristen keskusprosessien tietämys on edistynyt hitaasti, silläkaikki tunnetut keskusprosessien mallit ovat johtaneet laskennan kompleksisuudenongelmaan. Yhteistä näille malleille on olettaa ihmisen keskuskognition olevanvain näennäisesti monimutkainen systeemi. Artikkelissa tarkastellaan hypoteesia,jonka mukaan aivot olisivat osittain aidosti monimutkainen järjestelmä. "Aitomonimutkaisuus" määritellään algoritmisen informaatioteorian avulla.Avainsanat: Modulaarisuus, kompleksisuus, keskusprosessit, emergenssi,kompleksisuus, oppiminen, synnynnäisyysKeywords: Modularity, complexity, central processes, emergence, learning, innateness

Theory and applications of artificial neural networks

In this thesis some fundamental theoretical problems about artificial neural networks and their application in communication and control systems are discussed. We consider the convergence properties of the Back-Propagation algorithm which is widely used for training of artificial neural networks, and two stepsize variation techniques are proposed to accelerate convergence. Simulation results demonstrate significant improvement over conventional Back-Propagation algorithms. We also discuss the relationship between generalization performance of artificial neural networks and their structure and representation strategy. It is shown that the structure of the network which represent a priori knowledge of the environment has a strong influence on generalization performance. A Theorem about the number of hidden units and the capacity of self-association MLP (Multi-Layer Perceptron) type network is also given in the thesis. In the application part of the thesis, we discuss the feasibility of using artificial neural networks for nonlinear system identification. Some advantages and disadvantages of this approach are analyzed. The thesis continues with a study of artificial neural networks applied to communication channel equalization and the problem of call access control in broadband ATM (Asynchronous Transfer Mode) communication networks. A final chapter provides overall conclusions and suggestions for further work

The use of artificial neural networks in classifying lung scintigrams

An introduction to nuclear medical imaging and artificial neural networks (ANNs) is first given. Lung scintigrams are classified using ANNs in this study. Initial experiments using raw data are first reported. These networks did not produce suitable outputs, and a data compression method was next employed to present an orthogonal data input set containing the largest amount of information possible. This gave some encouraging results, but was neither sensitive nor accurate enough for clinical use. A set of experiments was performed to give local information on small windows of scintigram images. By this method areas of abnormality could be sent into a subsequent classification network to diagnose the cause of the defect. This automatic method of detecting potential defects did not work, though the networks explored were found to act as smoothing filters and edge detectors. Network design was investigated using genetic algorithms (GAs). The networks evolved had low connectivity but reduced error and faster convergence than fully connected networks. Subsequent simulations showed that randomly partially connected networks performed as well as GA designed ones. Dynamic parameter tuning was explored in an attempt to produce faster convergence, but the previous good results of other workers could not be replicated. Classification of scintigrams using manually delineated regions of interest was explored as inputs to ANNs, both in raw state and as principal components (PCs). Neither representation was shown to be effective on test data

Nature of the learning algorithms for feedforward neural networks

The neural network model (NN) comprised of relatively simple computing elements, operating in parallel, offers an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. Due to the amount of research developed in the area many types of networks have been defined. The one of interest here is the multi-layer perceptron as it is one of the simplest and it is considered a powerful representation tool whose complete potential has not been adequately exploited and whose limitations need yet to be specified in a formal and coherent framework. This dissertation addresses the theory of generalisation performance and architecture selection for the multi-layer perceptron; a subsidiary aim is to compare and integrate this model with existing data analysis techniques and exploit its potential by combining it with certain constructs from computational geometry creating a reliable, coherent network design process which conforms to the characteristics of a generative learning algorithm, ie. one including mechanisms for manipulating the connections and/or units that comprise the architecture in addition to the procedure for updating the weights of the connections. This means that it is unnecessary to provide an initial network as input to the complete training process.After discussing in general terms the motivation for this study, the multi-layer perceptron model is introduced and reviewed, along with the relevant supervised training algorithm, ie. backpropagation. More particularly, it is argued that a network developed employing this model can in general be trained and designed in a much better way by extracting more information about the domains of interest through the application of certain geometric constructs in a preprocessing stage, specifically by generating the Voronoi Diagram and Delaunav Triangulation [Okabe et al. 92] of the set of points comprising the training set and once a final architecture which performs appropriately on it has been obtained, Principal Component Analysis [Jolliffe 86] is applied to the outputs produced by the units in the network's hidden layer to eliminate the redundant dimensions of this space