Sensory integration : success & failure

The convergence of sensory signals plays an important role in perception. Two cases of convergence are examined in this thesis by means of modelling; the case when the converging signals are congruent, which leads to combination and an enhancement of perception, and the case when they are incongruent.The former modelling experiment considers the integration of phonemes and letters. Based on a series of reports on different aspects of letter-phoneme integration that has been presented from the Department of Cognitive Neuroscience at Maastricht University in the Netherlands we have developed a model for simulating some features of this course of events. Our model, the Artificial Cortical Network (ACN), is an artificial neural network whose essential parts are two types of modules, each employing its own learning law, none requiring manual intervention. The particular ACN architecture used in this thesis consists of three modules which are interconnected, where each module processes one of three different types of pre-processed stimuli: letters, phonemes and the bimodal combination of these. Modules of this architecture contain one or more neural lattices and inter-module information exchange is carried out using the coordinates of the neurons that respond the strongest to the inputs. The model is generally useful; it is, for example, equally suitable for modelling the integration of different features in a single modality, and the ideas behind it are new. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal module and feedback from the bimodal to the auditory module. Simulation results of the architecture show the same characteristics as corresponding results from psychology and neuroscience. One important result is the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality. This is what happens at the opera when the libretto is shown above the scene: one better perceives what is sung. Another is the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality. This mechanism is most probably active when having a conversation at a cocktail, on the bus or in the subway. In these situations the (possibly unconscious) monitoring of the communicating peer's mouth movements may improve comprehension. The latter modelling experiment considers a case when integration "fails" due to irreconcilable incoming signals. The particular phenomenon under examination is that of binocular rivalry: the alternating periods of dominance and suppression occasioned by stimulation of corresponding retinal areas with dissimilar monocular stimuli. Typically an observer only "sees" one stimulus at a time and the rate of recurrence is about 1-2 seconds on average. The switching that occurs has some well established properties; such as a skewed unimodal distribution of dominance times and an increase of switching speed with stimulus contrast. In this thesis an artificial neural network model of a cortical area in which competition between populations of neurons that should not be co-active is presented. The focus of the model is for it to be able to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry. These properties are Levelt's second and fourth proposition and the "flipped" case of Levelt's second proposition. The essential driving forces in this architecture are fatigue and noise. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using the same model parameters.Godkänd; 2009; 20091012 (tamjan); LICENTIATSEMINARIUM Ämnesområde: Industriell elektronik/Industrial Electronics Examinator: Professor Jerker Delsing, Luleå tekniska universitet Tid: Måndag den 9 november 2009 kl 11.00 Plats: Demostudion, Luleå tekniska universitet</p

Phenomenological modelling of sensory integration phenomena using self-organized feature maps

The main contribution of the work presented herein is a method of transferring information from one lattice of artificial neural units to another. The essence of the present method consists of four parts: (1) the wrapping of the artificial neural lattice into a module that given an input to this lattice outputs normalized coordinates for the location of peak activity and the magnitude of the activity at that location; (2) the inter-module exchange of information being carried out by using the coordinates of the neuron that respond the strongest to the input given to the artificial neural lattice; (3) the concept of the transformation map which re-maps normalized coordinates from one module's output space on another's; and (4) fusion carried out by summation of activity fields induced by artificial neural lattices. The method, that is to been seen as an alternative to Hebbian linkage, is validated by implementation in the artificial neural network architecture used for carrying out the two cortical modelling experiments described below.It is the modelling of sensory integration phenomena that is the main theme in this thesis. The two modelling experiments used for validating the presented method examine two cases of sensory convergence; the case when the converging signals are congruent and the case when they are incongruent. Convergence of sensory signals is interesting because it is a process taking place in the brain, and because it plays an important role in perception.The sensory convergence phenomenon of main interest is that of audiovisual integration. Based on a series of reports on different aspects of letter–phoneme integration an architecture is developed, the multimodal self-organizing network (MMSON). The result is an artificial neural network whose essential building blocks are two types of modules, each employing its own learning law, none requiring manual intervention. The built MMSON architecture consists of three modules which are interconnected, and each module processes one of three different types of pre-processed stimuli: visual, auditory, and the bimodal combination of these. The architecture is generally useful; it is, for example, equally suitable for simulating the integration of different features in a single modality, and the ideas behind it are new.In the first modelling experiment the focus is on integration of congruent stimuli, leading to combination and an enhancement of perception. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal level and feedback from the bimodal to the auditory level. Simulation results show that the architecture's dynamics parallel results from psychology and neuroscience: (1) the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality; and (2) the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality.The second modelling experiment considers a case when integration ``fails'' due to irreconcilable incoming signals. The main phenomena under examination is the McGurk effect. This effect occurs during the perception of audiovisual syllables when particular incongruent stimulus pairs are presented to the subject. In these particular cases the conclusion of the subject's perceptual system contains neither of the presented stimuli but instead a third stimulus is perceived; e.g. an auditory ``ba'' and a visual ``ga'' is often perceived as ``da''. We show here that this phenomenon can be successfully simulated with the MMSON architecture.Another phenomenon where integration ``fails'', which is examined in this thesis, is binocular rivalry. Although this work is not in complete resonance with with the work on audiovisual integration, the architecture used for modelling binocular rivalry is related to the MMSON architecture by sharing the same foundation; it consists of communicating lattices containing artificial neurons. The work on binocular rivalry has two foci. One is to have the architecture's dynamics to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry that are described by Levelt's second and fourth propositions and the ``reverse of'' Levelt's second proposition. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using one and the same set of model parameters. The other focus is to strengthen the links between binocular rivalry and sensory integration by arguing that the former phenomenon can be seen as a variant of the latter, and by indicating how the dynamics of the two coincide.Godkänd; 2011; 20110704 (tamjan); DISPUTATION Ämnesområde: Industriell elektronik/Industrial Electronics Opponent: Professor Wlodzislaw Duch, Dept of Informatics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University, Torun, Poland Ordförande: Professor Lennart Gustafsson, Avd för EISLAB, Institutionen för system- och rymdteknik, Luleå tekniska universitet Tid: Måndag den 19 december 2011, kl 10.00 Plats: D770, Luleå tekniska universite

Energy efficient routing in wireless networks

CMAX and max-min zPmin are two packet forwarding techniques designed to save energy wireless ad-hoc networks. Their aim is to forward packets in such a way that the network's energy is conserved better than if they were forwarded along the path with the least count of hops (i e using the traditional forwarding technique). This master thesis compares the two energy efficient forwarding techniques against each other and, against a least-hop scheme. The comparison is made by simulation in static networks using the Network Simulator (ns-2). Hence this thesis also shows that simulation of more abstract “routing protocols”, such as these bare packet forwarding techniques, can be performed in ns-2. CMAX and max-min zPmin are thus implemented as routing modules of ns-2, along with a least-hop technique. The work here is only concerned about the packet forwarding procedure itself, and because of this the implementations are somewhat centralized, or omnipotent: Each node is allowed to know the exact position of every other node in the network at all times. For disseminating battery updates limited-flooding is employed, a method which uses the idea of only propagating information a certain distance from the information source. ns-2 is also extended with all the necessary features that are needed by the algorithms, such as mechanisms to enable nodes gather global information from existing data structures, support for variable signaling range and a signaling range dependent energy consumption model. The work is described in detail and decisions are backed up with references to aid the understanding of it and make it easier to use, both as a product and a foundation. The simulations here only examine static networks where the nodes never move from their initial position. In this setting, using a “fair random traffic model”, they indicate that the energy efficient algorithms perform better than the shortest path scheme when the network is sparse and, that CMAX performs better than max-min zPmin. Performance is measured by both looking at network lifetime and how much user data the schemes manage to successfully carry from source to sink. This booklet contains a summary of other interesting approaches that aim to conserve energy in a wireless network, such as different methods to perform topology control and hierarchical routing. There is also a proposition of a new cluster based energy efficient algorithm that is based on ideas collected from some of the summarized schemes.Validerat; 20101217 (root

Sensory integration : success &amp; failure

The convergence of sensory signals plays an important role in perception. Two cases of convergence are examined in this thesis by means of modelling; the case when the converging signals are congruent, which leads to combination and an enhancement of perception, and the case when they are incongruent.The former modelling experiment considers the integration of phonemes and letters. Based on a series of reports on different aspects of letter-phoneme integration that has been presented from the Department of Cognitive Neuroscience at Maastricht University in the Netherlands we have developed a model for simulating some features of this course of events. Our model, the Artificial Cortical Network (ACN), is an artificial neural network whose essential parts are two types of modules, each employing its own learning law, none requiring manual intervention. The particular ACN architecture used in this thesis consists of three modules which are interconnected, where each module processes one of three different types of pre-processed stimuli: letters, phonemes and the bimodal combination of these. Modules of this architecture contain one or more neural lattices and inter-module information exchange is carried out using the coordinates of the neurons that respond the strongest to the inputs. The model is generally useful; it is, for example, equally suitable for modelling the integration of different features in a single modality, and the ideas behind it are new. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal module and feedback from the bimodal to the auditory module. Simulation results of the architecture show the same characteristics as corresponding results from psychology and neuroscience. One important result is the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality. This is what happens at the opera when the libretto is shown above the scene: one better perceives what is sung. Another is the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality. This mechanism is most probably active when having a conversation at a cocktail, on the bus or in the subway. In these situations the (possibly unconscious) monitoring of the communicating peer's mouth movements may improve comprehension. The latter modelling experiment considers a case when integration "fails" due to irreconcilable incoming signals. The particular phenomenon under examination is that of binocular rivalry: the alternating periods of dominance and suppression occasioned by stimulation of corresponding retinal areas with dissimilar monocular stimuli. Typically an observer only "sees" one stimulus at a time and the rate of recurrence is about 1-2 seconds on average. The switching that occurs has some well established properties; such as a skewed unimodal distribution of dominance times and an increase of switching speed with stimulus contrast. In this thesis an artificial neural network model of a cortical area in which competition between populations of neurons that should not be co-active is presented. The focus of the model is for it to be able to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry. These properties are Levelt's second and fourth proposition and the "flipped" case of Levelt's second proposition. The essential driving forces in this architecture are fatigue and noise. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using the same model parameters.Godkänd; 2009; 20091012 (tamjan); LICENTIATSEMINARIUM Ämnesområde: Industriell elektronik/Industrial Electronics Examinator: Professor Jerker Delsing, Luleå tekniska universitet Tid: Måndag den 9 november 2009 kl 11.00 Plats: Demostudion, Luleå tekniska universitet</p

Phenomenological modelling of sensory integration phenomena using self-organized feature maps

The main contribution of the work presented herein is a method of transferring information from one lattice of artificial neural units to another. The essence of the present method consists of four parts: (1) the wrapping of the artificial neural lattice into a module that given an input to this lattice outputs normalized coordinates for the location of peak activity and the magnitude of the activity at that location; (2) the inter-module exchange of information being carried out by using the coordinates of the neuron that respond the strongest to the input given to the artificial neural lattice; (3) the concept of the transformation map which re-maps normalized coordinates from one module's output space on another's; and (4) fusion carried out by summation of activity fields induced by artificial neural lattices. The method is validated by implementation in the artificial neural network architecture used for carrying out the two cortical modelling experiments described below. It is the modelling of sensory integration phenomena that is the main theme in this thesis. The two modelling experiments used for validating the presented method examine two cases of sensory convergence; the case when the converging signals are congruent and the case when they are incongruent. Convergence of sensory signals is interesting because it is a process taking place in the brain, and because it plays an important role in perception. The sensory convergence phenomenon of main interest is that of audiovisual integration. Based on a series of reports on different aspects of letter-phoneme integration an architecture is developed, the multimodal self-organizing network (MMSON). The result is an artificial neural network whose essential building blocks are two types of modules, each employing its own learning law, none requiring manual intervention. The built MMSON architecture consists of three modules which are interconnected, and each module processes one of three different types of pre-processed stimuli: visual, auditory, and the bimodal combination of these. The architecture is generally useful; it is, for example, equally suitable for simulating the integration of different features in a single modality, and the ideas behind it are new. In the first modelling experiment the focus is on integration of congruent stimuli, leading to combination and an enhancement of perception. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal level and feedback from the bimodal to the auditory level. Simulation results show that the architecture's dynamics parallel results from psychology and neuroscience: (1) the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality; and (2) the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality. The second modelling experiment considers a case when integration "fails" due to irreconcilable incoming signals. The main phenomena under examination is the McGurk effect. This effect occurs during the perception of audiovisual syllables when particular incongruent stimulus pairs are presented to the subject. In these particular cases the conclusion of the subject's perceptual system contains neither of the presented stimuli but instead a third stimulus is perceived; e.g. an auditory "ba" and a visual "ga" is often perceived as "da". We show here that this phenomenon can be successfully simulated with the MMSON architecture. Another phenomenon where integration "fails", which is examined in this thesis, is binocular rivalry. Although this work is not in complete resonance with the work on audiovisual integration, the architecture used for modelling binocular rivalry is related to the MMSON architecture by sharing the same foundation; it consists of communicating lattices containing artificial neurons. The work on binocular rivalry has two foci. One is to have the architecture's dynamics to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry that are described by Levelt's second and fourth propositions and the "reverse of" Levelt's second proposition. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using one and the same set of model parameters. The other focus is to strengthen the links between binocular rivalry and sensory integration by arguing that the former phenomenon can be seen as a variant of the latter, and by indicating how the dynamics of the two coincide.Submitted in partial fulfillment of the requirements for the Doctor of Philosophy (Dual Award) (Luleå University of Technology)

Phenomenological modelling of sensory integration phenomena using self-organized feature maps

The main contribution of the work presented herein is a method of transferring information from one lattice of artificial neural units to another. The essence of the present method consists of four parts: (1) the wrapping of the artificial neural lattice into a module that given an input to this lattice outputs normalized coordinates for the location of peak activity and the magnitude of the activity at that location; (2) the inter-module exchange of information being carried out by using the coordinates of the neuron that respond the strongest to the input given to the artificial neural lattice; (3) the concept of the transformation map which re-maps normalized coordinates from one module's output space on another's; and (4) fusion carried out by summation of activity fields induced by artificial neural lattices. The method is validated by implementation in the artificial neural network architecture used for carrying out the two cortical modelling experiments described below. It is the modelling of sensory integration phenomena that is the main theme in this thesis. The two modelling experiments used for validating the presented method examine two cases of sensory convergence; the case when the converging signals are congruent and the case when they are incongruent. Convergence of sensory signals is interesting because it is a process taking place in the brain, and because it plays an important role in perception. The sensory convergence phenomenon of main interest is that of audiovisual integration. Based on a series of reports on different aspects of letter-phoneme integration an architecture is developed, the multimodal self-organizing network (MMSON). The result is an artificial neural network whose essential building blocks are two types of modules, each employing its own learning law, none requiring manual intervention. The built MMSON architecture consists of three modules which are interconnected, and each module processes one of three different types of pre-processed stimuli: visual, auditory, and the bimodal combination of these. The architecture is generally useful; it is, for example, equally suitable for simulating the integration of different features in a single modality, and the ideas behind it are new. In the first modelling experiment the focus is on integration of congruent stimuli, leading to combination and an enhancement of perception. The main features modelled using the architecture, are the combination of unimodal signals at the bimodal level and feedback from the bimodal to the auditory level. Simulation results show that the architecture's dynamics parallel results from psychology and neuroscience: (1) the qualitative enhancement of the response to a noise-perturbed sensory signal in one modality using a congruent one from a complementary modality; and (2) the ability to use input to one modality as support for distinguishing the relevant signal from a collection of signals in another modality. The second modelling experiment considers a case when integration "fails" due to irreconcilable incoming signals. The main phenomena under examination is the McGurk effect. This effect occurs during the perception of audiovisual syllables when particular incongruent stimulus pairs are presented to the subject. In these particular cases the conclusion of the subject's perceptual system contains neither of the presented stimuli but instead a third stimulus is perceived; e.g. an auditory "ba" and a visual "ga" is often perceived as "da". We show here that this phenomenon can be successfully simulated with the MMSON architecture. Another phenomenon where integration "fails", which is examined in this thesis, is binocular rivalry. Although this work is not in complete resonance with the work on audiovisual integration, the architecture used for modelling binocular rivalry is related to the MMSON architecture by sharing the same foundation; it consists of communicating lattices containing artificial neurons. The work on binocular rivalry has two foci. One is to have the architecture's dynamics to simulate key properties of binocular rivalry; the skewed unimodal distribution of dominance times, and three other well-known properties of binocular rivalry that are described by Levelt's second and fourth propositions and the "reverse of" Levelt's second proposition. Simulation results obtained from the model are in fair quantitative agreement with psychophysical experiments in all four aspects, using one and the same set of model parameters. The other focus is to strengthen the links between binocular rivalry and sensory integration by arguing that the former phenomenon can be seen as a variant of the latter, and by indicating how the dynamics of the two coincide. Submitted in partial fulfillment of the requirements for the Doctor of Philosophy (Dual Award) (Luleå University of Technology)

A self-organized artificial neural network architecture for sensory integration with applications to letter-phoneme integration

The multimodal self-organizing network (MMSON), an artificial neural network architecture carrying out sensory integration, is presented here. The architecture is designed using neurophysiological findings and imaging studies that pertain to sensory integration and consists of interconnected lattices of artificial neurons. In this artificial neural architecture, the degree of recognition of stimuli, that is, the perceived reliability of stimuli in the various subnetworks, is included in the computation. The MMSON's behavior is compared to aspects of brain function that deal with sensory integration. According to human behavioral studies, integration of signals from sensory receptors of different modalities enhances perception of objects and events and also reduces time to detection. In neocortex, integration takes place in bimodal and multimodal association areas and result, not only in feedback-mediated enhanced unimodal perception and shortened reaction time, but also in robust bimodal or multimodal percepts. Simulation data from the presented artificial neural network architecture show that it replicates these important psychological and neuroscientific characteristics of sensory integration.Validerad; 2011; 20110427 (tamjan