Multimodal feedforward self-organizing maps

We introduce a novel system of interconnected Self- Organizing Maps that can be used to build feedforward and recurrent networks of maps. Prime application of interconnected maps is in modelling systems that operate with multimodal data as for example in visual and auditory cortices and multimodal association areas in cortex. A detailed example of animal categorization in which the feedworward network of self-organizing maps is employed is presented. In the example we operate with 18-dimensional data projected up on the 19-dimensional hyper-sphere so that the “dot-product” learning law can be used. One potential benefit of the multimodal map is that it allows a rich structure of parallel unimodal processing with many maps involved, followed by convergence into multimodal maps. More complex stimuli can therefore be processed without a growing map size.Validerad; 2005; 20100325 (andbra

Feedback in multimodal self-organizing networks enhances perception of corrupted stimuli

It is known from psychology and neuroscience that multimodal integration of sensory information enhances the perception of stimuli that are corrupted in one or more modalities. A prominent example of this is that auditory perception of speech is enhanced when speech is bimodal, i.e. when it also has a visual modality. The function of the cortical network processing speech in auditory and visual cortices and in multimodal association areas, is modeled with a Multimodal Self-Organizing Network (MuSON), consisting of several Kohonen Self-Organizing Maps (SOM) with both feedforward and feedback connections. Simulations with heavily corrupted phonemes and uncorrupted letters as inputs to the MuSON demonstrate a strongly enhanced auditory perception. This is explained by feedback from the bimodal area into the auditory stream, as in cortical processing.Validerad; 2006; Bibliografisk uppgift: Lecture Notes in Artificial Intelligence; 20061228 (ysko