Processing resources and interplay among sensory modalities: an EEG investigation

The primary aim of the present thesis was to investigate how the human brain handles and distributes limited processing resources among different sensory modalities. Two main hypothesis have been conventionally proposed: (1) common processing resources shared among sensory modalities (supra-modal attentional system) or (2) independent processing resources for each sensory modality. By means of four EEG experiments, we tested whether putative competitive interactions between sensory modalities – regardless of attentional influences – are present in early sensory areas. We observed no competitive interactions between sensory modalities, supporting independent processing resources in early sensory areas. Consequently, we tested the influence of top-down attention on a cross-modal dual task. We found evidence for shared attentional resources between visual and tactile modalities. Taken together, our results point toward a hybrid model of inter-modal attention. Attentional processing resources seem to be controlled by a supra-modal attentional system, however, in early sensory areas, the absence of competitive interactions strongly reduces interferences between sensory modalities, thus providing a strong processing resource independence

Audio-visual synchrony and feature-selective attention co-amplify early visual processing

Our brain relies on neural mechanisms of selective attention and converging sensory processing to efficiently cope with rich and unceasing multisensory inputs. One prominent assumption holds that audio-visual synchrony can act as a strong attractor for spatial attention. Here, we tested for a similar effect of audio-visual synchrony on feature-selective attention. We presented two superimposed Gabor patches that differed in colour and orientation. On each trial, participants were cued to selectively attend to one of the two patches. Over time, spatial frequencies of both patches varied sinusoidally at distinct rates (3.14 and 3.63 Hz), giving rise to pulse-like percepts. A simultaneously presented pure tone carried a frequency modulation at the pulse rate of one of the two visual stimuli to introduce audio-visual synchrony. Pulsed stimulation elicited distinct time-locked oscillatory electrophysiological brain responses. These steady-state responses were quantified in the spectral domain to examine individual stimulus processing under conditions of synchronous versus asynchronous tone presentation and when respective stimuli were attended versus unattended. We found that both, attending to the colour of a stimulus and its synchrony with the tone, enhanced its processing. Moreover, both gain effects combined linearly for attended in-sync stimuli. Our results suggest that audio-visual synchrony can attract attention to specific stimulus features when stimuli overlap in space

The workload implications of haptic displays in multi-display environments such as the cockpit: Dual-task interference of within-sense haptic inputs (tactile/proprioceptive) and between-sense inputs (tactile/proprioceptive/auditory/visual)

Visual workload demand within the cockpit is reaching saturation, whereas the haptic sense (proprioceptive and tactile sensation) is relatively untapped, despite studies suggesting the benefits of haptic displays. MRT suggests that inputs from haptic displays will not interfere with inputs from visual or auditory displays. MRT is based on the premise that multisensory integration occurs only after unisensory processing. However, recent neuroscientific findings suggest that the distinction between unisensory versus multisensory processing is much more blurred than previously thought. This programme of work had the following two research objectives: 1. To examine whether multiple haptic inputs can be processed at the same time without performance decrement - Study One 2. To examine whether haptic inputs can be processed at the same time as visual or auditory inputs without performance decrement - Study Two In Study One participants performed dual-tasks, consisting of same-sense tasks (tactile or proprioceptive) or different-sense tasks (tactile and proprioceptive). These tasks also varied in terms of processing code, in line with MRT. The results found significantly more performance decrement for the same-sense dual-tasks than for the different-sense dual-tasks, in accordance with MRT, suggesting that performance will suffer if two haptic displays of the same type are used concurrently. An adjustment to the MRT model is suggested to incorporate these results. In Study Two, participants performed different-sense dual-tasks, consisting of auditory or visual tasks with tactile or proprioceptive tasks. The tasks also varied in terms of processing code. Contrary to MRT, the results found that when processing code was different, there was significant performance decrement for all of the dual-tasks, but not when processing code was the same. These results reveal an exception to two key MRT rules, the sensory resource rule and the processing code rule. It is suggested that MRT may be oversimplistic and other factors highlighted by recent neuroscientific research should be taken into account in theories of dual-task performance

Perceptual Strategies and Neuronal Underpinnings underlying Pattern Recognition through Visual and Tactile Sensory Modalities in Rats

The aim of my PhD project was to investigate multisensory perception and multimodal recognition abilities in the rat, to better understand the underlying perceptual strategies and neuronal mechanisms. I have chosen to carry out this project on the laboratory rat, for two reasons. First, the rat is a flexible and highly accessible experimental model, where it is possible to combine state-of-the-art neurophysiological approaches (such as multi-electrode neuronal recordings) with behavioral investigation of perception and (more in general) cognition. Second, extensive research concerning multimodal integration has already been conducted in this species, both at the neurophysiological and behavioral level. My thesis work has been organized in two projects: a psychophysical assessment of object categorization abilities in rats, and a neurophysiological study of neuronal tuning in the primary visual cortex of anaesthetized rats. In both experiments, unisensory (visual and tactile) and multisensory (visuo-tactile) stimulation has been used for training and testing, depending on the task. The first project has required development of a new experimental rig for the study of object categorization in rat, using solid objects, so as to be able to assess their recognition abilities under different modalities: vision, touch and both together. The second project involved an electrophysiological study of rat primary visual cortex, during visual, tactile and visuo-tactile stimulation, with the aim of understanding whether any interaction between these modalities exists, in an area that is mainly deputed to one of them. The results of both of the studies are still preliminary, but they already offer some interesting insights on the defining features of these abilities

The neurophysiology of intersensory selective attention and task switching

Our ability to selectively attend to certain aspects of the world and ignore others is fundamental to our day-to-day lives. The need for selective attention stems from capacity limitations inherent in our perceptual and cognitive processing architecture. Because not every elemental piece of our environment can be fully processed in parallel, the nervous system must prioritize processing. This prioritization is generally referred to as selective attention. Meanwhile, we are faced with a world that is constantly in flux, such that we have to frequently shift our attention from one piece of the environment to another and from one task to another. This process is generally referred to as task-switching. Neural oscillations in the alpha band (~8-14 Hz) have been shown to index the distribution of selective attention, and there is increasing evidence that oscillations in this band are in fact utilized by the nervous system to suppress distracting, task-irrelevant information. In order to elaborate on what is known of the function of alpha oscillations as well as current models of both intersensory selective attention and task switching, I investigated the dynamics of alpha amplitude modulations within the context of intersenory selective attention and task switching in neurologically typical young adults. Participants were alternately cued to attend to either the visual or auditory aspect of a compound audio-visual stimulus while high-density electroencephalography was recorded. It is typically found that alpha power increases over parieto-occipital cortices when attention is directed away from the visual modality and to the auditory modality. I report evidence that alpha oscillations play a role in task-switching (e.g., when switching from attending the visual task versus repeating this task), specifically as biasing signals, that may operate to re-weight competition among two tasks-sets. I further investigated the development of these same processes in school-aged children and adolescents. While exhibiting typical patterns of alpha modulations relevant to selective attention, Young school-aged children (8-12 years), compared to older participants, did not demonstrate specific task switching modulation of alpha oscillations, suggesting that this process does not fully develop until late adolescence. Finally, children and adolescents on the autism spectrum failed altogether to exhibit differentiation of alpha power between attend-visual and attend-auditory conditions--an effect present in age and IQ matched controls--suggesting that ASD individuals may have a deficit in the overall top-down deployment of alpha oscillatory biasing signals. This could result in an inability to ignore distracting information in the environment, leading to an overwhelming, disordered experience of the world, resulting in profound effects on the both social interaction and cognitive development. Altogether, these findings add to growing evidence that alpha oscillations serve as domain general biasing signals and are integral to our flexible goal-oriented behavior. Furthermore, the flexible use of these biasing signals in selective attention and task switching develops over a protracted period, and appears to be aberrant in autism spectrum disorder

Perception and Its Implication for the “Perceptually Handicapped Child” with Emphasis on Auditory Modality

It was the purpose of this study to present a review of the modern literature on perception, with an emphasis on the auditory modality, in an effort to summarize what research says concerning: 1. the neurological makeup of the perceiving organism; 2. the linguistic makeup of the perceiving organism; 3. the nature of innate perceptual capacities and of acquired percepts; 4. the nature of perceptual deficits and the importance of early identification; 5. the interrelationship of feedback and perception; 6. the interrelationship of auditory discrimination to perception; 7. the dynamics of system functions as it applies to perceptual choices; and 8. the feasibility of behavior modification as a remediation technique for perception deficits

More than human aesthetics: interactive enrichment for elephants

Species-specific aesthetics is an important consideration for interaction designers working with animals. The paper explores the concept of species-specific aesthetics with particular reference to elephants. Applying existing aesthetic dimensions and design principles to the challenge of designing interactive enrichment for them, we show how the insights gained can inform more than human centered design in different settings. We offer a multi-faceted, multi-sensory lens for examining an animal-centred aesthetic experience of technology