Distortions of Subjective Time Perception Within and Across Senses

Background: The ability to estimate the passage of time is of fundamental importance for perceptual and cognitive processes. One experience of time is the perception of duration, which is not isomorphic to physical duration and can be distorted by a number of factors. Yet, the critical features generating these perceptual shifts in subjective duration are not understood. Methodology/Findings: We used prospective duration judgments within and across sensory modalities to examine the effect of stimulus predictability and feature change on the perception of duration. First, we found robust distortions of perceived duration in auditory, visual and auditory-visual presentations despite the predictability of the feature changes in the stimuli. For example, a looming disc embedded in a series of steady discs led to time dilation, whereas a steady disc embedded in a series of looming discs led to time compression. Second, we addressed whether visual (auditory) inputs could alter the perception of duration of auditory (visual) inputs. When participants were presented with incongruent audio-visual stimuli, the perceived duration of auditory events could be shortened or lengthened by the presence of conflicting visual information; however, the perceived duration of visual events was seldom distorted by the presence of auditory information and was never perceived shorter than their actual durations. Conclusions/Significance: These results support the existence of multisensory interactions in the perception of duration and, importantly, suggest that vision can modify auditory temporal perception in a pure timing task. Insofar as distortions in subjective duration can neither be accounted for by the unpredictability of an auditory, visual or auditory-visual event, we propose that it is the intrinsic features of the stimulus that critically affect subjective time distortions

Processing resources and interplay among sensory modalities: an EEG investigation

The primary aim of the present thesis was to investigate how the human brain handles and distributes limited processing resources among different sensory modalities. Two main hypothesis have been conventionally proposed: (1) common processing resources shared among sensory modalities (supra-modal attentional system) or (2) independent processing resources for each sensory modality. By means of four EEG experiments, we tested whether putative competitive interactions between sensory modalities – regardless of attentional influences – are present in early sensory areas. We observed no competitive interactions between sensory modalities, supporting independent processing resources in early sensory areas. Consequently, we tested the influence of top-down attention on a cross-modal dual task. We found evidence for shared attentional resources between visual and tactile modalities. Taken together, our results point toward a hybrid model of inter-modal attention. Attentional processing resources seem to be controlled by a supra-modal attentional system, however, in early sensory areas, the absence of competitive interactions strongly reduces interferences between sensory modalities, thus providing a strong processing resource independence

Amodal Atypical Neural Oscillatory Activity in Dyslexia: A Cross-Linguistic Perspective

First Published December 21, 2016It has been proposed that atypical neural oscillations in both the auditory and the visual modalities could explain why some individuals fail to learn to read and suffer from developmental dyslexia. However, the role of specific oscillatory mechanisms in reading acquisition is still under debate. In this article, we take a cross-linguistic approach and argue that both the phonological and orthographic specifics of a language (e.g., linguistic rhythm, orthographic depth) shape the oscillatory activity thought to contribute to reading development. The proposed theoretical framework should allow future research to test cross-linguistic hypotheses that will shed light on the heterogeneity of auditory and visual disorders and their underlying brain dysfunction(s) in developmental dyslexia, and inform clinical practice by helping us to diagnose dyslexia across languages.This research was funded by the European Research Council (ERC Advanced Grant, BILITERACY Project, to M.C.), and the Spanish government (Plan Nacional-PSI2012-32128 and PSI2015-65338-P to M.L., Plan Nacional-PSI2012-32350 and PSI2015-65694-P to N.M., and Plan Nacional-PSI2015-67353-R to M.C.). The Basque Center on Brain Cognition and Language acknowledges funding from Ayuda Centro de Excelencia Severo Ochoa SEV-2015-0490

The impact of reward value on early sensory processing and its interaction with selective attention

Reward value affects the earliest stages of sensory perception. Whereas a host of previous studies have investigated the underlying mechanisms of reward-driven modulation of visual perception, reward effects in other sensory modalities have remained underexplored. Specifically, it has remained unknown how reward signals should be coordinated and communicated across sensory modalities. The current PhD thesis aimed to gain insight into the underlying mechanisms of reward-driven modulation of perception and its interaction with attention across sensory modalities. To this end, three experiments were conducted to identify the behavioral and electrophysiological correlates of reward effects. In Study 1, we found that high reward, task-irrelevant visual cues (intra-modal) elicited an early suppression of visual event-related potentials (ERPs). High reward auditory cues (cross-modal), on the other hand, led to a later modulation of visual ERPs and facilitated behavioral performance. Study 2 tested the dependence of reward effects on the spatial and temporal arrangement of intra-modal and cross-modal cues relative to the target, and showed that each reward cue maximally exerts its effect under a specific size of attentional focus. Study 3 explicitly manipulated the spatial attention and tested how reward associations of an audiovisual stimulus influence the allocation of attention. We found that auditory rewards enhanced the attentional modulation of both visual and auditory ERPs. Interestingly, although visual rewards did not lead to a distinguishable ERP modulation, they led to strong modulations when they were combined with auditory rewards, suggesting that integration across modalities boosts the reward effects. Taken together, the current PhD thesis identified the behavioral and neural signatures of reward-driven modulation of perception under different modes of reward signaling and different degrees of attentional engagement. Our findings inspire a two-stage model of reward processing, with local, intra-modal reward effects occurring at an early stage and long-range, multimodal reward effects arising at a later stage. Cross-modal reward signals have important ramifications for clinical applications where the impaired function of one sense can be rehabilitated by motivational signals conveyed through another sensory modality.2021-11-2

Nägemistaju automaatsete protsesside eksperimentaalne uurimine

Väitekirja elektrooniline versioon ei sisalda publikatsiooneVäitekiri keskendub nägemistaju protsesside eksperimentaalsele uurimisele, mis on suuremal või vähemal määral automaatsed. Uurimistöös on kasutatud erinevaid eksperimentaalseid katseparadigmasid ja katsestiimuleid ning nii käitumuslikke- kui ka ajukuvamismeetodeid. Esimesed kolm empiirilist uurimust käsitlevad liikumisinformatsiooni töötlust, mis on evolutsiooni käigus kujunenud üheks olulisemaks baasprotsessiks nägemistajus. Esmalt huvitas meid, kuidas avastatakse liikuva objekti suunamuutusi, kui samal ajal toimub ka taustal liikumine (Uurimus I). Nägemistaju uurijad on pikka aega arvanud, et liikumist arvutatakse alati mõne välise objekti või tausta suhtes. Meie uurimistulemused ei kinnitanud taolise suhtelise liikumise printsiibi paikapidavust ning toetavad pigem seisukohta, et eesmärkobjekti liikumisinformatsiooni töötlus on automaatne protsess, mis tuvastab silma põhjas toimuvaid nihkeid, ja taustal toimuv seda eriti ei mõjuta. Teise uurimuse tulemused (Uurimus II) näitasid, et nägemissüsteem töötleb väga edukalt ka seda liikumisinformatsiooni, millele vaatleja teadlikult tähelepanu ei pööra. See tähendab, et samal ajal, kui inimene on mõne tähelepanu hõlmava tegevusega ametis, suudab tema aju taustal toimuvaid sündmusi automaatselt registreerida. Igapäevaselt on inimese nägemisväljas alati palju erinevaid objekte, millel on erinevad omadused, mistõttu järgmiseks huvitas meid (Uurimus III), kuidas ühe tunnuse (antud juhul värvimuutuse) töötlemist mõjutab mõne teise tunnusega toimuv (antud juhul liikumiskiiruse) muutus. Näitasime, et objekti liikumine parandas sama objekti värvimuutuse avastamist, mis viitab, et nende kahe omaduse töötlemine ajus ei ole päris eraldiseisev protsess. Samuti tähendab taoline tulemus, et hoolimata ühele tunnusele keskendumisest ei suuda inimene ignoreerida teist tähelepanu tõmbavat tunnust (liikumine), mis viitab taas kord automaatsetele töötlusprotsessidele. Neljas uurimus keskendus emotsionaalsete näoväljenduste töötlusele, kuna need kannavad keskkonnas hakkamasaamiseks vajalikke sotsiaalseid signaale, mistõttu on alust arvata, et nende töötlus on kujunenud suuresti automaatseks protsessiks. Näitasime, et emotsiooni väljendavaid nägusid avastati kiiremini ja kergemini kui neutraalse ilmega nägusid ning et vihane nägu tõmbas rohkem tähelepanu kui rõõmus (Uurimus IV). Väitekirja viimane osa puudutab visuaalset lahknevusnegatiivsust (ingl Visual Mismatch Negativity ehk vMMN), mis näitab aju võimet avastada automaatselt erinevusi enda loodud mudelist ümbritseva keskkonna kohta. Selle automaatse erinevuse avastamise mehhanismi uurimisse andsid oma panuse nii Uurimus II kui Uurimus IV, mis mõlemad pakuvad välja tõendusi vMMN tekkimise kohta eri tingimustel ja katseparadigmades ning ka vajalikke metodoloogilisi täiendusi. Uurimus V on esimene kogu siiani ilmunud temaatilist teadustööd hõlmav ülevaateartikkel ja metaanalüüs visuaalsest lahknevusnegatiivsusest psühhiaatriliste ja neuroloogiliste haiguste korral, mis panustab oluliselt visuaalse lahknevusnegatiivsuse valdkonna arengusse.The research presented and discussed in the thesis is an experimental exploration of processes in visual perception, which all display a considerable amount of automaticity. These processes are targeted from different angles using different experimental paradigms and stimuli, and by measuring both behavioural and brain responses. In the first three empirical studies, the focus is on motion detection that is regarded one of the most basic processes shaped by evolution. Study I investigated how motion information of an object is processed in the presence of background motion. Although it is widely believed that no motion can be perceived without establishing a frame of reference with other objects or motion on the background, our results found no support for relative motion principle. This finding speaks in favour of a simple and automatic process of detecting motion, which is largely insensitive to the surrounding context. Study II shows that the visual system is built to automatically process motion information that is outside of our attentional focus. This means that even if we are concentrating on some task, our brain constantly monitors the surrounding environment. Study III addressed the question of what happens when multiple stimulus qualities (motion and colour) are present and varied, which is the everyday reality of our visual input. We showed that velocity facilitated the detection of colour changes, which suggests that processing motion and colour is not entirely isolated. These results also indicate that it is hard to ignore motion information, and processing it is rather automatically initiated. The fourth empirical study focusses on another example of visual input that is processed in a rather automatic way and carries high survival value – emotional expressions. In Study IV, participants detected emotional facial expressions faster and more easily compared with neutral facial expressions, with a tendency towards more automatic attention to angry faces. In addition, we investigated the emergence of visual mismatch negativity (vMMN) that is one of the most objective and efficient methods for analysing automatic processes in the brain. Study II and Study IV proposed several methodological gains for registering this automatic change-detection mechanism. Study V is an important contribution to the vMMN research field as it is the first comprehensive review and meta-analysis of the vMMN studies in psychiatric and neurological disorders

Taking attention away from the auditory modality : investigations of the effect on speech processing using machine learning

Real-world speech processing often takes place in complex multisensory environments. Listeners may need to prioritize sensory inputs from modalities other than audition. Selective attention is thought to be critical in selecting the sensory modality most relevant to the task at hand. Two critical research questions have driven crossmodal attention research thus far: first, how early does crossmodal attention influences processing in the unattended modality? Second, is there a limitation in attentional resources between sensory modalities? Set within the context of this prior work, this dissertation aims to examine the effects of crossmodal attention on speech processing when sensory inputs from vision are prioritized. In study 1, we demonstrate that modulating visual perceptual load can impact the early sensory representation of linguistically-relevant pitch contours (Mandarin tones), a suprasegmental feature that is critical to the percept of lexical tones. Further, we provide novel evidence that the impact of the visual load is highly dependent on the predictability of the incoming speech stream. In study 2, we utilized ecologically valid, continuous speech and tested the extent to which dividing attention to a visual task affects neural processing of speech signals. We show that dividing attention between auditory and visual tasks leads to both behavioral and electrophysiological costs in the processing of continuous speech stimuli. The results also demonstrate that the neural encoding of suprasegmental features (e.g., envelope and fundamental frequency) in continuous speech is modulated by diverting attention away from the auditory modality. In contrast, the neural encoding of segmental features (e.g., phonetic features) may be unaffected by taking attention away from the auditory stream. The theoretical and practical implications of the two studies are discussed.Communication Sciences and Disorder

Modality-Specific Effects of Perceptual Load in Multimedia Processing

Digital media are sensory-rich, multimodal, and often highly interactive. An extensive collection of theories and models within the field of media psychology assume the multimodal nature of media stimuli, yet there is current ambiguity as to the independent contributions of visual and auditory content to message complexity and to resource availability in the human processing system. In this article, we argue that explicating the concepts of perceptual and cognitive load can create progress toward a deeper understanding of modality-specific effects in media processing. In addition, we report findings from an experiment showing that perceptual load leads to modality-specific reductions in resource availability, whereas cognitive load leads to a modality-general reduction in resource availability. We conclude with a brief discussion regarding the critical importance of separating modality-specific forms of load in an increasingly multisensory media environment