Alignment of Continuous Auditory and Visual Distractor Stimuli Is Leading to an Increased Performance

Information across different senses can affect our behavior in both positive and negative ways. Stimuli aligned with a target stimulus can lead to improved behavioral performances, while competing, transient stimuli often negatively affect our task performance. But what about subtle changes in task-irrelevant multisensory stimuli? Within this experiment we tested the effect of the alignment of subtle auditory and visual distractor stimuli on the performance of detection and discrimination tasks respectively. Participants performed either a detection or a discrimination task on a centrally presented Gabor patch, while being simultaneously subjected to a random dot kinematogram, which alternated its color from green to red with a frequency of 7.5 Hz and a continuous tone, which was either a frequency modulated pure tone for the audiovisual congruent and incongruent conditions or white noise for the visual control condition. While the modulation frequency of the pure tone initially differed from the modulation frequency of the random dot kinematogram, the modulation frequencies of both stimuli could align after a variable delay, and we measured accuracy and reaction times around the possible alignment time. We found increases in accuracy for the audiovisual congruent condition suggesting subtle alignments of multisensory background stimuli can increase performance on the current task

SEMPAI: a Self‐Enhancing Multi‐Photon Artificial Intelligence for Prior‐Informed Assessment of Muscle Function and Pathology

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as black boxes, exclude biomedical experts, and need extensive data. This is especially problematic for fundamental research in the laboratory, where often only small and sparse data are available and the objective is knowledge discovery rather than automation. Furthermore, basic research is usually hypothesis‐driven and extensive prior knowledge (priors) exists. To address this, the Self‐Enhancing Multi‐Photon Artificial Intelligence (SEMPAI) that is designed for multiphoton microscopy (MPM)‐based laboratory research is presented. It utilizes meta‐learning to optimize prior (and hypothesis) integration, data representation, and neural network architecture simultaneously. By this, the method allows hypothesis testing with DL and provides interpretable feedback about the origin of biological information in 3D images. SEMPAI performs multi‐task learning of several related tasks to enable prediction for small datasets. SEMPAI is applied on an extensive MPM database of single muscle fibers from a decade of experiments, resulting in the largest joint analysis of pathologies and function for single muscle fibers to date. It outperforms state‐of‐the‐art biomarkers in six of seven prediction tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior‐only approaches.The Self‐Enhancing Multi‐Photon AI (SEMPAI) that is designed specifically for basic laboratory research with microscopy is presented. It allows to integrate hypotheses and uses meta‐learning in a biologically interpretable configuration space for knowledge discovery. SEMPAI is applied to a large database of multi‐photon microscopy images of single muscle fibers to gain a deeper understanding of structure–function relationships and pathologies. image European Union's Horizon Marie Skłodowska‐Curie2021 Emerging Talents Initiative of the Friedrich‐Alexander UniversityGerman Research Foundation http://dx.doi.org/10.13039/50110000165

Selective deployment of attention to time and modality and its impact upon behavior and brain oscillations

This thesis investigates the relationship between attention to time and modality at the levels of behavior and pre-stimulus brain oscillations, measured with EEG. Participants were presented with target stimuli from one of two possible modalities, which could appear at one of two different time points. The factors time and modality were interlaced with each other by the fact that each of the modalities was more likely to appear at a different point in time and additionally one of the modalities being more likely overall. We observed that attention to each modality followed its respective temporal likelihood, independently which combination of modalities was used, suggesting a general mechanism for cross-modal temporal decoupling in time. This result is in contrast with cross-modal attention in space, which occurs in a coupled way. At the physiological level, the decoupling effect in time also seems to modulate ongoing neural oscillations in different frequency bands. Based on the results obtained in the time-frequency analysis, we put forward the following tentative hypotheses: alpha oscillations appear to encode switches in modality expectation over sensory cortices, while the beta band might encode the expected modality of the next upcoming stimulus and for the effect of temporal attention itself.Esta tesis investiga la relación entre la atención al tiempo y a la modalidad al nivel de comportamiento y oscilaciones cerebrales pre-estímulo. En los estudios que se presentan aquí, los participantes debían responder a estímulos que se presentaban en una de dos modalidades distintas, y podían aparecer en uno de dos momentos distintos. Los factores de tiempo y modalidad fueron entrelazados el uno con el otro a través la manipulación de su probabilidad, según la cual una modalidad era más probable en un momento u otro y una de las modalidades era más probable en general. Los resultados mostraron que la atención a cada modalidad seguía su respectiva probabilidad temporal, independientemente de la combinación de modalidades, lo cual sugiere un mecanismo general para el desacoplamiento inter-sensorial en la atención temporal. Este resultado es interesante porque contrasta con la atención inter-sensorial en el dominio espacial, donde las distintas modalidades parecen funcionar de manera acoplada. El efecto de desacoplamiento en atención temporal también parece modular las oscilaciones cerebrales antes del momento en anticipación al estímulo, en bandas de frecuencia distintas. Según los resultados obtenidos mediante el registro de estas oscilaciones, elaboramos las siguientes hipótesis: las oscilaciones alfa parecen codificar expectativas sobre cambios de modalidad en las cortezas sensoriales, mientras que la banda beta parece codificar las expectativas sobre la modalidad del siguiente estímulo, y el efecto de atención temporal en sí

Selective deployment of attention to time and modality and its impact upon behavior and brain oscillations

This thesis investigates the relationship between attention to time and modality at the levels of behavior and pre-stimulus brain oscillations, measured with EEG. Participants were presented with target stimuli from one of two possible modalities, which could appear at one of two different time points. The factors time and modality were interlaced with each other by the fact that each of the modalities was more likely to appear at a different point in time and additionally one of the modalities being more likely overall. We observed that attention to each modality followed its respective temporal likelihood, independently which combination of modalities was used, suggesting a general mechanism for cross-modal temporal decoupling in time. This result is in contrast with cross-modal attention in space, which occurs in a coupled way. At the physiological level, the decoupling effect in time also seems to modulate ongoing neural oscillations in different frequency bands. Based on the results obtained in the time-frequency analysis, we put forward the following tentative hypotheses: alpha oscillations appear to encode switches in modality expectation over sensory cortices, while the beta band might encode the expected modality of the next upcoming stimulus and for the effect of temporal attention itself.Esta tesis investiga la relación entre la atención al tiempo y a la modalidad al nivel de comportamiento y oscilaciones cerebrales pre-estímulo. En los estudios que se presentan aquí, los participantes debían responder a estímulos que se presentaban en una de dos modalidades distintas, y podían aparecer en uno de dos momentos distintos. Los factores de tiempo y modalidad fueron entrelazados el uno con el otro a través la manipulación de su probabilidad, según la cual una modalidad era más probable en un momento u otro y una de las modalidades era más probable en general. Los resultados mostraron que la atención a cada modalidad seguía su respectiva probabilidad temporal, independientemente de la combinación de modalidades, lo cual sugiere un mecanismo general para el desacoplamiento inter-sensorial en la atención temporal. Este resultado es interesante porque contrasta con la atención inter-sensorial en el dominio espacial, donde las distintas modalidades parecen funcionar de manera acoplada. El efecto de desacoplamiento en atención temporal también parece modular las oscilaciones cerebrales antes del momento en anticipación al estímulo, en bandas de frecuencia distintas. Según los resultados obtenidos mediante el registro de estas oscilaciones, elaboramos las siguientes hipótesis: las oscilaciones alfa parecen codificar expectativas sobre cambios de modalidad en las cortezas sensoriales, mientras que la banda beta parece codificar las expectativas sobre la modalidad del siguiente estímulo, y el efecto de atención temporal en sí

Alignment of Continuous Auditory and Visual Distractor Stimuli Is Leading to an Increased Performance

Information across different senses can affect our behavior in both positive and negative ways. Stimuli aligned with a target stimulus can lead to improved behavioral performances, while competing, transient stimuli often negatively affect our task performance. But what about subtle changes in task-irrelevant multisensory stimuli? Within this experiment we tested the effect of the alignment of subtle auditory and visual distractor stimuli on the performance of detection and discrimination tasks respectively. Participants performed either a detection or a discrimination task on a centrally presented Gabor patch, while being simultaneously subjected to a random dot kinematogram, which alternated its color from green to red with a frequency of 7.5 Hz and a continuous tone, which was either a frequency modulated pure tone for the audiovisual congruent and incongruent conditions or white noise for the visual control condition. While the modulation frequency of the pure tone initially differed from the modulation frequency of the random dot kinematogram, the modulation frequencies of both stimuli could align after a variable delay, and we measured accuracy and reaction times around the possible alignment time. We found increases in accuracy for the audiovisual congruent condition suggesting subtle alignments of multisensory background stimuli can increase performance on the current task

Alignment of Continuous Auditory and Visual Distractor Stimuli Is Leading to an Increased Performance

Information across different senses can affect our behavior in both positive and negative ways. Stimuli aligned with a target stimulus can lead to improved behavioral performances, while competing, transient stimuli often negatively affect our task performance. But what about subtle changes in task-irrelevant multisensory stimuli? Within this experiment we tested the effect of the alignment of subtle auditory and visual distractor stimuli on the performance of detection and discrimination tasks respectively. Participants performed either a detection or a discrimination task on a centrally presented Gabor patch, while being simultaneously subjected to a random dot kinematogram, which alternated its color from green to red with a frequency of 7.5 Hz and a continuous tone, which was either a frequency modulated pure tone for the audiovisual congruent and incongruent conditions or white noise for the visual control condition. While the modulation frequency of the pure tone initially differed from the modulation frequency of the random dot kinematogram, the modulation frequencies of both stimuli could align after a variable delay, and we measured accuracy and reaction times around the possible alignment time. We found increases in accuracy for the audiovisual congruent condition suggesting subtle alignments of multisensory background stimuli can increase performance on the current task

Cross-modal decoupling in temporal attention between audition and touch

Data de publicació electrónica: 17-05-2018Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting.This research was funded by the Spanish Ministry of Science and Innovation (PSI2016-75558-P(AEI/FEDER)), the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145) to S.S.F

Cross-modal decoupling in temporal attention

Prior studies have repeatedly reported behavioural benefits to events occurring at attended, compared to unattended, points in time. It has been suggested that, as for spatial orienting, temporal orienting of attention spreads across sensory modalities in a synergistic fashion. However, the consequences of cross-modal temporal orienting of attention remain poorly understood. One challenge is that the passage of time leads to an increase in event predictability throughout a trial, thus making it difficult to interpret possible effects (or lack thereof). Here we used a design that avoids complete temporal predictability to investigate whether attending to a sensory modality (vision or touch) at a point in time confers beneficial access to events in the other, non-attended, sensory modality (touch or vision, respectively). In contrast to previous studies and to what happens with spatial attention, we found that events in one (unattended) modality do not automatically benefit from happening at the time point when another modality is expected. Instead, it seems that attention can be deployed in time with relative independence for different sensory modalities. Based on these findings, we argue that temporal orienting of attention can be cross-modally decoupled in order to flexibly react according to the environmental demands, and that the efficiency of this selective decoupling unfolds in time.This research was funded by the Spanish Ministry of Science and Innovation (PSI2010-15426), the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092) and the European Research Council (StG-2010 263145) to S.S.F

Effect of field spread on resting-state magneto encephalography functional network analysis: a computational modeling study

A popular way to analyze resting-state electroencephalography (EEG) and magneto encephalography (MEG) data is to treat them as a functional network in which sensors are identified with nodes and the interaction between channel time series and the network connections. Although conceptually appealing, the network-theoretical approach to sensor-level EEG and MEG data is challenged by the fact that EEG and MEG time series are mixtures of source activity. It is, therefore, of interest to assess the relationship between functional networks of source activity and the ensuing sensor-level networks. Since these topological features are of high interest in experimental studies, we address the question of to what extent the network topology can be reconstructed from sensor-level functional connectivity (FC) measures in case of MEG data. Simple simulations that consider only a small number of regions do not allow to assess network properties; therefore, we use a diffusion magnetic resonance imaging-constrained whole-brain computational model of resting-state activity. Our motivation lies behind the fact that still many contributions found in the literature perform network analysis at sensor level, and we aim at showing the discrepancies between source- and sensor-level network topologies by using realistic simulations of resting-state cortical activity. Our main findings are that the effect of field spread on network topology depends on the type of interaction (instantaneous or lagged) and leads to an underestimation of lagged FC at sensor level due to instantaneous mixing of cortical signals, instantaneous interaction is more sensitive to field spread than lagged interaction, and discrepancies are reduced when using planar gradiometers rather than axial gradiometers. We, therefore, recommend using lagged interaction measures on planar gradiometer data when investigating network properties of resting-state sensor-level MEG data.R. Hindriks and G. Deco were funded by the European Research Council (Advanced Grant DYSTRUCTURE No.295129), the Spanish Research Project PSI2013-42091-P, the CONSOLIDER-INGENIO 2010 Program CSD2007-00012, and the FP7-ICT Brainscales (269921)

Effect of field spread on resting-state magneto encephalography functional network analysis: a computational modeling study

A popular way to analyze resting-state electroencephalography (EEG) and magneto encephalography (MEG) data is to treat them as a functional network in which sensors are identified with nodes and the interaction between channel time series and the network connections. Although conceptually appealing, the network-theoretical approach to sensor-level EEG and MEG data is challenged by the fact that EEG and MEG time series are mixtures of source activity. It is, therefore, of interest to assess the relationship between functional networks of source activity and the ensuing sensor-level networks. Since these topological features are of high interest in experimental studies, we address the question of to what extent the network topology can be reconstructed from sensor-level functional connectivity (FC) measures in case of MEG data. Simple simulations that consider only a small number of regions do not allow to assess network properties; therefore, we use a diffusion magnetic resonance imaging-constrained whole-brain computational model of resting-state activity. Our motivation lies behind the fact that still many contributions found in the literature perform network analysis at sensor level, and we aim at showing the discrepancies between source- and sensor-level network topologies by using realistic simulations of resting-state cortical activity. Our main findings are that the effect of field spread on network topology depends on the type of interaction (instantaneous or lagged) and leads to an underestimation of lagged FC at sensor level due to instantaneous mixing of cortical signals, instantaneous interaction is more sensitive to field spread than lagged interaction, and discrepancies are reduced when using planar gradiometers rather than axial gradiometers. We, therefore, recommend using lagged interaction measures on planar gradiometer data when investigating network properties of resting-state sensor-level MEG data.R. Hindriks and G. Deco were funded by the European Research Council (Advanced Grant DYSTRUCTURE No.295129), the Spanish Research Project PSI2013-42091-P, the CONSOLIDER-INGENIO 2010 Program CSD2007-00012, and the FP7-ICT Brainscales (269921)