Decoding the auditory brain with canonical component analysis

The relation between a stimulus and the evoked brain response can shed light on perceptual processes within the brain. Signals derived from this relation can also be harnessed to control external devices for Brain Computer Interface (BCI) applications. While the classic event-related potential (ERP) is appropriate for isolated stimuli, more sophisticated “decoding” strategies are needed to address continuous stimuli such as speech, music or environmental sounds. Here we describe an approach based on Canonical Correlation Analysis (CCA) that finds the optimal transform to apply to both the stimulus and the response to reveal correlations between the two. Compared to prior methods based on forward or backward models for stimulus-response mapping, CCA finds significantly higher correlation scores, thus providing increased sensitivity to relatively small effects, and supports classifier schemes that yield higher classification scores. CCA strips the brain response of variance unrelated to the stimulus, and the stimulus representation of variance that does not affect the response, and thus improves observations of the relation between stimulus and response

Decoding the auditory brain with canonical component analysis

The relation between a stimulus and the evoked brain response can shed light on perceptual processes within the brain. Signals derived from this relation can also be harnessed to control external devices for Brain Computer Interface (BCI) applications. While the classic event-related potential (ERP) is appropriate for isolated stimuli, more sophisticated “decoding” strategies are needed to address continuous stimuli such as speech, music or environmental sounds. Here we describe an approach based on Canonical Correlation Analysis (CCA) that finds the optimal transform to apply to both the stimulus and the response to reveal correlations between the two. Compared to prior methods based on forward or backward models for stimulus-response mapping, CCA finds significantly higher correlation scores, thus providing increased sensitivity to relatively small effects, and supports classifier schemes that yield higher classification scores. CCA strips the brain response of variance unrelated to the stimulus, and the stimulus representation of variance that does not affect the response, and thus improves observations of the relation between stimulus and response

Perceptual and physiological measures of auditory selective attention in normal-hearing and hearing-impaired listeners

Human listeners can direct top-down spatial auditory attention to listen selectively to one sound source amidst competing sounds. However, many listeners with hearing loss (HL) have trouble on tasks requiring selective auditory attention; even listeners with normal hearing thresholds (NHTs) differ in this ability. Selective attention depends on both top-down executive control and coding fidelity of the peripheral auditory system. Here we explore how low-level sensory perception and high-level attentional modulation interact to contribute to auditory selective attention for listeners with NHTs and HL. In the first study, we designed a paradigm to allow simultaneous measurement of envelope following responses (EFRs), onset event-related potentials (ERPs), and behavioral performance. We varied conditions to alter the degree to which the bottleneck limiting behavior was due to the coding of fine stimulus details vs. top-down control of attentional focus. We found attention modulated ERPs, from cortex, but not EFRs from the brainstem. Importantly, when coding fidelity limited the task, EFRs but not ERPs correlated with behavior; conversely, when sensory cues for segregation were robust, individual behavior correlated with both EFR strength and strength of attentional modulation of cortical responses. In the second study, we explored how HL affects control of auditory selective attention. Listeners with NHTs or with HL identified a simple melody presented simultaneously with two competing melodies, each from different spatial locations. Compared to NHT listeners, HL listeners both performed more poorly and showed less robust attentional modulation of cortical ERPs. While both groups showed some cortical suppression of distracting streams, this modulation was weaker in HL listeners, especially when spatial separation between attended and distracting streams was small. In the final study, we compared temporal coding precision in listeners with NHT and HL using both behavioral and physiological measures. We found that listeners with HL are more sensitive than listeners with NHT to amplitude modulation in both measures. Within the NHT listener group, we found a strong correlation between behavioral and electrophysiological measurements, consistent with cochlear synaptopathy. Overall, these studies demonstrate that everyday communication abilities depend jointly on both low-level differences in sensory coding and high-level ability to control attention

Brain-Computer Interfaces for Non-clinical (Home, Sports, Art, Entertainment, Education, Well-being) Applications

HCI researchers interest in BCI is increasing because the technology industry is expanding into application areas where efficiency is not the main goal of concern. Domestic or public space use of information and communication technology raise awareness of the importance of affect, comfort, family, community, or playfulness, rather than efficiency. Therefore, in addition to non-clinical BCI applications that require efficiency and precision, this Research Topic also addresses the use of BCI for various types of domestic, entertainment, educational, sports, and well-being applications. These applications can relate to an individual user as well as to multiple cooperating or competing users. We also see a renewed interest of artists to make use of such devices to design interactive art installations that know about the brain activity of an individual user or the collective brain activity of a group of users, for example, an audience. Hence, this Research Topic also addresses how BCI technology influences artistic creation and practice, and the use of BCI technology to manipulate and control sound, video, and virtual and augmented reality (VR/AR)