The listening talker: A review of human and algorithmic context-induced modifications of speech

International audienceSpeech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output

Irregular speech rate dissociates auditory cortical entrainment, evoked responses, and frontal alpha

The entrainment of slow rhythmic auditory cortical activity to the temporal regularities in speech is considered to be a central mechanism underlying auditory perception. Previous work has shown that entrainment is reduced when the quality of the acoustic input is degraded, but has also linked rhythmic activity at similar time scales to the encoding of temporal expectations. To understand these bottom-up and top-down contributions to rhythmic entrainment, we manipulated the temporal predictive structure of speech by parametrically altering the distribution of pauses between syllables or words, thereby rendering the local speech rate irregular while preserving intelligibility and the envelope fluctuations of the acoustic signal. Recording EEG activity in human participants, we found that this manipulation did not alter neural processes reflecting the encoding of individual sound transients, such as evoked potentials. However, the manipulation significantly reduced the fidelity of auditory delta (but not theta) band entrainment to the speech envelope. It also reduced left frontal alpha power and this alpha reduction was predictive of the reduced delta entrainment across participants. Our results show that rhythmic auditory entrainment in delta and theta bands reflect functionally distinct processes. Furthermore, they reveal that delta entrainment is under top-down control and likely reflects prefrontal processes that are sensitive to acoustical regularities rather than the bottom-up encoding of acoustic features

The local translation of KNa in dendritic projections of auditory neurons and the roles of KNa in the transition from hidden to overt hearing loss

Local and privileged expression of dendritic proteins allows segregation of distinct functions in a single neuron but may represent one of the underlying mechanisms for early and insidious presentation of sensory neuropathy. Tangible characteristics of early hearing loss (HL) are defined in correlation with nascent hidden hearing loss (HHL) in humans and animal models. Despite the plethora of causes of HL, only two prevailing mechanisms for HHL have been identified, and in both cases, common structural deficits are implicated in inner hair cell synapses, and demyelination of the auditory nerve (AN). We uncovered that N

Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices

The brain should integrate related but not unrelated information from different senses. Temporal patterning of inputs to different modalities may provide critical information about whether those inputs are related or not. We studied effects of temporal correspondence between auditory and visual streams on human brain activity with functional magnetic resonance imaging ( fMRI). Streams of visual flashes with irregularly jittered, arrhythmic timing could appear on right or left, with or without a stream of auditory tones that coincided perfectly when present ( highly unlikely by chance), were noncoincident with vision ( different erratic, arrhythmic pattern with same temporal statistics), or an auditory stream appeared alone. fMRI revealed blood oxygenation level-dependent ( BOLD) increases in multisensory superior temporal sulcus (mSTS), contralateral to a visual stream when coincident with an auditory stream, and BOLD decreases for noncoincidence relative to unisensory baselines. Contralateral primary visual cortex and auditory cortex were also affected by audiovisual temporal correspondence or noncorrespondence, as confirmed in individuals. Connectivity analyses indicated enhanced influence from mSTS on primary sensory areas, rather than vice versa, during audiovisual correspondence. Temporal correspondence between auditory and visual streams affects a network of both multisensory ( mSTS) and sensory-specific areas in humans, including even primary visual and auditory cortex, with stronger responses for corresponding and thus related audiovisual inputs

Theories of developmental dyslexia: Insights from a multiple case study of dyslexic adults

A multiple case study was conducted in order to assess three leading theories of developmental dyslexia: the phonological, the magnocellular (auditory and visual) and the cerebellar theories. Sixteen dyslexic and 16 control university students were administered a full battery of psychometric, phonological, auditory, visual and cerebellar tests. Individual data reveal that all 16 dyslexics suffer from a phonological deficit, 10 from an auditory deficit, 4 from a motor deficit, and 2 from a visual magnocellular deficit. Results suggest that a phonological deficit can appear in the absence of any other sensory or motor disorder, and is sufficient to cause a literacy impairment, as demonstrated by 5 of the dyslexics. Auditory disorders, when present, aggravate the phonological deficit, hence the literacy impairment. However, auditory deficits cannot be characterised simply as rapid auditory processing problems, as would be predicted by the magnocellular theory. Nor are they restricted to speech. Contrary to the cerebellar theory, we find little support for the notion that motor impairments, when found, have a cerebellar origin, or reflect an automaticity deficit. Overall, the present data support the phonological theory of dyslexia, while acknowledging the presence of additional sensory and motor disorders in certain individuals

Synchronization of a Nonlinear Oscillator: Processing the Cf Component of the Echo-Response Signal in the Cochlea of the Mustached Bat

Cochlear microphonic potential (CM) was recorded from the CF2 region and the sparsely innervated zone (the mustached bat's cochlea fovea) that is specialized for analyzing the Doppler-shifted echoes of the first-harmonic (~61 kHz) of the constant-frequency component of the echolocation call. Temporal analysis of the CM, which is tuned sharply to the 61 kHz cochlear resonance, revealed that at the resonance frequency, and within 1 msec of tone onset, CM is broadly tuned with linear magnitude level functions. CM measured during the ongoing tone and in the ringing after tone offset is 50 dB more sensitive, is sharply tuned, has compressive level functions, and the phase leads onset CM by 90°: an indication that cochlear responses are amplified during maximum basilar membrane velocity. For high-level tones above the resonance frequency, CM appears at tone onset and after tone offset. Measurements indicate that the two oscillators responsible for the cochlear resonance, presumably the basilar and tectorial membranes, move together in phase during the ongoing tone, thereby minimizing net shear between them and hair cell excitation. For tones within 2 kHz of the cochlear resonance the frequency of CM measured within 2 msec of tone onset is not that of the stimulus but is proportional to it. For tones just below the cochlear resonance region CM frequency is a constant amount below that of the stimulus depending on CM measurement delay from tone onset. The frequency responses of the CM recorded from the cochlear fovea can be accounted for through synchronization between the nonlinear oscillators responsible for the cochlear resonance and the stimulus tone

Time Domain Computation of a Nonlinear Nonlocal Cochlear Model with Applications to Multitone Interaction in Hearing

A nonlinear nonlocal cochlear model of the transmission line type is studied in order to capture the multitone interactions and resulting tonal suppression effects. The model can serve as a module for voice signal processing, it is a one dimensional (in space) damped dispersive nonlinear PDE based on mechanics and phenomenology of hearing. It describes the motion of basilar membrane (BM) in the cochlea driven by input pressure waves. Both elastic damping and selective longitudinal fluid damping are present. The former is nonlinear and nonlocal in BM displacement, and plays a key role in capturing tonal interactions. The latter is active only near the exit boundary (helicotrema), and is built in to damp out the remaining long waves. The initial boundary value problem is numerically solved with a semi-implicit second order finite difference method. Solutions reach a multi-frequency quasi-steady state. Numerical results are shown on two tone suppression from both high-frequency and low-frequency sides, consistent with known behavior of two tone suppression. Suppression effects among three tones are demonstrated by showing how the response magnitudes of the fixed two tones are reduced as we vary the third tone in frequency and amplitude. We observe qualitative agreement of our model solutions with existing cat auditory neural data. The model is thus simple and efficient as a processing tool for voice signals.Comment: 23 pages,7 figures; added reference

Aerospace Medicine and Biology: A continuing bibliography with indexes (supplement 141)

This special bibliography lists 267 reports, articles, and other documents introduced into the NASA scientific and technical information system in April 1975

A roadmap to integrate astrocytes into Systems Neuroscience.

Systems neuroscience is still mainly a neuronal field, despite the plethora of evidence supporting the fact that astrocytes modulate local neural circuits, networks, and complex behaviors. In this article, we sought to identify which types of studies are necessary to establish whether astrocytes, beyond their well-documented homeostatic and metabolic functions, perform computations implementing mathematical algorithms that sub-serve coding and higher-brain functions. First, we reviewed Systems-like studies that include astrocytes in order to identify computational operations that these cells may perform, using Ca2+ transients as their encoding language. The analysis suggests that astrocytes may carry out canonical computations in a time scale of subseconds to seconds in sensory processing, neuromodulation, brain state, memory formation, fear, and complex homeostatic reflexes. Next, we propose a list of actions to gain insight into the outstanding question of which variables are encoded by such computations. The application of statistical analyses based on machine learning, such as dimensionality reduction and decoding in the context of complex behaviors, combined with connectomics of astrocyte-neuronal circuits, is, in our view, fundamental undertakings. We also discuss technical and analytical approaches to study neuronal and astrocytic populations simultaneously, and the inclusion of astrocytes in advanced modeling of neural circuits, as well as in theories currently under exploration such as predictive coding and energy-efficient coding. Clarifying the relationship between astrocytic Ca2+ and brain coding may represent a leap forward toward novel approaches in the study of astrocytes in health and disease