The use of acoustic cues in phonetic perception: Effects of spectral degradation, limited bandwidth and background noise

Hearing impairment, cochlear implantation, background noise and other auditory degradations result in the loss or distortion of sound information thought to be critical to speech perception. In many cases, listeners can still identify speech sounds despite degradations, but understanding of how this is accomplished is incomplete. Experiments presented here tested the hypothesis that listeners would utilize acoustic-phonetic cues differently if one or more cues were degraded by hearing impairment or simulated hearing impairment. Results supported this hypothesis for various listening conditions that are directly relevant for clinical populations. Analysis included mixed-effects logistic modeling of contributions of individual acoustic cues for various contrasts. Listeners with cochlear implants (CIs) or normal-hearing (NH) listeners in CI simulations showed increased use of acoustic cues in the temporal domain and decreased use of cues in the spectral domain for the tense/lax vowel contrast and the word-final fricative voicing contrast. For the word-initial stop voicing contrast, NH listeners made less use of voice-onset time and greater use of voice pitch in conditions that simulated high-frequency hearing impairment and/or masking noise; influence of these cues was further modulated by consonant place of articulation. A pair of experiments measured phonetic context effects for the "s/sh" contrast, replicating previously observed effects for NH listeners and generalizing them to CI listeners as well, despite known deficiencies in spectral resolution for CI listeners. For NH listeners in CI simulations, these context effects were absent or negligible. Audio-visual delivery of this experiment revealed enhanced influence of visual lip-rounding cues for CI listeners and NH listeners in CI simulations. Additionally, CI listeners demonstrated that visual cues to gender influence phonetic perception in a manner consistent with gender-related voice acoustics. All of these results suggest that listeners are able to accommodate challenging listening situations by capitalizing on the natural (multimodal) covariance in speech signals. Additionally, these results imply that there are potential differences in speech perception by NH listeners and listeners with hearing impairment that would be overlooked by traditional word recognition or consonant confusion matrix analysis

The listening talker: A review of human and algorithmic context-induced modifications of speech

International audienceSpeech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output

Bilaterally Combined Electric and Acoustic Hearing in Mandarin-Speaking Listeners: The Population With Poor Residual Hearing

The hearing loss criterion for cochlear implant candidacy in mainland China is extremely stringent (bilateral severe to profound hearing loss), resulting in few patients with substantial residual hearing in the nonimplanted ear. The main objective of the current study was to examine the benefit of bimodal hearing in typical Mandarin-speaking implant users who have poorer residual hearing in the nonimplanted ear relative to those used in the English-speaking studies. Seventeen Mandarinspeaking bimodal users with pure-tone averages of 80 dB HL participated in the study. Sentence recognition in quiet and in noise as well as tone and word recognition in quiet were measured in monaural and bilateral conditions. There was no significant bimodal effect for word and sentence recognition in quiet. Small bimodal effects were observed for sentence recognition in noise (6%) and tone recognition (4%). The magnitude of both effects was correlated with unaided thresholds at frequencies near voice fundamental frequencies (F0s). A weak correlation between the bimodal effect for word recognition and unaided thresholds at frequencies higher than F0s was identified. These results were consistent with previous findings that showed more robust bimodal benefits for speech recognition tasks that require higher spectral resolution than speech recognition in quiet. The significant but small F0-related bimodal benefit was also consistent with the limited acoustic hearing in the nonimplanted ear of the current subject sample, who are representative of the bimodal users in mainland China. These results advocate for a more relaxed implant candidacy criterion to be used in mainland China

The Speech Critical Band (S-CB) in Cochlear Implant Users: Frequency Resolution Employed During the Reception of Everyday Speech

It is widely recognized that cochlear implant (CI) users have limited spectral resolution and that this represents a primary limitation. In contrast to traditional measures, Healy and Bacon [(2006) 119, J. Acoust. Soc. Am.] established a procedure for directly measuring the spectral resolution employed during processing of running speech. This Speech-Critical Band (S-CB) reflects the listeners’ ability to extract spectral detail from an acoustic speech signal. The goal of the current study was to better determine the resolution that CI users are able to employ when processing speech. Ten CI users between the ages of 32 and 72 years using Cochlear Ltd. devices participated. The original standard recordings from the Hearing In Noise Test (HINT) were filtered to a 1.5-octave band, which was then partitioned into sub-bands. Spectral information was removed from each partition and replaced with an amplitude-modulated noise carrier band; the modulated carriers were then summed for presentation. CI subject performance increased with increasing spectral resolution (increasing number of partitions), never reaching asymptote. This result stands in stark contrast to expectation, as it indicates that increases in spectral resolution up to that of normal hearing produced increases in performance. Accordingly, it is concluded that CI users can access spectral information as high as that of normal hearing (NH) when presented with narrowband speech stimuli. These results have implications for the design of future devices that allow better representation of tonal languages, music, and speech in noise

Melodic contour identification and speech recognition by school-aged children

Using the Sung Speech Corpus (SSC), which encompasses a single database that contains musical pitch, timbre variations and speech information in identification tasks, the current study aimed to explore the development of normal-hearing children’s ability to use the pitch and timbre cues. Thirteen normal hearing children were recruited for the study ages ranging from 7 to 16 years old. Participants were separated into two separate groups: Younger (7-9) and Older (10-16). Musical Experience was taken into account as well. The Angel Sound ™ program was utilized for testing which was adopted from previous studies, most recently Crew, Galvin, and Fu (2015). Participants were asked to identify either pitch contour or a five word sentence while the one not being identified was manipulated in quiet. Each sentence recognition task was also tested at three different SNRs (-3, 0, 3 dB). For sentence recognition in quiet, children with musical training performed better than those without. A significant interaction between Age-Group and Musical Experience was also seen, such that Younger children showed more benefit from musical training than Older, musically trained children. Significant effect of pitch contour on sentence recognition in noise was found showing that naturally produced speech stimuli were easier to identify when competing background noise was introduced for all children than speech stimuli with an unnatural pitch contour. Significant effect of speech timbre on MCI was found which demonstrates that as the timbre complexity increases, the MCI performance decreases. The current study concluded that pitch and timbre cues interfered with each other in child listeners, depending on the listening demands (SNR, tasks, etc.). Music training can improve overall speech and music perception