Speech playback is harder to understand in noise. Near End Listening Enhancement algorithms try to overcomethe problem by enhancing the speech signal before it is played by a device. Different strategies have beentried, achieving variable degrees of success in specific noise conditions. Such technologies, however, are oftentested in artificial settings - with controlled noise sources and no reverberation. The purpose of this study isto compare a set of state-of-the-art algorithms based on different approaches (adaptive vs non-adaptive, withor without a compensation for reverberation) in simulated real- life scenarios. Binaural noise recordings andimpulse responses of real environments have been used to create two representative scenarios in which speechplayback may occur, namely a domestic and a public space. A preliminary study with N=24 subjects revealedthe need for higher SNRs in the realistic settings (in comparison to controlled noise conditions) in order to obtainthe same levels of intelligibility for plain speech. The goal of the main study is to assess the impact of noiseadaptivity and reverberation awareness in realistic scenarios, in order to better understand how to make speechplayback more robust to noise in real-life situations

Chermaz, Carol

King, Simon

Schepker, Henning

Valentini-Botinhao, Cassia

English

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY

Near end listening enhancement in realistic environments

Publikationsserver der RWTH Aachen University

Near End Listening Enhancement in Realistic Environments

ZENODO

RWTH Publications

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Near end listening enhancement in realistic environmentsCitation for published version:Chermaz, C, Valentini-Botinhao, C, Schepker, H & King, S 2019, Near end listening enhancement inrealistic environments. in M Ochmann, V Michael & J Fels (eds), Proceedings of the 23rd InternationalCongress on Acoustics: Integrating 4th EAA Euroregio 2019. Proceedings of the International Congress onAcoustics, vol. 2019-September, International Commission for Acoustics (ICA), pp. 5731-5735, 23rdInternational Congress on Acoustics: Integrating 4th EAA Euroregio, Aachen, Germany, 9/09/19.https://doi.org/10.18154/RWTH-CONV-239327Digital Object Identifier (DOI):10.18154/RWTH-CONV-239327Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordPublished In:Proceedings of the 23rd International Congress on AcousticsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 23. Jul. 2021[Near end listening enhancement in realistic environments]Carol CHERMAZ(1), Cassia VALENTINI-BOTINHAO(1), Henning SCHEPKER(2), Simon KING(1)(1)The Centre for Speech Technology Research, University of Edinburgh , United Kingdom, c.chermaz@sms.ed.ac.uk(2)Dept. Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, GermanyAbstractSpeech playback is harder to understand in noise. Near End Listening Enhancement algorithms try to overcomethe problem by enhancing the speech signal before it is played by a device. Different strategies have beentried, achieving variable degrees of success in specific noise conditions. Such technologies, however, are oftentested in artificial settings - with controlled noise sources and no reverberation. The purpose of this study isto compare a set of state-of-the-art algorithms based on different approaches (adaptive vs non-adaptive, withor without a compensation for reverberation) in simulated real- life scenarios. Binaural noise recordings andimpulse responses of real environments have been used to create two representative scenarios in which speechplayback may occur, namely a domestic and a public space. A preliminary study with N=24 subjects revealedthe need for higher SNRs in the realistic settings (in comparison to controlled noise conditions) in order to obtainthe same levels of intelligibility for plain speech. The goal of the main study is to assess the impact of noiseadaptivity and reverberation awareness in realistic scenarios, in order to better understand how to make speechplayback more robust to noise in real-life situations.Keywords: Near-end listening enhancement, Realistic Scenarios, Speech intelligibility1 INTRODUCTION1.1 Improving speech playbackSpeech playback refers to signals played by a device over loudspeakers or headphones, e.g. TV, radio or PAsystems. When played over the air, speech sounds becomes mixed with background noise; moreover, reverbera-tion is present in enclosed spaces, posing a hindrance to communication even in the absence of noise. In recentyears a range of NELE (Near End Listening Enhancement) algorithms have been proposed in order to makespeech playback clearer for the listener. Many of these methods are inspired by behaviours observed in nature,i.e. the ways humans change their speaking style when exposed to noise - such as shifting the vocal efforttowards higher frequencies and increasing the consonant/vowel ratio (a phenomenon known as the "Lombardeffect").1.2 State of the art NELE algorithmsIn this study we chose to test SSDRC (Spectral Shaping and Dynamic Range Compression) [8] and Adapt-DRC [6], which proved to be among the best strategies for natural speech in a large scale NELE evaluationknown as "The Hurricane Challenge" [2]. Both algorithms operate on an equal-power-before-and-after process-ing constraint, but manage to improve intelligibility by relocating the energy in the signal to the regions – infrequency and time – that are most relevant for speech comprehension. While AdaptDRC is aware of the noiseand modifies the signal accordingly (applying no modification when there is no noise), SSDRC adapts to thevoice of the speaker (as does AdaptDRC) but does not differentiate among different types of noise. Neitherof the algorithms accounts for reverberation, nor does any of those analysed in [2], as the problem is typicallytreated separately in NELE technologies. A common strategy consists in steady state suppression, as vowels (thesteady states) ”linger on” in reverberation, and therefore ”smear” signal. The OE (Overlap Masking Reductionand Onset Enhancement)[4] algorithm follows this principle and operates a dynamic amplification control basedon the direct-to-reverberant ratio of the speech signal. The impulse response of the environment is assumed to5731be known. In this study, the OE algorithm was added to a processing chain after AdaptDRC, creating hence athird algorithm that will be denoted by ADOE.1.3 Motivation of the study: a realistic test platformNELE technologies are often tested against artificial noise; furthermore, they are typically not tested againstreverberation - besides those algorithms which are specifically designed to tackle this problem. In [2] algorithmswere tested against SSN (Speech Shaped Noise) and a competing speaker, whose voice was recorded in astudio by a professional actor. Such signals represent respectively a source of stationary and of fluctuatingnoise. In this study we take a step towards real-world scenarios, by testing NELE algorithms in simulatedrealistic environments. Artificial noise may be easy to produce and control, but we argue it fails to represent thecomplex nature of real acoustic environments – potentially leading to inaccurate predictions on the performanceof the technologies at test. For this reason, we have simulated a representative set of scenarios in which speechplayback may occur: one domestic space (“the living room”) and one public space (“the cafeteria”), whichfeature recordings of real noise and impulse responses of real spaces.2 MATERIALS AND METHODS2.1 Speech, noise and reverberationBinaural noise recordings from the The University of Oldenburg’s HRIR (Head-Related Impulse Response)database [5] were used in order to simulate the cafeteria, while stimuli from The University of Sheffield’sCHiME corpus [1] were used to create the living room. Impulse responses were taken also from [5]. Thecafeteria is a large public space with a relatively long impulse response; noise recordings were were originallyperformed at a time in which the place was very crowded, obtaining a relatively stationary signal - which iscomparable by definition to SSN. The living room is a small domestic place, with different sources of noisewhich are active at different times: children playing, appliances from the kitchen, footsteps, etc. The noise canbe defined as fluctuating and compared, in principle, to the competing speaker used in [2]. Speech materialwas taken from a recording of the Harvard sentences (https://doi.org/10.7488/ds/2482), which are phonemicallybalanced and are characterised by a relatively low semantic predictability. Sentences where processed with theaforementioned NELE algorithms and convolved with impulse responses from [5] in order to simulate speechsources located in a different position in the room in respect to listener. A schematic representation can be seenin Figure 1.Figure 1. Schematic structure of the simulated environments; dotted line around the cafeteria indicates the spaceis wider than represented. Images are adapted from [5].57322.2 Calculating the SNRsSNR was calculated as 10 log (speech power/noise power), where power is represented by the sum of squaredsamples. Noise snippets were selected at random from the long recordings; endpoints for the sum were definedby the start and end of each sentence.In a real-world situation, the relative power of speech playback at the position of the listener can be measuredwith an SPL meter; the measurement, however, will be performed on the reverberant signal, as it will havetravelled through the air and will be affected by room acoustics. For this reason, the power of the speechsignal was measured on the convolved signal. As the impulse response will change the frequency profile ofthe signal, special care needs to be taken when performing multiple stages of processing. In the main test,speech stimuli were first processed with the NELE algorithms (as this operation must be performed before thesignal is played by a loudspeaker), then convolved with the impulse responses and eventually scaled in orderto achieve the desired SNR. We calculated the amount of gain needed for reverberant plain speech in order toreach the desired power over noise; we then applied the same amount of gain to the modified signals – whichhad the same RMS as plain speech by definition. Noise stimuli were kept at a fixed level. Playback levelswere calibrated at 75 dBA (over headphones) for the cafeteria and 65 dBA for the living room. The speech +noise mixtures were presented to normal hearing listeners over Beyerdynamic 770 headphones in sound treatedbooths.3 RESULTS AND CONCLUSIONS3.1 Preliminary testsTwo preliminary listening tests (with respectively N=24 and N=30 normal hearing listeners) were run in orderto find the psychometric curves for plain speech in the two realistic scenarios. From this data we calculatedthe SNRs required for 25, 50 and 75% intelligibility in terms of WAR, which were used in the main test ashigh, medium and low SNR. We compared the curves for the cafeteria and the living room to those found forSSN and competing speaker in [3]. Our results suggest that higher SNRs are needed in realistic scenarios incomparison to lab controlled noise – up to more than 8 dBs difference for 75% WAR in the SSN vs cafeteriacase.Figure 2. Comparison between the psychometric curves for unmodified speech in [3] and present study. Com-peting speaker is compared to living room as fluctuating noise, while SSN is compared to cafeteria as stationarynoise.5733Figure 3. WAR scores for the different algorithms, at each SNR, in the two simulated environments. LSD =Fisher’s Least Significant Difference.3.2 Main testResults from the main test suggest that different approaches might be best suited for different scenarios. Whilein the cafeteria scene the higher scores of SSDRC are clear, in the living room scene algorithms offer a compa-rable performance. No significant improvement was obtained with the addition of OE to AdaptDRC. In almostall conditions SSDRC outperformed the other modifications, suggesting that a blind approach can be very ef-ficient while computationally convenient and non-intrusive, as SSDRC does not require any noise or impulseresponse measurement. On the other hand, in this study modifications have not been rated for sound quality,which is an important factor for listening comfort in case of long playback times [7]. In SSDRC intelligibilityis boosted at the cost of unnatural sounding speech, while AdaptDRC tries to conserve the naturalness of soundat the cost of lower scores in terms of WAR. An important addition to this study would be a subjective and anobjective measure of listening comfort in addition to intelligibility measurements.ACKNOWLEDGEMENTSThis project has received funding from the EU’s H2020 research and innovation programme under the MSCAGA 67532 (the ENRICH network: www.enrich-etn.eu).This work was supported by the Deutsche Forschungsgesellschaft (DFG, German Research Foundation) - ProjectID 352015383 SFB 1330 C1 and Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) -Project ID 390895286 - EXC 2177/1.Multimedia files from the experiment are available at http://homepages.inf.ed.ac.uk/s1758351/NELE RE.htmlREFERENCES[1] H. Christensen, J. Barker, N. Ma, and P. D. Green. The CHiME corpus: a resource and a challenge forcomputational hearing in multisource environments. In Proc. Interspeech, Chiba, Japan, 2010.[2] M. Cooke, C. Mayo, and C. Valentini-Botinhao. Intelligibility-enhancing speech modifications: the HurricaneChallenge. In Proc. Interspeech, Lyon, France, August 2013.[3] M. Cooke, C. Mayo, C. Valentini-Botinhao, Y. Stylianou, B. Sauert, and Y. Tang. Evaluating the intelli-5734gibility benefit of speech modifications in known noise conditions. Speech Communication, 55(4):572–585,2013.[4] J. Grosse and S. van de Par. A speech preprocessing method based on overlap-masking reduction to increaseintelligibility in reverberant environments. J. Audio Eng. Soc, 65(1/2):31–41, 2017.[5] H. Kayser, S. D. Ewert, J. Anemüller, T. Rohdenburg, V. Hohmann, and B. Kollmeier. Database of multi-channel in-ear and behind-the-ear head-related and binaural room impulse responses. EURASIP Journal onAdvances in Signal Processing, 2009:6, 2009.[6] H. Schepker, J. Rennies, and S. Doclo. Improving speech intelligibility in noise by sii-dependent preprocess-ing using frequency-dependent amplification and dynamic range compression. In Proc. Interspeech, pages3577–3581, Lyon, France, 2013.[7] Y. Tang, C. Arnold, and T. Cox. A study on the relationship between the intelligibility and quality ofalgorithmically-modified speech for normal hearing listeners. Journal of Otorhinolaryngology, Hearing andBalance Medicine, 1(1):5, 2018.[8] T. C. Zorilă, V. Kandia, and Y. Stylianou. Speech-in-noise intelligibility improvement based on spectralshaping and dynamic range compression. In Proc. Interspeech, Portland, USA, September 2012.5735

https://www.research.ed.ac.uk/files/218818623/Near_end_listening_CHERMAZ_DOA04032019_VOR_CC_BY_NC_SA.pdf

Near end listening enhancement in realistic environments

Abstract

Similar works

Full text

Available Versions

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY

Publikationsserver der RWTH Aachen University

ZENODO

RWTH Publications

Edinburgh Research Explorer