Current Use and Future Perspectives of Spatial Audio Technologies in Electronic Travel Aids

Electronic travel aids (ETAs) have been in focus since technology allowed designing relatively small, light, and mobile devices for assisting the visually impaired. Since visually impaired persons rely on spatial audio cues as their primary sense of orientation, providing an accurate virtual auditory representation of the environment is essential. This paper gives an overview of the current state of spatial audio technologies that can be incorporated in ETAs, with a focus on user requirements. Most currently available ETAs either fail to address user requirements or underestimate the potential of spatial sound itself, which may explain, among other reasons, why no single ETA has gained a widespread acceptance in the blind community. We believe there is ample space for applying the technologies presented in this paper, with the aim of progressively bridging the gap between accessibility and accuracy of spatial audio in ETAs.This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement no. 643636.Peer Reviewe

Short-term plasticity of neuro-auditory processing induced by musical active listening training

Although there is strong evidence for the positive effects of musical training on auditory perception, processing, and training-induced neuroplasticity, there is still little knowledge on the auditory and neurophysiological short-term plasticity through listening training. In a sample of 37 adolescents (20 musicians and 17 nonmusicians) that was compared to a control group matched for age, gender, and musical experience, we conducted a 2-week active listening training (AULOS: Active IndividUalized Listening OptimizationS). Using magnetoencephalography and psychoacoustic tests, the short-term plasticity of auditory evoked fields and auditory skills were examined in a pre-post design, adapted to the individual neuro-auditory profiles. We found bilateral, but more pronounced plastic changes in the right auditory cortex. Moreover, we observed synchronization of the auditory evoked P1, N1, and P2 responses and threefold larger amplitudes of the late P2 response, similar to the reported effects of musical long-term training. Auditory skills and thresholds benefited largely from the AULOS training. Remarkably, after training, the mean thresholds improved by 12 dB for bone conduction and by 3–4 dB for air conduction. Thus, our findings indicate a strong positive influence of active listening training on neural auditory processing and perception in adolescence, when the auditory system is still developing

Experimental Analysis of a Spatialised Audio Interface for People with Visual Impairments

Sound perception is a fundamental skill for many people with severe sight impairments. The research presented in this paper is part of an ongoing project with the aim to create a mobile guidance aid to help people with vision impairments find objects within an unknown indoor environment. This system requires an effective non-visual interface and uses bone-conduction headphones to transmit audio instructions to the user. It has been implemented and tested with spatialised audio cues, which convey the direction of a predefined target in 3D space. We present an in-depth evaluation of the audio interface with several experiments that involve a large number of participants, both blindfolded and with actual visual impairments, and analyse the pros and cons of our design choices. In addition to producing results comparable to the state-of-the-art, we found that Fitts’s Law (a predictive model for human movement) provides a suitable a metric that can be used to improve and refine the quality of the audio interface in future mobile navigation aids

Mixed Structural Models for 3D Audio in Virtual Environments

In the world of ICT, strategies for innovation and development are increasingly focusing on applications that require spatial representation and real-time interaction with and within 3D media environments. One of the major challenges that such applications have to address is user-centricity, reflecting e.g. on developing complexity-hiding services so that people can personalize their own delivery of services. In these terms, multimodal interfaces represent a key factor for enabling an inclusive use of the new technology by everyone. In order to achieve this, multimodal realistic models that describe our environment are needed, and in particular models that accurately describe the acoustics of the environment and communication through the auditory modality. Examples of currently active research directions and application areas include 3DTV and future internet, 3D visual-sound scene coding, transmission and reconstruction and teleconferencing systems, to name but a few. The concurrent presence of multimodal senses and activities make multimodal virtual environments potentially flexible and adaptive, allowing users to switch between modalities as needed during the continuously changing conditions of use situation. Augmentation through additional modalities and sensory substitution techniques are compelling ingredients for presenting information non-visually, when the visual bandwidth is overloaded, when data are visually occluded, or when the visual channel is not available to the user (e.g., for visually impaired people). Multimodal systems for the representation of spatial information will largely benefit from the implementation of audio engines that have extensive knowledge of spatial hearing and virtual acoustics. Models for spatial audio can provide accurate dynamic information about the relation between the sound source and the surrounding environment, including the listener and his/her body which acts as an additional filter. Indeed, this information cannot be substituted by any other modality (i.e., visual or tactile). Nevertheless, today's spatial representation of audio within sonification tends to be simplistic and with poor interaction capabilities, being multimedia systems currently focused on graphics processing mostly, and integrated with simple stereo or multi-channel surround-sound. On a much different level lie binaural rendering approaches based on headphone reproduction, taking into account that possible disadvantages (e.g. invasiveness, non-flat frequency responses) are counterbalanced by a number of desirable features. Indeed, these systems might control and/or eliminate reverberation and other acoustic effects of the real listening space, reduce background noise, and provide adaptable and portable audio displays, which are all relevant aspects especially in enhanced contexts. Most of the binaural sound rendering techniques currently exploited in research rely on the use of Head-Related Transfer Functions (HRTFs), i.e. peculiar filters that capture the acoustic effects of the human head and ears. HRTFs allow loyal simulation of the audio signal that arrives at the entrance of the ear canal as a function of the sound source's spatial position. HRTF filters are usually presented under the form of acoustic signals acquired on dummy heads built according to mean anthropometric measurements. Nevertheless, anthropometric features of the human body have a key role in HRTF shaping: several studies have attested how listening to non-individual binaural sounds results in evident localization errors. On the other hand, individual HRTF measurements on a significant number of subjects result both time- and resource-expensive. Several techniques for synthetic HRTF design have been proposed during the last two decades and the most promising one relies on structural HRTF models. In this revolutionary approach, the most important effects involved in spatial sound perception (acoustic delays and shadowing due to head diffraction, reflections on pinna contours and shoulders, resonances inside the ear cavities) are isolated and modeled separately with a corresponding filtering element. HRTF selection and modeling procedures can be determined by physical interpretation: parameters of each rendering blocks or selection criteria can be estimated from real and simulated data and related to anthropometric geometries. Effective personal auditory displays represent an innovative breakthrough for a plethora of applications and structural approach can also allow for effective scalability depending on the available computational resources or bandwidth. Scenes with multiple highly realistic audiovisual objects are easily managed exploiting parallelism of increasingly ubiquitous GPUs (Graphics Processing Units). Building individual headphone equalization with perceptually robust inverse filtering techniques represents a fundamental step towards the creation of personal virtual auditory displays (VADs). To this regard, several examples might benefit from these considerations: multi-channel downmix over headphones, personal cinema, spatial audio rendering in mobile devices, computer-game engines and individual binaural audio standards for movie and music production. This thesis presents a family of approaches that overcome the current limitations of headphone-based 3D audio systems, aiming at building personal auditory displays through structural binaural audio models for an immersive sound reproduction. The resulting models allow for an interesting form of content adaptation and personalization, since they include parameters related to the user's anthropometry in addition to those related to the sound sources and the environment. The covered research directions converge to a novel framework for synthetic HRTF design and customization that combines the structural modeling paradigm with other HRTF selection techniques (inspired by non-individualized HRTF selection procedures) and represents the main novel contribution of this thesis: the Mixed Structural Modeling (MSM) approach considers the global HRTF as a combination of structural components, which can be chosen to be either synthetic or recorded components. In both cases, customization is based on individual anthropometric data, which are used to either fit the model parameters or to select a measured/simulated component within a set of available responses. The definition and experimental validation of the MSM approach addresses several pivotal issues towards the acquisition and delivery of binaural sound scenes and designing guidelines for personalized 3D audio virtual environments holding the potential of novel forms of customized communication and interaction with sound and music content. The thesis also presents a multimodal interactive system which is used to conduct subjective test on multi-sensory integration in virtual environments. Four experimental scenarios are proposed in order to test the capabilities of auditory feedback jointly to tactile or visual modalities. 3D audio feedback related to user’s movements during simple target following tasks is tested as an applicative example of audio-visual rehabilitation system. Perception of direction of footstep sounds interactively generated during walking and provided through headphones highlights how spatial information can clarify the semantic congruence between movement and multimodal feedback. A real time, physically informed audio-tactile interactive system encodes spatial information in the context of virtual map presentation with particular attention to orientation and mobility (O&M) learning processes addressed to visually impaired people. Finally, an experiment analyzes the haptic estimation of size of a virtual 3D object (a stair-step) whereas the exploration is accompanied by a real-time generated auditory feedback whose parameters vary as a function of the height of the interaction point. The collected data from these experiments suggest that well-designed multimodal feedback, exploiting 3D audio models, can definitely be used to improve performance in virtual reality and learning processes in orientation and complex motor tasks, thanks to the high level of attention, engagement, and presence provided to the user. The research framework, based on the MSM approach, serves as an important evaluation tool with the aim of progressively determining the relevant spatial attributes of sound for each application domain. In this perspective, such studies represent a novelty in the current literature on virtual and augmented reality, especially concerning the use of sonification techniques in several aspects of spatial cognition and internal multisensory representation of the body. This thesis is organized as follows. An overview of spatial hearing and binaural technology through headphones is given in Chapter 1. Chapter 2 is devoted to the Mixed Structural Modeling formalism and philosophy. In Chapter 3, topics in structural modeling for each body component are studied, previous research and two new models, i.e. near-field distance dependency and external-ear spectral cue, are presented. Chapter 4 deals with a complete case study of the mixed structural modeling approach and provides insights about the main innovative aspects of such modus operandi. Chapter 5 gives an overview of number of a number of proposed tools for the analysis and synthesis of HRTFs. System architectural guidelines and constraints are discussed in terms of real-time issues, mobility requirements and customized audio delivery. In Chapter 6, two case studies investigate the behavioral importance of spatial attribute of sound and how continuous interaction with virtual environments can benefit from using spatial audio algorithms. Chapter 7 describes a set of experiments aimed at assessing the contribution of binaural audio through headphones in learning processes of spatial cognitive maps and exploration of virtual objects. Finally, conclusions are drawn and new research horizons for further work are exposed in Chapter 8

Sound localization accuracy in the blind population

The ability to accurately locate a sound source is crucial in the blind population to orient and mobilize independently in the environment. Sound localization is accomplished by the detection of binaural differences in intensity and time of incoming sound waves along with phase differences and spectral cues. It is dependent on auditory sensitivity and processing. However, localization ability can not be predicted from the audiogram or an auditory processing evaluation. Auditory information is not received only from objects making sound, but also from objects reflecting sound. Auditory information used in this manner is called echolocation. Echolocation significantly enhances localization in the absence of vision. Research has shown that echolocation is an important form of localization used by the blind to facilitate independent mobility. However, the ability to localize sound is not evaluated in the blind population. Due to the importance of localization and echolocation for independent mobility in the blind, it would seem appropriate to evaluate the accuracy of this skill set. Echolocation is dependent upon the same auditory processes as localization. More specifically, localization is a precursor to echolocation. Therefore, localization ability will be evaluated in two normal hearing groups, a young normal vision population and young blind population. Both groups will have normal hearing and auditory processing verified by an audiological evaluation that includes a central auditory screening. The localization assessment will be performed using a 24-speaker array in a sound treated chamber with four different testing conditions (1) low-pass broadband stimuli in quiet, (2) low-pass broadband stimuli in noise, (3) high-pass broadband stimuli in quiet, and (4) high-pass broadband speech stimuli in noise. It is hypothesized that blind individuals may exhibit keener localization skills than their normal vision counterparts, particularly if they are experienced, independent travelers. Results of this study may lead to future research in localization assessment, and possibly localization training for blind individuals

On Direct Drive Bone Conduction Devices - Hearing Rehabilitation and Objective Assessment of Stimulation Methods

Bone conduction devices (BCDs) rely on the transmission of sound in form of vibrations generated by a transducer to the inner ear via the skull and surrounding soft tissues. Direct drive BCDs stimulate the skull bone directly, either via a skin-penetrating abutment (BAHAs, Bone Anchored Hearing Aids), or with a transducer implanted under intact skin (active transcutaneous devices).In this thesis, several aspects related to direct drive stimulation were addressed with objective and subjective measurements. Vibrational measurements were performed to assess how the transducer to bone attachment affects the vibrations transmission to the cochleae. Three different attachments for active transcutaneous stimulation were compared to each other and to the BAHA screw. A comparative study was done also between the BAHA system and the novel active transcutaneous Bone Conduction Implant (BCI), where the transducer is attached to the skull bone via a flat surface contact. The BCI is currently on a clinical trial, and a comprehensive assessment of the rehabilitation after three years of device usage is included in this thesis, reporting on a number of audiometric tests, self-reported questionnaires, and objective measurements. Among the objective measures, a new method for intra and post operative verification of the implant functionality was evaluated, consisting in the measurement of the induced sound pressure in the nostril under bone conduction stimulation. In addition to the test battery from the clinical trial protocol, an exploratory study was conducted to investigate the effect of the BCI in a complex multi-talker listening environment.The results from the vibrational measurements were highly frequency-dependent, with a general trend of improved transmission when the contact area between transducer and bone was limited, especially for frequencies above 5 kHz. Sound field tone and speech tests, and user reported questionnaires show that the BCI provides considerable improvement from the unaided condition and contributes to a general increase of patients\u27 life quality, with consistent outcomes over time. The implant verification method seems promising and showed stable properties of the implant to bone transmission. When compared to BAHAs, the BCI was found to be a viable alternative for indicated patients. In noisy and complex listening environments, the BCI users showed a lower ability to make use of the spatial cues when aided with their device, but an overall greater tolerance to interfering talkers

Binaural hearing with bone conduction stimulation

It has been argued that apparent masking-level differences (MLDs) in users of bilateral bone-anchored hearing aids (BAHAs) provide evidence of binaural hearing. However, there is considerably less acoustical isolation between the two ears with bone conduction (BC) compared to air conduction (AC). The apparent MLDs may have arisen, at least in part, from inter-cranial interference between signals arising from the two BAHAs (i.e. monaural effect). That might also explain some of the inter-individual variation in both the magnitude and the direction of the MLDs reported in BAHA users. The present study was composed of three experimental stages with the main aim to investigate the influence of interference in normal hearing participants by measuring masking level difference in AC and BC to explore the conditions contributing to the reported variation. An additional aim was to investigate the performance of a newly designed BC transducer; the balanced electromagnetic separation transducer (BEST), for bone conduction research as well as more general clinical use.Stage 1 evaluated the performance of the BEST in comparison to the clinically used RadioEar B71 in a series of acoustical (sensitivity and harmonic distortion) and psychoacoustical (hearing thresholds and vibrotactile thresholds) measurements. The results from these studies led to the use of the BEST in the second and third stages because they produced significantly lower harmonic distortion at low frequencies (mainly 250 Hz). The psychoacoustic measurements alluded to the need to use different calibration values with the BESTs.Stage 2 was a preliminary investigation comparing the MLDs with standard bilateral configurations between the AC and BC in nine normal-hearing participants. Signals were pure tones at one of three frequencies (250, 500, 1000 Hz), presented via AC or BC. Broadband noise (100- 5000 Hz) was always presented via AC at 70 dB SPL. Thresholds were estimated using a three-alternative forced choice procedure combined with an adaptive staircase. Transducers used were insert earphones and the BESTs for BC testing. The results from this stage showed a statistical significant difference between AC and BC MLDs at 250, 500 and 1000 Hz (mean difference is 9.4, 6.6 and 3.5 dB respectively). Evidence of the change in the MLDs direction is observed at 250 Hz in three participants.Stage 3 consisted of the investigation of inter-cranial interference in eighteen normal hearing participants. This stage was composed of three main measurements. The first measurement compared the AC and BC MLDs at three test frequencies. The second measurement evaluated the transcranial attenuation (TA). The third measurement was the novel feature of the study it evaluated the monaural interference effect through the measurement of the diotic and dichotic conditions in one test ear. A significant discrepancy was found between the AC and BC MLDs of approximately 6, 1.5 and 2.5 dB at 500, 1000 and 2000 Hz, respectively. The TA was found to be lower than 10 dB at the three test frequencies. Measurable MTLDs were reported in some of the participants, high inter-subject variability was observed in the direction of the MTLDs.The BEST can reliably replace the B71 in clinical setup. Formal adjustment of the reference equivalent threshold force levels is advised. Binaural hearing was achieved through bilateral BC stimulation to a lesser magnitude compared to AC MLDs in normal hearing participants. The discrepancy between the AC and BC MLDs was reduced with the increase in the frequency. The discrepancy can partially be explained by the cross-talk of the signal in one ear. The results showed that in some participants the magnitude of the monaural tone level difference was similar to the magnitude of the BC MLD. Further investigation is recommended to investigate the association of the transcranial delay with the discrepancy between the AC and BC MLDs. This investigation also recommends the investigation of the AC and BC MLDs in patients fitted with bilateral BAHAs

Discrimination of sounds in hearing

Imperial Users onl