Flexible binaural resynthesis of room impulse responses for augmented reality research

International audienceA basic building block in audio for Augmented Reality (AR) is the use of virtual sound sources layered on top of any real sources present in an environment. In order to perceive these virtual sources as belonging to the natural scene it is important to match their acoustic parameters to those of a real source with the same characteristics, i.e. radiation properties, sound propagation and head-related impulse response (HRIR). However, it is still unclear to what extent these parameters need to be matched in order to generate plausible scenes in which virtual sound sources blend seamlessly with real sound sources. This contribution presents an auralization framework that allows protyping of augmented reality scenarios from measured multichannel room impulse responses to get a better understanding of the relevance of individual acoustic parameters.A well-established approach for binaural measurement and reproduction of sound scenes is based on capturing binaural room impulse responses (BRIR) using a head and torso simulator (HATS) and convolving these BRIRs dynamically with audio content according to the listener head orientation. However, such measurements are laborious and time consuming, requiring measuring the scene with the HATS in multiple orientations. Additionally, the HATS HRIR is inherently encoded in the BRIRs, making them unsuitable for personalization for different listeners. The approach presented here consists of the resynthesis and dynamic binaural reproduction of multichannel room impulse responses (RIR) using an arbitrary HRIR dataset. Using a compact microphone array, we obtained a pressure RIR and a set of auxiliary RIRs, and we applied the Spatial Decomposition Method (SDM) to estimate the direction-of-arrival (DOA) of the different sound events in the RIR. The DOA information was used to map sound pressure to different locations by means of an HRIR dataset, generating a binaural room impulse response (BRIR) for a specific orientation. By either rotating the DOA or the HRIR data set, BRIRs for any direction may be obtained. Auralizations using SDM are known to whiten the spectrum of late reverberation. Available alternatives such as time-frequency equalization were not feasible in this case, as a different time-frequency filter would be necessary for each direction, resulting in a non-homogeneous equalization of the BRIRs. Instead, the resynthesized BRIRs were decomposed into sub-bands and the decay slope of each sub-band was modified independently to match the reverberation time of the original pressure RIR. In this way we could apply the same reverberation correction factor to all BRIRs. In addition, we used a direction independent equalization to correct for timbral effects of equipment, HRIR, and signal processing. Real-time reproduction was achieved by means of a custom Max/MSP patch, in which the direct sound, early reflections and late reverberation were convolved separately to allow real-time changes in the time-energy properties of the BRIRs. The mixing time of the reproduced BRIRs is configurable and a single direction independent reverberation tail is used. To evaluate the quality of the resynthesis method in a real room, we conducted both objective and perceptual comparisons for a variety of source positions. The objective analysis was performed by comparing real measurements of a KEMAR mannequin with the resynthesis at the same receiver location using a simulated KEMAR HRIR. Typical room acoustic parameters of both real and resynthsized acoustics were found to be in good agreement. The perceptual validation consisted of a comparison of a loudspeaker and its resynthesized counterpart. Non-occluding headphones with individual equalization were used to ensure that listeners were able to simultaneously listen to the real and the virtual samples. Subjects were allowed to listen to the sounds for as long as they desired and freely switch between the real and virtual stimuli in real time. The integration of an Optitrack motion tracking system allowed us to present world-locked audio, accounting for head rotations.We present here the results of this listening test (N = 14) with three sections: discrimination, identification, and qualitative ratings. Preliminary analysis revealed that in these conditions listeners were generally able to discriminate between real and virtual sources and were able to consistently identify which of the presented sources was real and which was virtual. The qualitative analysis revealed that timbral differences are the most prominent cues for discrimination and identification, while spatial cues are well preserved. All the listeners reported good externalization of the binaural audio.Future work includes extending the presented validation to more environments, as well as implementing tools to arbitrarily modify BRIRs in the spatial, temporal, and frequency domains in order to study the perceptual requirements of room acoustics reproduction in AR

SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning

We introduce SoundSpaces 2.0, a platform for on-the-fly geometry-based audio rendering for 3D environments. Given a 3D mesh of a real-world environment, SoundSpaces can generate highly realistic acoustics for arbitrary sounds captured from arbitrary microphone locations. Together with existing 3D visual assets, it supports an array of audio-visual research tasks, such as audio-visual navigation, mapping, source localization and separation, and acoustic matching. Compared to existing resources, SoundSpaces 2.0 has the advantages of allowing continuous spatial sampling, generalization to novel environments, and configurable microphone and material properties. To our knowledge, this is the first geometry-based acoustic simulation that offers high fidelity and realism while also being fast enough to use for embodied learning. We showcase the simulator's properties and benchmark its performance against real-world audio measurements. In addition, we demonstrate two downstream tasks -- embodied navigation and far-field automatic speech recognition -- and highlight sim2real performance for the latter. SoundSpaces 2.0 is publicly available to facilitate wider research for perceptual systems that can both see and hear.Comment: Camera-ready version. Website: https://soundspaces.org. Project page: https://vision.cs.utexas.edu/projects/soundspaces

Auralization systems for simulation of augmented reality experiences in virtual environments

Augmented reality has the potential to connect people anywhere, anytime, and provide them with interactive virtual objects that enhance their lives. To deliver contextually appropriate audio for these experiences, a much greater understanding of how users will interact with augmented content and each other is needed. This contribution presents a system for evaluating human behavior and augmented reality device performance in calibrated synthesized environments. The system consists of a spherical loudspeaker array capable of spatial audio reproduction in a noise isolated and acoustically dampened room. The space is equipped with motion capture systems that track listener position, orientation, and eye gaze direction in temporal synchrony with audio playback and capture to allow for interactive control over the acoustic environment. In addition to spatial audio content from the loudspeaker array, supplementary virtual objects can be presented to listeners using motion-tracked unoccluding headphones. The system facilitates a wide array of studies relating to augmented reality research including communication ecology, spatial hearing, room acoustics, and device performance. System applications and configuration, calibration, processing, and validation routines are presented

The most creative organization in the world? The BBC, 'creativity' and managerial style

The managerial styles of two BBC directors-general, John Birt and Greg Dyke, have often been contrasted but not so far analysed from the perspective of their different views of 'creative management'. This article first addresses the orthodox reading of 'Birtism'; second, it locates Dyke's 'creative' turn in the wider context of fashionable neo-management theory and UK government creative industries policy; third, it details Dyke's drive to change the BBC's culture; and finally, it concludes with some reflections on the uncertainties inherent in managing a creative organisation

New OH Zeeman measurements of magnetic field strengths in molecular clouds

We present the results of a new survey of 23 molecular clouds for the Zeeman effect in OH undertaken with the ATNF Parkes 64-m radio telescope and the NRAO Green Bank 43-m radio telescope. The Zeeman effect was clearly detected in the cloud associated with the HII region RCW 38, with a field strength of 38+/-3 micro-Gauss, and possibly detected in a cloud associated with the HII region RCW 57, with a field strength of -203+/-24 micro-Gauss. The remaining 21 measurements give formal upper limits to the magnetic field strength, with typical 1-sigma sensitivities <20 micro-Gauss. For 22 of the molecular clouds we are also able to determine thecolumn density of the gas in which we have made a sensitive search for the Zeeman effect. We combine these results with previous Zeeman studies of 29 molecular clouds, most of which were compiled by Crutcher (1999), for a comparsion of theoretical models with the data. This comparison implies that if the clouds can be modeled as initially spherical with uniform magnetic fields and densities that evolve to their final equilibrium state assuming flux-freezing then the typical cloud is magnetically supercritical, as was found by Crutcher (1999). If the clouds can be modeled as highly flattened sheets threaded by uniform perpendicular fields, then the typical cloud is approximately magnetically critical, in agreement with Shu et al. (1999), but only if the true values of the field for the non-detections are close to the 3-sigma upper limits. If instead these values are significantly lower (for example, similar to the 1-sigma limits), then the typical cloud is generally magnetically supercritical.Comment: 39 pages, 7 figures. Accepted for publication in Ap

What voters want: reactions to candidate characteristics in a survey experiment

There has been extensive research into the extent to which voters utilise short cuts based on gender and race stereotypes when evaluating candidates, but relatively little is known about how they respond to other background characteristics. We compare the impact of candidates' sex, religion, age, education, occupation and location/residence through a survey experiment in which respondents rate two candidates based on short biographies. We find small differences in the ratings of candidates in response to sex, religion, age and education cues but more sizeable effects are apparent for the candidate's occupation and place of residence. Even once we introduce a control for political party into our experimental scenarios the effect of candidate's place of residence continues to have a sizeable impact on candidate evaluations. Our research suggests that students of electoral behaviour should pay attention to a wider range of candidate cues

Detection and Attribution of Observed Changes in Northern Hemisphere Spring Snow Cover

Abstract: Significant declines in spring Northern Hemisphere (NH) snow cover extent (SCE) have been observed over the last five decades. As one step toward understanding the causes of this decline, an optimal fingerprinting technique is used to look for consistency in the temporal pattern of spring NH SCE between observations and simulations from 15 global climate models (GCMs) that form part of phase 5 of the Coupled Model Intercomparison Project. The authors examined simulations from 15 GCMs that included both natural and anthropogenic forcing and simulations from 7 GCMs that included only natural forcing. The decline in observed NH SCE could be largely explained by the combined natural and anthropogenic forcing but not by natural forcing alone. However, the 15 GCMs, taken as a whole, underpredicted the combined forcing response by a factor of 2. How much of this underprediction was due to underrepresentation of the sensitivity to external forcing of the GCMs or to their underrepresentation of internal variability has yet to be determined.This is the publisher’s final pdf. The published article is copyrighted by the American Meteorological Society and can be found at: http://www.ametsoc.org/pubs/journals/jcli/.Keywords: Statistical techniques, Climate models, Anthropogenic effects, Snow, Northern Hemispher

Evaluation of the rheumatoid arthritis susceptibility loci HLA-DRB1, PTPN22, OLIG3/TNFAIP3, STAT4 and TRAF1/C5 in an inception cohort.

INTRODUCTION: This study investigated five confirmed rheumatoid arthritis (RA) susceptibility genes/loci (HLA-DRB1, PTPN22, STAT4, OLIG3/TNFAIP3 and TRAF1/C5) for association with susceptibility and severity in an inception cohort. METHODS: The magnitude of association for each genotype was assessed in 1,046 RA subjects from the Yorkshire Early RA cohort and in 5,968 healthy UK controls. Additional exploratory subanalyses were undertaken in subgroups defined by autoantibody status (rheumatoid factor and anti-cyclic citrullinated peptide) or disease severity (baseline articular erosions, Health Assessment Questionnaire (HAQ) score and swollen joint count (SJC)). RESULTS: In the total RA inception cohort, the HLA-DRB1 shared epitope (per-allele odds ratio (OR) = 2.1, trend P < 0.0001), PTPN22 (per-allele OR = 1.5, trend P < 0.0001), OLIG3/TNFAIP3 locus (per-allele OR = 1.2, trend P = 0.009) and TRAF1/C5 locus (per-allele OR = 1.1, trend P = 0.04) were associated with RA. The magnitude of association for these loci was increased in those patients who were autoantibody-positive. PTPN22 was associated with autoantibody-negative RA (per-allele OR = 1.3, trend P = 0.04). There was no evidence of association between these five genetic loci and baseline erosions or SJC in the total RA cohort, after adjustment for symptom duration. TRAF1/C5 was significantly associated with baseline HAQ, however, following adjustment for symptom duration (P trend = 0.03). CONCLUSIONS: These findings support the mounting evidence that different genetic loci are associated with autoantibody-positive and autoantibody-negative RA, possibly suggesting that many of the genes identified to date are associated with autoantibody production. Additional studies with a specific focus on autoantibody-negative RA will be needed to identify the genes predisposing to this RA subgroup. The TRAF1/C5 locus in particular warrants further investigation in RA as a potential disease severity locus