Creation of auditory augmented reality using a position-dynamic binaural synthesis system - technical components, psychoacoustic needs, and perceptual evaluation

For a spatial audio reproduction in the context of augmented reality, a position-dynamic binaural synthesis system can be used to synthesize the ear signals for a moving listener. The goal is the fusion of the auditory perception of the virtual audio objects with the real listening environment. Such a system has several components, each of which help to enable a plausible auditory simulation. For each possible position of the listener in the room, a set of binaural room impulse responses (BRIRs) congruent with the expected auditory environment is required to avoid room divergence effects. Adequate and efficient approaches are methods to synthesize new BRIRs using very few measurements of the listening room. The required spatial resolution of the BRIR positions can be estimated by spatial auditory perception thresholds. Retrieving and processing the tracking data of the listener’s head-pose and position as well as convolving BRIRs with an audio signal needs to be done in real-time. This contribution presents work done by the authors including several technical components of such a system in detail. It shows how the single components are affected by psychoacoustics. Furthermore, the paper also discusses the perceptive effect by means of listening tests demonstrating the appropriateness of the approaches

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Publisher Copyright: © 2022 IEEE.Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.Peer reviewe

Eyes on the Narrative : Exploring the Impact of Visual Realism and Audio Presentation on Gaze Behavior in AR Storytelling

Augmented Reality (AR) and Virtual Reality (VR) are essential tools for researchers and practitioners, serving purposes from training to entertainment: many of these applications rely on agents. This study explores the impact of agent characteristics on user reactions, focusing on gaze as a primary visual attention indicator in AR and VR. While existing research has investigated the agent’s gaze and its influence on the user, it is unclear how the agent’s auralization and visualization influence gaze behaviour. We investigate this by studying the impact of rendering style and type of audio on gaze behaviour during a narrative AR experience. Participants listened to a story with the agent visualized as a cartoon-style or realistic virtual human and auralized with spatial or non-spatial audio. The results revealed that the agent’s rendering style significantly influenced gaze behaviour, with cartoon-style agents capturing more visual attention. Audio variations did not yield significant differences. Together, our findings inform the design of AR user interfaces with agents, suggesting that low-realism visualizations are more captivating and, thus, more suitable for experiences where the user is supposed to look at the storyteller

Climatic fluctuations in the hyperarid core of the Atacama Desert during the past 215 ka

Paleoclimate records from the Atacama Desert are rare and mostly discontinuous, mainly recording runoff from the Precordillera to the east, rather than local precipitation. Until now, paleoclimate records have not been reported from the hyperarid core of the Atacama Desert (<2 mm/yr). Here we report the results from multi-disciplinary investigation of a 6.2 m drill core retrieved from an endorheic basin within the Coastal Cordillera. The record spans the last 215 ka and indicates that the long-term hyperarid climate in the Central Atacama witnessed small but significant changes in precipitation since the penultimate interglacial. Somewhat 'wetter' climate with enhanced erosion and transport of material into the investigated basin, commenced during interglacial times (MIS 7, MIS 5), whereas during glacial times (MIS 6, MIS 4-1) sediment transport into the catchment was reduced or even absent. Pelagic diatom assemblages even suggest the existence of ephemeral lakes in the basin. The reconstructed wetter phases are asynchronous with wet phases in the Altiplano but synchronous with increased sea-surface temperatures off the coasts of Chile and Peru, i.e. resembling modern El Nino-like conditions