Audio, visual, and audio-visual egocentric distance perception by moving participants in virtual environments

International audienceA study on audio, visual, and audio-visual egocentric distance perception by moving participants in virtual environments is presented. Audio-visual rendering is provided using tracked passive visual stereoscopy and acoustic wave fi eld synthesis (WFS). Distances are estimated using indirect blind-walking (triangulation) under each rendering condition. Experimental results show that distances perceived in the virtual environment are accurately estimated or overestimated for rendered distances closer than the position of the audio-visual rendering system and underestimated for distances farther. Interestingly, participants perceived each virtual object at a modality-independent distance when using the audio modality, the visual modality, or the combination of both. Results show WFS capable of synthesizing perceptually meaningful sound fields in terms of distance. Dynamic audio-visual cues were used by participants when estimating the distances in the virtual world. Moving may have provided participants with a better visual distance perception of close distances than if they were static. No correlation between the feeling of presence and the visual distance underestimation has been found. To explain the observed perceptual distance compression, it is proposed that, due to con flicting distance cues, the audio-visual rendering system physically anchors the virtual world to the real world. Virtual objects are thus attracted by the physical audio-visual rendering system

Interaction between auditory and visual perceptions on distance estimations in a virtual environment

International audienceNavigation in virtual environments relies on an accurate spatial rendering. A virtual object is localized according to its position in the environment, which is usually defined by the following three coordinates: azimuth, elevation and distance. Even though several studies investigated the perception of auditory and visual cues in azimuth and elevation, little has been made on the distance dimension. This study aims at investigating the way humans estimate visual and auditory egocentric distances of virtual objects. Subjects were asked to estimate the egocentric distance of 2–20 m distant objects in three contexts: auditory perception alone, visual one alone, combination of both perceptions (with coherent and incoherent visual and auditory cues). Even though egocentric distance was under-estimated in all contexts, the results showed a higher influence of visual information than auditory information on the perceived distance. Specifically, the bimodal incoherent condition gave perceived distances equivalent to those in the visual-only condition only when the visual target was closer to the subject than the auditory target

Shaping the auditory peripersonal space with motor planning in immersive virtual reality

Immersive audio technologies require personalized binaural synthesis through headphones to provide perceptually plausible virtual and augmented reality (VR/AR) simulations. We introduce and apply for the first time in VR contexts the quantitative measure called premotor reaction time (pmRT) for characterizing sonic interactions between humans and the technology through motor planning. In the proposed basic virtual acoustic scenario, listeners are asked to react to a virtual sound approaching from different directions and stopping at different distances within their peripersonal space (PPS). PPS is highly sensitive to embodied and environmentally situated interactions, anticipating the motor system activation for a prompt preparation for action. Since immersive VR applications benefit from spatial interactions, modeling the PPS around the listeners is crucial to reveal individual behaviors and performances. Our methodology centered around the pmRT is able to provide a compact description and approximation of the spatiotemporal PPS processing and boundaries around the head by replicating several well-known neurophysiological phenomena related to PPS, such as auditory asymmetry, front/back calibration and confusion, and ellipsoidal action fields

Reaching measures and feedback effects in auditory peripersonal space

We analyse the effects of exploration feedback on reaching measures of perceived auditory peripersonal space (APS) boundary and the auditory distance perception (ADP) of sound sources located within it. We conducted an experiment in which the participants had to estimate if a sound source was (or not) reachable and to estimate its distance (40 to 150 cm in 5-cm steps) by reaching to a small loudspeaker. The stimulus consisted of a train of three bursts of Gaussian broadband noise. Participants were randomly assigned to two groups: Experimental (EG) and Control (CG). There were three phases in the following order: Pretest–Test–Posttest. For all phases, the listeners performed the same task except for the EG-Test phase where the participants reach in order to touch the sound source. We applied models to characterise the participants’ responses and provide evidence that feedback significantly reduces the response bias of both the perceived boundary of the APS and the ADP of sound sources located within reach. In the CG, the repetition of the task did not affect APS and ADP accuracy, but it improved the performance consistency: the reachable uncertainty zone in APS was reduced and there was a tendency to decrease variability in ADP.Fil: Hug, Mercedes Ximena. Universidad Tecnológica Nacional. Facultad Regional Córdoba. Centro de Investigación y Transferencia en Acústica. - Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigación y Transferencia en Acústica; Argentina. Universidad Nacional de Córdoba. Facultad de Psicología; ArgentinaFil: Vergara, Ramiro Oscar. Universidad Nacional de Quilmes. Escuela Universitaria de Artes. Laboratorio de Acustica y Percepcion Sonora.; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Tommasini, Fabián Carlos. Universidad Tecnológica Nacional. Facultad Regional Córdoba. Centro de Investigación y Transferencia en Acústica. - Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigación y Transferencia en Acústica; ArgentinaFil: Etchemendy, Pablo Esteban. Universidad Nacional de Quilmes. Escuela Universitaria de Artes. Laboratorio de Acustica y Percepcion Sonora.; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Bermejo, Fernando Raul. Universidad Tecnológica Nacional. Facultad Regional Córdoba. Centro de Investigación y Transferencia en Acústica. - Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigación y Transferencia en Acústica; Argentina. Universidad Nacional de Córdoba. Facultad de Psicología; ArgentinaFil: Fernandez, Laura Gabriela. Universidad Tecnológica Nacional. Facultad Regional Córdoba. Centro de Investigación y Transferencia en Acústica. - Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigación y Transferencia en Acústica; Argentina. Hôpital Saint-Antoine. Centre d’évaluation et de traitement de la douleur. Service de psychologie; Franci

Les retours tactile et kinesthésique améliorent la perception de distance en réalité virtuelle

National audienceResearch spanning psychology, neuroscience and HCI found that depth perception distortion is a common problem in virtual reality. This distortion results in depth compression, where users perceive objects closer than their intended distance. Studies suggested that cues, such as audio and haptic, help to solve this issue. We focus on haptic feedback and investigate how force feedback compares to tactile feedback within peripersonal space in reducing depth perception distortion. Our study (N=12) compares the use of haptic force feedback, vibration haptic feedback, a combination of both or no feedback. Our results show that both vibration and force feedback improve depth perception distortion over no feedback (8.3 times better distance estimation than with no haptic feedback vs. 1.4 to 1.5 times better with either vibration or force feedback on their own). Participants also subjectively preferred using force feedback, or a combination of force and vibration feedback, over no feedback.Des recherches en psychologie, neurosciences et IHM ont montré que la distorsion de la perception des distances est un problème courant en réalité virtuelle. Cette distorsion entraîne une compression des profondeurs, et les utilisateurs perçoivent des objets plus proches qu'ils ne le sont. Dans ce papier, nous nous concentrons sur le retour haptique et examinons comment le retour de force se compare au retour tactile pour réduire la compression des profondeurs. Notre étude (N = 12) compare l'utilisation du retour de force, le retour tactile vibratoire, la combinaison des deux ou l'absence de retour. Nos résultats montrent que le retour tactile et le retour de force améliorent la perception de la profondeur. L'estimation de distance est 8.3 fois meilleure que sans retour, par rapport à 1.4-1.5 fois avec retour tactile vibratoire ou de force non-combinés. Les participants ont également préféré utiliser le retour de force, ou une combinaison de force et tactile

Designing passenger experiences for in-car Mixed Reality

In day-to-day life, people spend a considerable amount of their time on the road. People seek to invest travel time for work and well-being through interaction with mobile and multimedia applications on personal devices such as smartphones and tablets. However, for new computing paradigms, such as mobile mixed reality (MR), their usefulness in this everyday transport context, in-car MR remains challenging. When future passengers immerse in three-dimensional virtual environments, they become increasingly disconnected from the cabin space, vehicle motion, and other people around them. This degraded awareness of the real environment endangers the passenger experience on the road, which initially motivates this thesis to question: can immersive technology become useful in the everyday transport context, such as for in-car scenarios? If so, how should we design in-car MR technology to foster passenger access and connectedness to both physical and virtual worlds, ensuring ride safety, comfort, and joy? To this aim, this thesis contributes via three aspects: 1) Understanding passenger use of in-car MR —first, I present a model for in-car MR interaction through user research. As interviews with daily commuters reveal, passengers are concerned with their physical integrity when facing spatial conflicts between borderless virtual environments and the confined cabin space. From this, the model aims to help researchers spatially organize information and how user interfaces vary in the proximity of the user. Additionally, a field experiment reveals contextual feedback about motion sickness when using immersive technology on the road. This helps refine the model and instruct the following experiments. 2) Mixing realities in car rides —second, this thesis explores a series of prototypes and experiments to examine how in-car MR technology can enable passengers to feel present in virtual environments while maintaining awareness of the real environment. The results demonstrate technical solutions for physical integrity and situational awareness by incorporating essential elements of the RE into virtual reality. Empirical evidence provides a set of dimensions into the in-car MR model, guiding the design decisions of mixing realities. 3) Transcending the transport context —third, I extend the model to other everyday contexts beyond transport that share spatial and social constraints, such as the confined and shared living space at home. A literature review consolidates leveraging daily physical objects as haptic feedback for MR interaction across spatial scales. A laboratory experiment discovers how context-aware MR systems that consider physical configurations can support social interaction with copresent others in close shared spaces. These results substantiate the scalability of the in-car MR model to other contexts. Finally, I conclude with a holistic model for mobile MR interaction across everyday contexts, from home to on the road. With my user research, prototypes, empirical evaluation, and model, this thesis paves the way for understanding the future passenger use of immersive technology, addressing today’s technical limitations of MR in mobile interaction, and ultimately fostering mobile users’ ubiquitous access and close connectedness to MR anytime and anywhere in their daily lives.Im modernen Leben verbringen die Menschen einen beträchtlichen Teil ihrer Zeit mit dem täglichen Pendeln. Die Menschen versuchen, die Reisezeit für ihre Arbeit und ihr Wohlbefinden durch die Interaktion mit mobilen und multimedialen Anwendungen auf persönlichen Geräten wie Smartphones und Tablets zu nutzen. Doch für neue Computing-Paradigmen, wie der mobilen Mixed Reality (MR), bleibt ihre Nützlichkeit in diesem alltäglichen Verkehrskontext, der MR im Auto, eine Herausforderung. Wenn künftige Passagiere in dreidimensionale virtuelle Umgebungen eintauchen, werden sie zunehmend von der Kabine, der Fahrzeugbewegung und den Menschen in ihrer Umgebung abgekoppelt. Diese verminderte Wahrnehmung der realen Umgebung gefährdet das Fahrverhalten der Passagiere im Straßenverkehr, was diese Arbeit zunächst zu der Frage motiviert: Können immersive Systeme im alltäglichen Verkehrskontext, z.B. in Fahrzeugszenarien, nützlich werden? Wenn ja, wie sollten wir die MR-Technologie im Auto gestalten, um den Zugang und die Verbindung der Passagiere mit der physischen und der virtuellen Welt zu fördern und dabei Sicherheit, Komfort und Freude an der Fahrt zu gewährleisten? Zu diesem Zweck trägt diese Arbeit zu drei Aspekten bei: 1) Verständnis der Nutzung von MR im Auto durch die Passagiere - Zunächst wird ein Modell für die MR-Interaktion im Auto durch user research vorgestellt. Wie aus Interviews mit täglichen Pendlern hervorgeht, sind die Passagiere um ihre körperliche Unversehrtheit besorgt, wenn sie mit räumlichen Konflikten zwischen grenzenlosen virtuellen Umgebungen und dem begrenzten Kabinenraum konfrontiert werden. Das Modell soll Forschern dabei helfen, Informationen und Benutzerschnittstellen räumlich zu organisieren, die in der Nähe des Benutzers variieren. Darüber hinaus zeigt ein Feldexperiment kontextbezogenes Feedback zur Reisekrankheit bei der Nutzung immersiver Technologien auf der Straße. Dies hilft, das Modell zu verfeinern und die folgenden Experimente zu instruieren. 2) Vermischung von Realitäten bei Autofahrten - Zweitens wird in dieser Arbeit anhand einer Reihe von Prototypen und Experimenten untersucht, wie die MR-Technologie im Auto es den Passagieren ermöglichen kann, sich in virtuellen Umgebungen präsent zu fühlen und gleichzeitig das Bewusstsein für die reale Umgebung zu behalten. Die Ergebnisse zeigen technische Lösungen für räumliche Beschränkungen und Situationsbewusstsein, indem wesentliche Elemente der realen Umgebung in VR integriert werden. Die empirischen Erkenntnisse bringen eine Reihe von Dimensionen in das Modell der MR im Auto ein, die die Designentscheidungen für gemischte Realitäten leiten. 3) Über den Verkehrskontext hinaus - Drittens erweitere ich das Modell auf andere Alltagskontexte jenseits des Verkehrs, in denen räumliche und soziale Zwänge herrschen, wie z.B. in einem begrenzten und gemeinsam genutzten Wohnbereich zu Hause. Eine Literaturrecherche konsolidiert die Nutzung von Alltagsgegenständen als haptisches Feedback für MR-Interaktion über räumliche Skalen hinweg. Ein Laborexperiment zeigt, wie kontextbewusste MR-Systeme, die physische Konfigurationen berücksichtigen, soziale Interaktion mit anderen Personen in engen gemeinsamen Räumen ermöglichen. Diese Ergebnisse belegen die Übertragbarkeit des MR-Modells im Auto auf andere Kontexte. Schließlich schließe ich mit einem ganzheitlichen Modell für mobile MR-Interaktion in alltäglichen Kontexten, von zu Hause bis unterwegs. Mit meiner user research, meinen Prototypen und Evaluierungsexperimenten sowie meinem Modell ebnet diese Dissertation den Weg für das Verständnis der zukünftigen Nutzung immersiver Technologien durch Passagiere, für die Überwindung der heutigen technischen Beschränkungen von MR in der mobilen Interaktion und schließlich für die Förderung des allgegenwärtigen Zugangs und der engen Verbindung der mobilen Nutzer zu MR jederzeit und überall in ihrem täglichen Leben

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments