Detection Thresholds in Audio-visual Redirected Walking

(Abstract to follow

Multimodality in VR: A survey

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer

Multimodality in {VR}: {A} Survey

Virtual reality has the potential to change the way we create and consume content in our everyday life. Entertainment, training, design and manufacturing, communication, or advertising are all applications that already benefit from this new medium reaching consumer level. VR is inherently different from traditional media: it offers a more immersive experience, and has the ability to elicit a sense of presence through the place and plausibility illusions. It also gives the user unprecedented capabilities to explore their environment, in contrast with traditional media. In VR, like in the real world, users integrate the multimodal sensory information they receive to create a unified perception of the virtual world. Therefore, the sensory cues that are available in a virtual environment can be leveraged to enhance the final experience. This may include increasing realism, or the sense of presence; predicting or guiding the attention of the user through the experience; or increasing their performance if the experience involves the completion of certain tasks. In this state-of-the-art report, we survey the body of work addressing multimodality in virtual reality, its role and benefits in the final user experience. The works here reviewed thus encompass several fields of research, including computer graphics, human computer interaction, or psychology and perception. Additionally, we give an overview of different applications that leverage multimodal input in areas such as medicine, training and education, or entertainment; we include works in which the integration of multiple sensory information yields significant improvements, demonstrating how multimodality can play a fundamental role in the way VR systems are designed, and VR experiences created and consumed

08231 Abstracts Collection -- Virtual Realities

From 1st to 6th June 2008, the Dagstuhl Seminar 08231 ``Virtual Realities\u27\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. Virtual Reality (VR) is a multidisciplinary area of research aimed at interactive human-computer mediated simulations of artificial environments. Typical applications include simulation, training, scientific visualization, and entertainment. An important aspect of VR-based systems is the stimulation of the human senses -- typically sight, sound, and touch -- such that a user feels a sense of presence (or immersion) in the virtual environment. Different applications require different levels of presence, with corresponding levels of realism, sensory immersion, and spatiotemporal interactive fidelity. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. Links to extended abstracts or full papers are provided, if available

Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

The relationships between the listener, physical world and virtual environment (VE) should not only inspire the design of natural multimodal interfaces but should be discovered to make sense of the mediating action of VR technologies. This chapter aims to transform an archipelago of studies related to sonic interactions in virtual environments (SIVE) into a research field equipped with a first theoretical framework with an inclusive vision of the challenges to come: the egocentric perspective of the auditory digital twin. In a VE with immersive audio technologies implemented, the role of VR simulations must be enacted by a participatory exploration of sense-making in a network of human and non-human agents, called actors. The guardian of such locus of agency is the auditory digital twin that fosters intra-actions between humans and technology, dynamically and fluidly redefining all those configurations that are crucial for an immersive and coherent experience. The idea of entanglement theory is here mainly declined in an egocentric-spatial perspective related to emerging knowledge of the listener's perceptual capabilities. This is an actively transformative relation with the digital twin potentials to create movement, transparency, and provocative activities in VEs. The chapter contains an original theoretical perspective complemented by several bibliographical references and links to the other book chapters that have contributed significantly to the proposal presented here.Comment: 46 pages, 5 figures. Pre-print version of the introduction to the book "Sonic Interactions in Virtual Environments" in press for Springer's Human-Computer Interaction Series, Open Access license. The pre-print editors' copy of the book can be found at https://vbn.aau.dk/en/publications/sonic-interactions-in-virtual-environments - full book info: https://sive.create.aau.dk/index.php/sivebook

An Approach to Reducing Distance Compression in Audiovisual Virtual Environments

Perception of distances in virtual reality (VR) is compressed: objects are consistently perceived as closer than intended. Although this phenomenon has been well documented, it is still not fully understood or defined with respect to the factors influencing such compression. This is a problem in scenarios where veridical perception of distance and scale is essential. We report the results of an experiment investigating an approach to reducing distance compression in audiovisual VR based on a predictive model of distance perception. Our test environment involved photorealistic 3D images captured through stereo photography, with corresponding spatial audio rendered binaurally over headphones. In a perceptual matching task, participants positioned an auditory stimulus with respect to the corresponding visual stimulus. We found a high correlation between the distance perception predicted by our model and how participants perceived the distance. Through automated manipulation of the audio and visual displays based on the model, our approach can be used to reposition auditory and visual components of a scene to reduce distance compression. The approach is adaptable to different environments and agnostic of scene content, and can be calibrated to individual observers

Distributed Fiber Ultrasonic Sensor and Pattern Recognition Analytics

Ultrasound interrogation and structural health monitoring technologies have found a wide array of applications in the health care, aerospace, automobile, and energy sectors. To achieve high spatial resolution, large array electrical transducers have been used in these applications to harness sufficient data for both monitoring and diagnoses. Electronic-based sensors have been the standard technology for ultrasonic detection, which are often expensive and cumbersome for use in large scale deployments. Fiber optical sensors have advantageous characteristics of smaller cross-sectional area, humidity-resistance, immunity to electromagnetic interference, as well as compatibility with telemetry and telecommunications applications, which make them attractive alternatives for use as ultrasonic sensors. A unique trait of fiber sensors is its ability to perform distributed acoustic measurements to achieve high spatial resolution detection using a single fiber. Using ultrafast laser direct-writing techniques, nano-reflectors can be induced inside fiber cores to drastically improve the signal-to-noise ratio of distributed fiber sensors. This dissertation explores the applications of laser-fabricated nano-reflectors in optical fiber cores for both multi-point intrinsic Fabry–Perot (FP) interferometer sensors and a distributed phase-sensitive optical time-domain reflectometry (φ-OTDR) to be used in ultrasound detection. Multi-point intrinsic FP interferometer was based on swept-frequency interferometry with optoelectronic phase-locked loop that interrogated cascaded FP cavities to obtain ultrasound patterns. The ultrasound was demodulated through reassigned short time Fourier transform incorporating with maximum-energy ridges tracking. With tens of centimeters cavity length, this approach achieved 20kHz ultrasound detection that was finesse-insensitive, noise-free, high-sensitivity and multiplex-scalability. The use of φ-OTDR with enhanced Rayleigh backscattering compensated the deficiencies of low inherent signal-to-noise ratio (SNR). The dynamic strain between two adjacent nano-reflectors was extracted by using 3×3 coupler demodulation within Michelson interferometer. With an improvement of over 35 dB SNR, this was adequate for the recognition of the subtle differences in signals, such as footstep of human locomotion and abnormal acoustic echoes from pipeline corrosion. With the help of artificial intelligence in pattern recognition, high accuracy of events’ identification can be achieved in perimeter security and structural health monitoring, with further potential that can be harnessed using unsurprised learning

World Settings

Acoustical building materials, with their ability to absorb and diffuse sound, can reshape the character of interior spaces in profound ways. Woven textiles often perform as acoustical materials, whether by coincidence or by design; strategic use of textile structure and dimensionality can yield specific experiential qualities in homes, offices and shared spaces. The way certain materials manipulate sound can feel otherworldly, as if they break the laws of physics or the familiar parameters of one’s surroundings. The same properties can be found in emergent visual patterns and illusory lighting conditions, which provoke an investigative, deliberate way of looking. In this thesis, I explore the history of architectural acoustics and the meaning of noise as a sonic, conceptual and technical term. Visual metaphors of windows and screens, digital and analog noise and perceptual phenomena shape this work, while the “aliveness” of self-organizing materials provides a rationale for new variations on weaving techniques. The result is a collection of interior fabrics that aim to modify room environments acoustically and visually, suggesting that the static “settings” of such places have shifted. I argue that this sense of unfamiliarity can be fruitful, prompting the viewer to spend time in a focused, exploratory state and become aware of the cognitive processes by which they make sense of the physical world