Interactive Sound Propagation for Massive Multi-user and Dynamic Virtual Environments

Hearing is an important sense and it is known that rendering sound effects can enhance the level of immersion in virtual environments. Modeling sound waves is a complex problem, requiring vast computing resources to solve accurately. Prior methods are restricted to static scenes or limited acoustic effects. In this thesis, we present methods to improve the quality and performance of interactive geometric sound propagation in dynamic scenes and precomputation algorithms for acoustic propagation in enormous multi-user virtual environments. We present a method for finding edge diffraction propagation paths on arbitrary 3D scenes for dynamic sources and receivers. Using this algorithm, we present a unified framework for interactive simulation of specular reflections, diffuse reflections, diffraction scattering, and reverberation effects. We also define a guidance algorithm for ray tracing that responds to dynamic environments and reorders queries to minimize simulation time. Our approach works well on modern GPUs and can achieve more than an order of magnitude performance improvement over prior methods. Modern multi-user virtual environments support many types of client devices, and current phones and mobile devices may lack the resources to run acoustic simulations. To provide such devices the benefits of sound simulation, we have developed a precomputation algorithm that efficiently computes and stores acoustic data on a server in the cloud. Using novel algorithms, the server can render enhanced spatial audio in scenes spanning several square kilometers for hundreds of clients in realtime. Our method provides the benefits of immersive audio to collaborative telephony, video games, and multi-user virtual environments.Doctor of Philosoph

Efficient geometric sound propagation using visibility culling

Simulating propagation of sound can improve the sense of realism in interactive applications such as video games and can lead to better designs in engineering applications such as architectural acoustics. In this thesis, we present geometric sound propagation techniques which are faster than prior methods and map well to upcoming parallel multi-core CPUs. We model specular reflections by using the image-source method and model finite-edge diffraction by using the well-known Biot-Tolstoy-Medwin (BTM) model. We accelerate the computation of specular reflections by applying novel visibility algorithms, FastV and AD-Frustum, which compute visibility from a point. We accelerate finite-edge diffraction modeling by applying a novel visibility algorithm which computes visibility from a region. Our visibility algorithms are based on frustum tracing and exploit recent advances in fast ray-hierarchy intersections, data-parallel computations, and scalable, multi-core algorithms. The AD-Frustum algorithm adapts its computation to the scene complexity and allows small errors in computing specular reflection paths for higher computational efficiency. FastV and our visibility algorithm from a region are general, object-space, conservative visibility algorithms that together significantly reduce the number of image sources compared to other techniques while preserving the same accuracy. Our geometric propagation algorithms are an order of magnitude faster than prior approaches for modeling specular reflections and two to ten times faster for modeling finite-edge diffraction. Our algorithms are interactive, scale almost linearly on multi-core CPUs, and can handle large, complex, and dynamic scenes. We also compare the accuracy of our sound propagation algorithms with other methods. Once sound propagation is performed, it is desirable to listen to the propagated sound in interactive and engineering applications. We can generate smooth, artifact-free output audio signals by applying efficient audio-processing algorithms. We also present the first efficient audio-processing algorithm for scenarios with simultaneously moving source and moving receiver (MS-MR) which incurs less than 25% overhead compared to static source and moving receiver (SS-MR) or moving source and static receiver (MS-SR) scenario

AZB Rectangle Shrinkage Method and Heterogeneous Computing Accelerated Full Image Theory Method Ray Tracing Enabling Complex and Massive Outdoor 6G Propagation Modeling

Until now, despite their high accuracy, the utilization of the conventional image theory method ray tracers was limited to simple simulation environments with small number of field observation points and low maximum ray bouncing order due to their poor computational efficiency. This study presents a novel full-3D AZB rectangle shrinkage method and heterogeneous computing accelerated image theory method ray tracing framework for complex and massive outdoor propagation modeling. The proposed framework is divided into three parts: 1. Visibility preprocessing part. 2. Visibility tree generation part: in this part, a novel AZB rectangle shrinkage method that accelerates and reduces generation speed and size of visibility tree is proposed. 3. Shadow testing and field calculation part: in this part, a heterogeneous computing algorithm that can make possible to handle a large amount of field observation points is proposed. It is demonstrated that the proposed framework is faster more than 651 times than the image theory method solver of WinProp. Also, it is confirmed that the proposed ray tracing framework can handle 1km x 1km wide and dense urban outdoor simulation with up to the maximum ray bouncing order of 6 and thousands of field observation points. The proposed ray tracing framework would be a cornerstone of future image theory method ray tracing techniques for complex and massive scenarios that was exclusive to the shooting and bouncing rays method ray tracers

Interactive physically-based sound simulation

The realization of interactive, immersive virtual worlds requires the ability to present a realistic audio experience that convincingly compliments their visual rendering. Physical simulation is a natural way to achieve such realism, enabling deeply immersive virtual worlds. However, physically-based sound simulation is very computationally expensive owing to the high-frequency, transient oscillations underlying audible sounds. The increasing computational power of desktop computers has served to reduce the gap between required and available computation, and it has become possible to bridge this gap further by using a combination of algorithmic improvements that exploit the physical, as well as perceptual properties of audible sounds. My thesis is a step in this direction. My dissertation concentrates on developing real-time techniques for both sub-problems of sound simulation: synthesis and propagation. Sound synthesis is concerned with generating the sounds produced by objects due to elastic surface vibrations upon interaction with the environment, such as collisions. I present novel techniques that exploit human auditory perception to simulate scenes with hundreds of sounding objects undergoing impact and rolling in real time. Sound propagation is the complementary problem of modeling the high-order scattering and diffraction of sound in an environment as it travels from source to listener. I discuss my work on a novel numerical acoustic simulator (ARD) that is hundred times faster and consumes ten times less memory than a high-accuracy finite-difference technique, allowing acoustic simulations on previously intractable spaces, such as a cathedral, on a desktop computer. Lastly, I present my work on interactive sound propagation that leverages my ARD simulator to render the acoustics of arbitrary static scenes for multiple moving sources and listener in real time, while accounting for scene-dependent effects such as low-pass filtering and smooth attenuation behind obstructions, reverberation, scattering from complex geometry and sound focusing. This is enabled by a novel compact representation that takes a thousand times less memory than a direct scheme, thus reducing memory footprints to within available main memory. To the best of my knowledge, this is the only technique and system in existence to demonstrate auralization of physical wave-based effects in real-time on large, complex 3D scenes

Proceedings of the EAA Spatial Audio Signal Processing symposium: SASP 2019

International audienc

Interactive ray tracing of massive and deformable models

Ray tracing is a fundamental algorithm used for many applications such as computer graphics, geometric simulation, collision detection and line-of-sight computation. Even though the performance of ray tracing algorithms scales with the model complexity, the high memory requirements and the use of static hierarchical structures pose problems with massive models and dynamic data-sets. We present several approaches to address these problems based on new acceleration structures and traversal algorithms. We introduce a compact representation for storing the model and hierarchy while ray tracing triangle meshes that can reduce the memory footprint by up to 80%, while maintaining high performance. As a result, can ray trace massive models with hundreds of millions of triangles on workstations with a few gigabytes of memory. We also show how to use bounding volume hierarchies for ray tracing complex models with interactive performance. In order to handle dynamic scenes, we use refitting algorithms and also present highly-parallel GPU-based algorithms to reconstruct the hierarchies. In practice, our method can construct hierarchies for models with hundreds of thousands of triangles at interactive speeds. Finally, we demonstrate several applications that are enabled by these algorithms. Using deformable BVH and fast data parallel techniques, we introduce a geometric sound propagation algorithm that can run on complex deformable scenes interactively and orders of magnitude faster than comparable previous approaches. In addition, we also use these hierarchical algorithms for fast collision detection between deformable models and GPU rendering of shadows on massive models by employing our compact representations for hybrid ray tracing and rasterization

Higher-order Finite Difference Time Domain Algorithms for Room Acoustic Modelling

The acoustic qualities of indoor spaces are fundamental to the intelligibility of speech, the quality of musical performances, and perceived noise levels. Computationally heavy wave-based acoustic modelling algorithms have gained momentum in the field of room acoustic modelling, as ever-increasing computational power makes their use more feasible. Most notably the Finite Difference Time Domain (FDTD) method is often employed for rendering the low- and mid-frequency part of room impulse responses (RIRs). However, this algorithm has known disadvantages, most prominently dispersion error, which renders a large part of the simulated RIR invalid. This thesis is concerned with the implementation and analysis of higher-order FDTD stencils as a means to improve the current state-of-art FDTD methods that solve the room acoustic wave equation. A detailed analysis of dispersive properties, stability, and required grid spacing of current and higher-order stencils is presented, and has been verified using a GPU implementation of the different algorithms. It is argued that the 4th-order stencil gives the best result in terms of output quality versus computational effort. In addition, this thesis focusses on the derivation of absorbing boundaries for the 4th-order scheme, its stability analysis, and detailed analysis of absorptive properties compared to established boundary models for 2nd-order schemes. The newly proposed 4th-order scheme and its boundaries are tested in two case studies: a large shoebox model, in order to test the validity against a common benchmark and a complex acoustic space. For the latter study, impulse responses were measured in the National Centre for Early Music in York, UK, and computationally generated using the current state-of-the-art as well as the proposed 4th-order FDTD algorithm and boundaries. It is shown that the 4th-order stencil gives at least as good as, or better results than those achieved using the 2nd-order stencil, at lower computational costs