Design and development of a VR system for exploration of medical data using haptic rendering and high quality visualization

[no abstract

Towards Predictive Rendering in Virtual Reality

The strive for generating predictive images, i.e., images representing radiometrically correct renditions of reality, has been a longstanding problem in computer graphics. The exactness of such images is extremely important for Virtual Reality applications like Virtual Prototyping, where users need to make decisions impacting large investments based on the simulated images. Unfortunately, generation of predictive imagery is still an unsolved problem due to manifold reasons, especially if real-time restrictions apply. First, existing scenes used for rendering are not modeled accurately enough to create predictive images. Second, even with huge computational efforts existing rendering algorithms are not able to produce radiometrically correct images. Third, current display devices need to convert rendered images into some low-dimensional color space, which prohibits display of radiometrically correct images. Overcoming these limitations is the focus of current state-of-the-art research. This thesis also contributes to this task. First, it briefly introduces the necessary background and identifies the steps required for real-time predictive image generation. Then, existing techniques targeting these steps are presented and their limitations are pointed out. To solve some of the remaining problems, novel techniques are proposed. They cover various steps in the predictive image generation process, ranging from accurate scene modeling over efficient data representation to high-quality, real-time rendering. A special focus of this thesis lays on real-time generation of predictive images using bidirectional texture functions (BTFs), i.e., very accurate representations for spatially varying surface materials. The techniques proposed by this thesis enable efficient handling of BTFs by compressing the huge amount of data contained in this material representation, applying them to geometric surfaces using texture and BTF synthesis techniques, and rendering BTF covered objects in real-time. Further approaches proposed in this thesis target inclusion of real-time global illumination effects or more efficient rendering using novel level-of-detail representations for geometric objects. Finally, this thesis assesses the rendering quality achievable with BTF materials, indicating a significant increase in realism but also confirming the remainder of problems to be solved to achieve truly predictive image generation

Shallow waters simulation

Dissertação de mestrado integrado em Informatics EngineeringRealistic simulation and rendering of water in real-time is a challenge within the field of computer graphics, as it is very computationally demanding. A common simulation approach is to reduce the problem from 3D to 2D by treating the water surface as a 2D heightfield. When simulating 2D fluids, the Shallow Water Equations (SWE) are often employed, which work under the assumption that the water’s horizontal scale is much greater than it’s vertical scale. There are several methods that have been developed or adapted to model the SWE, each with its own advantages and disadvantages. A common solution is to use grid-based methods where there is the classic approach of solving the equations in a grid, but also the Lattice-Boltzmann Method (LBM) which originated from the field of statistical physics. Particle based methods have also been used for modeling the SWE, namely as a variation of the popular Smoothed-Particle Hydrodynamics (SPH) method. This thesis presents an implementation for real-time simulation and rendering of a heightfield surface water volume. The water’s behavior is modeled by a grid-based SWE scheme with an efficient single kernel compute shader implementation. When it comes to visualizing the water volume created by the simulation, there are a variety of effects that can contribute to its realism and provide visual cues for its motion. In particular, When considering shallow water, there are certain features that can be highlighted, such as the refraction of the ground below and corresponding light attenuation, and the caustics patterns projected on it. Using the state produced by the simulation, a water surface mesh is rendered, where set of visual effects are explored. First, the water’s color is defined as a combination of reflected and transmitted light, while using a Cook- Torrance Bidirectional Reflectance Distribution Function (BRDF) to describe the Sun’s reflection. These results are then enhanced by data from a separate pass which provides caustics patterns and improved attenuation computations. Lastly, small-scale details are added to the surface by applying a normal map generated using noise. As part of the work, a thorough evaluation of the developed application is performed, providing a showcase of the results, insight into some of the parameters and options, and performance benchmarks.Simulação e renderização realista de água em tempo real é um desafio dentro do campo de computação gráfica, visto que é muito computacionalmente exigente. Uma abordagem comum de simulação é de reduzir o problema de 3D para 2D ao tratar a superfície da água como um campo de alturas 2D. Ao simular fluidos em 2D, é frequente usar as equações de águas rasas, que funcionam sobre o pressuposto de que a escala horizontal da água é muito maior que a sua escala vertical. Há vários métodos que foram desenvolvidos ou adaptados para modelar as equações de águas rasas, cada uma com as suas vantagens e desvantagens. Uma solução comum é utilizar métodos baseados em grelhas onde existe a abordagem clássica de resolver as equações numa grelha, mas também existe o método de Lattice Boltzmann que originou do campo de física estatística. Métodos baseados em partículas também já foram usados para modelar as equações de águas rasas, nomeadamente como uma variação do popular método de SPH. Esta tese apresenta uma implementação para simulação e renderização em tempo real de um volume de água com uma superfície de campo de alturas. O comportamento da água é modelado por um esquema de equações de águas rasas baseado na grelha com uma implementação eficiente de um único kernel de compute shader. No que toca a visualizar o volume de água criado pela simulação, existe uma variedade de efeitos que podem contribuir para o seu realismo e fornecer dicas visuais sobre o seu movimento. Ao considerar águas rasas, existem certas características que podem ser destacadas, como a refração do terreno por baixo e correspondente atenuação da luz, e padrões de cáusticas projetados nele. Usando o estado produzido pela simulação, uma malha da superfície da água é renderizada, onde um conjunto de efeitos visuais são explorados. Em primeiro lugar, a cor da água é definida como uma combinação de luz refletida e transmitida, sendo que uma BRDF de Cook-Torrance é usada para descrever a reflexão do Sol. Estes resultados são depois complementados com dados gerados num passo separado que fornece padrões de cáusticas e melhora as computações de atenuação. Por fim, detalhes de pequena escala são adicionados à superfície ao aplicar um mapa de normais gerado com ruído. Como parte do trabalho desenvolvido, é feita uma avaliação detalhada da aplicação desenvolvida, onde é apresentada uma demonstração dos resultados, comentários sobre alguns dos parâmetros e opções, e referências de desempenho

The application of three-dimensional mass-spring structures in the real-time simulation of sheet materials for computer generated imagery

Despite the resources devoted to computer graphics technology over the last 40 years, there is still a need to increase the realism with which flexible materials are simulated. However, to date reported methods are restricted in their application by their use of two-dimensional structures and implicit integration methods that lend themselves to modelling cloth-like sheets but not stiffer, thicker materials in which bending moments play a significant role. This thesis presents a real-time, computationally efficient environment for simulations of sheet materials. The approach described differs from other techniques principally through its novel use of multilayer sheet structures. In addition to more accurately modelling bending moment effects, it also allows the effects of increased temperature within the environment to be simulated. Limitations of this approach include the increased difficulties of calibrating a realistic and stable simulation compared to implicit based methods. A series of experiments are conducted to establish the effectiveness of the technique, evaluating the suitability of different integration methods, sheet structures, and simulation parameters, before conducting a Human Computer Interaction (HCI) based evaluation to establish the effectiveness with which the technique can produce credible simulations. These results are also compared against a system that utilises an established method for sheet simulation and a hybrid solution that combines the use of 3D (i.e. multilayer) lattice structures with the recognised sheet simulation approach. The results suggest that the use of a three-dimensional structure does provide a level of enhanced realism when simulating stiff laminar materials although the best overall results were achieved through the use of the hybrid model

Adaptive Modeling of Details for Physically-based Sound Synthesis and Propagation

In order to create an immersive virtual world, it is crucial to incorporate a realistic aural experience that complements the visual sense. Physically-based sound simulation is a method to achieve this goal and automatically provides audio-visual correspondence. It simulates the physical process of sound: the pressure variations of a medium originated from some vibrating surface (sound synthesis), propagating as waves in space and reaching human ears (sound propagation). The perceived realism of simulated sounds depends on the accuracy of the computation methods and the computational resource available, and oftentimes it is not feasible to use the most accurate technique for all simulation targets. I propose techniques that model the general sense of sounds and their details separately and adaptively to balance the realism and computational costs of sound simulations. For synthesizing liquid sounds, I present a novel approach that generate sounds due to the vibration of resonating bubbles. My approach uses three levels of bubble modeling to control the trade-offs between quality and efficiency: statistical generation from liquid surface configuration,explicitly tracking of spherical bubbles, and decomposition of non-spherical bubbles to spherical harmonics. For synthesizing rigid-body contact sounds, I propose to improve the realism in two levels using example recordings: first, material parameters that preserve the inherent quality of the recorded material are estimated; then extra details from the example recording that are not fully captured by the material parameters are computed and added. For simulating sound propagation in large, complex scenes, I present a novel hybrid approach that couples numerical and geometric acoustic techniques. By decomposing the spatial domain of a scene and applying the more accurate and expensive numerical acoustic techniques only in limited regions, a user is able to allocate computation resources on where it matters most.Doctor of Philosoph

Algorithms and data structures for interactive ray tracing on commodity hardware

Rendering methods based on ray tracing provide high image realism, but have been historically regarded as offline only. This has changed in the past decade, due to significant advances in the construction and traversal performance of acceleration structures and the efficient use of data-parallel processing. Today, all major graphics companies offer real-time ray tracing solutions. The following work has contributed to this development with some key insights. We first address the limited support of dynamic scenes in previous work, by proposing two new parallel-friendly construction algorithms for KD-trees and BVHs. By approximating the cost function, we accelerate construction by up to an order of magnitude (especially for BVHs), at the expense of only tiny degradation to traversal performance. For the static portions of the scene, we also address the topic of creating the “perfect” acceleration structure. We develop a polynomial time non-greedy BVH construction algorithm. We then modify it to produce a new type of acceleration structure that inherits both the high performance of KD-trees and the small size of BVHs. Finally, we focus on bringing real-time ray tracing to commodity desktop computers. We develop several new KD-tree and BVH traversal algorithms specifically tailored for the GPU. With them, we show for the first time that GPU ray tracing is indeed feasible, and it can outperform CPU ray tracing by almost an order of magnitude, even on large CAD models.Ray-Tracing basierte Bildsynthese-Verfahren bieten einen hohen Grad an Realismus, wurden allerdings in der Vergangenheit ausschließlich als nicht echtzeitfähig betrachtet. Dies hat sich innerhalb des letzten Jahrzehnts geändert durch signifikante Fortschritte sowohl im Bereich der Erstellung und Traversierung von Beschleunigungs-Strukturen, als auch im effizienten Einsatz paralleler Berechnung. Heute bieten alle großen Grafik-Firmen Echtzeit-Ray-Tracing Lösungen an. Die vorliegende Dissertation behandelt Beträge zu dieser Entwicklung in mehreren Kernaspekten. Der erste Teil beschäftigt sich mit der eingeschränkten Unterstützung von dynamischen Szenen in bisherigen Verfahren. Hierbei behandeln wir zwei zur Parallelisierung geeignete Algorithmen zur Erstellung von KD-Bäumen und Bounding-Volume-Hierarchien. Durch Approximation von Kosten-Funktionen kann eine Verbesserung der Konstruktionszeit von bis zu einer Größenordnung erreicht werden (speziell für BVH-Strukturen), bei nur geringem Verlust von Traversierungs-Effizienz. Mit Blick auf den statischen Teil einer Szene beschäftigen wir uns mit der Erstellung “perfekter” Beschleunigungs-Strukturen. Wir entwickeln einen Algorithmus zur BVH-Erstellung, der ein globales Optimum in polynomialer Zeit liefert. Dies führt zu einer neuartigen Beschleunigungs-Struktur, welche sowohl die hohe Leistung von KD-Bäumen, als auch den geringen Platzbedarf von BVH-Strukturen in sich vereinigt. Abschließend betrachten wir Echtzeit-Ray-Tracing auf Desktop-Computern. Wir entwickeln neuartige KD-Baum- und BVH-Traversierungs-Algorithmen, die speziell auf den Einsatz von Grafikprozessoren zugeschnitten sind. Wir zeigen damit zum ersten Mal, dass GPU-Ray-Tracing nicht nur praktikabel ist, sondern auch mehr als eine Größenordnung effizienter sein kann als CPU basierte Ray-Tracing-Verfahren, selbst bei der Darstellung großer CAD Modelle