Hybrid visibility compositing and masking for illustrative rendering

In this paper, we introduce a novel framework for the compositing of interactively rendered 3D layers tailored to the needs of scientific illustration. Currently, traditional scientific illustrations are produced in a series of composition stages, combining different pictorial elements using 2D digital layering. Our approach extends the layer metaphor into 3D without giving up the advantages of 2D methods. The new compositing approach allows for effects such as selective transparency, occlusion overrides, and soft depth buffering. Furthermore, we show how common manipulation techniques such as masking can be integrated into this concept. These tools behave just like in 2D, but their influence extends beyond a single viewpoint. Since the presented approach makes no assumptions about the underlying rendering algorithms, layers can be generated based on polygonal geometry, volumetric data, point-based representations, or others. Our implementation exploits current graphics hardware and permits real-time interaction and rendering.publishedVersio

Enhanced perception in volume visualization

Due to the nature of scientic data sets, the generation of convenient visualizations may be a difficult task, but crucial to correctly convey the relevant information of the data. When working with complex volume models, such as the anatomical ones, it is important to provide accurate representations, since a misinterpretation can lead to serious mistakes while diagnosing a disease or planning surgery. In these cases, enhancing the perception of the features of interest usually helps to properly understand the data. Throughout years, researchers have focused on different methods to improve the visualization of volume data sets. For instance, the definition of good transfer functions is a key issue in Volume Visualization, since transfer functions determine how materials are classified. Other approaches are based on simulating realistic illumination models to enhance the spatial perception, or using illustrative effects to provide the level of abstraction needed to correctly interpret the data. This thesis contributes with new approaches to enhance the visual and spatial perception in Volume Visualization. Thanks to the new computing capabilities of modern graphics hardware, the proposed algorithms are capable of modifying the illumination model and simulating illustrative motifs in real time. In order to enhance local details, which are useful to better perceive the shape and the surfaces of the volume, our first contribution is an algorithm that employs a common sharpening operator to modify the lighting applied. As a result, the overall contrast of the visualization is enhanced by brightening the salient features and darkening the deeper regions of the volume model. The enhancement of depth perception in Direct Volume Rendering is also covered in the thesis. To do this, we propose two algorithms to simulate ambient occlusion: a screen-space technique based on using depth information to estimate the amount of light occluded, and a view-independent method that uses the density values of the data set to estimate the occlusion. Additionally, depth perception is also enhanced by adding halos around the structures of interest. Maximum Intensity Projection images provide a good understanding of the high intensity features of the data, but lack any contextual information. In order to enhance the depth perception in such a case, we present a novel technique based on changing how intensity is accumulated. Furthermore, the perception of the spatial arrangement of the displayed structures is also enhanced by adding certain colour cues. The last contribution is a new manipulation tool designed for adding contextual information when cutting the volume. Based on traditional illustrative effects, this method allows the user to directly extrude structures from the cross-section of the cut. As a result, the clipped structures are displayed at different heights, preserving the information needed to correctly perceive them.Debido a la naturaleza de los datos científicos, visualizarlos correctamente puede ser una tarea complicada, pero crucial para interpretarlos de forma adecuada. Cuando se trabaja con modelos de volumen complejos, como es el caso de los modelos anatómicos, es importante generar imágenes precisas, ya que una mala interpretación de las mismas puede producir errores graves en el diagnóstico de enfermedades o en la planificación de operaciones quirúrgicas. En estos casos, mejorar la percepción de las zonas de interés, facilita la comprensión de la información inherente a los datos. Durante décadas, los investigadores se han centrado en el desarrollo de técnicas para mejorar la visualización de datos volumétricos. Por ejemplo, los métodos que permiten definir buenas funciones de transferencia son clave, ya que éstas determinan cómo se clasifican los materiales. Otros ejemplos son las técnicas que simulan modelos de iluminación realista, que permiten percibir mejor la distribución espacial de los elementos del volumen, o bien los que imitan efectos ilustrativos, que proporcionan el nivel de abstracción necesario para interpretar correctamente los datos. El trabajo presentado en esta tesis se centra en mejorar la percepción de los elementos del volumen, ya sea modificando el modelo de iluminación aplicado en la visualización, o simulando efectos ilustrativos. Aprovechando la capacidad de cálculo de los nuevos procesadores gráficos, se describen un conjunto de algoritmos que permiten obtener los resultados en tiempo real. Para mejorar la percepción de detalles locales, proponemos modificar el modelo de iluminación utilizando una conocida herramienta de procesado de imágenes (unsharp masking). Iluminando aquellos detalles que sobresalen de las superficies y oscureciendo las zonas profundas, se mejora el contraste local de la imagen, con lo que se consigue realzar los detalles de superficie. También se presentan diferentes técnicas para mejorar la percepción de la profundidad en Direct Volume Rendering. Concretamente, se propone modificar la iluminación teniendo en cuenta la oclusión ambiente de dos maneras diferentes: la primera utiliza los valores de profundidad en espacio imagen para calcular el factor de oclusión del entorno de cada pixel, mientras que la segunda utiliza los valores de densidad del volumen para aproximar dicha oclusión en cada vóxel. Además de estas dos técnicas, también se propone mejorar la percepción espacial y de la profundidad de ciertas estructuras mediante la generación de halos. La técnica conocida como Maximum Intensity Projection (MIP) permite visualizar los elementos de mayor intensidad del volumen, pero no aporta ningún tipo de información contextual. Para mejorar la percepción de la profundidad, proponemos una nueva técnica basada en cambiar la forma en la que se acumula la intensidad en MIP. También se describe un esquema de color para mejorar la percepción espacial de los elementos visualizados. La última contribución de la tesis es una herramienta de manipulación directa de los datos, que permite preservar la información contextual cuando se realizan cortes en el modelo de volumen. Basada en técnicas ilustrativas tradicionales, esta técnica permite al usuario estirar las estructuras visibles en las secciones de los cortes. Como resultado, las estructuras de interés se visualizan a diferentes alturas sobre la sección, lo que permite al observador percibirlas correctamente

Identifying Problems Associated with Focus and Context Awareness in 3D Modelling Tasks

Creating complex 3D models is a challenging process. One of the main reasons for this is that 3D models are usually created using software developed for conventional 2D displays which lack true depth perspective, and therefore do not support correct perception of spatial placement and depth-ordering of displayed content. As a result, modellers often have to deal with many overlapping components of 3D models (e.g. vertices, edges, faces, etc.) on a 2D display surface. This in turn causes them to have difficulties in distinguishing distances, maintaining position and orientation awareness, etc. To better understand the nature of these problems, which can collectively be defined as ‘focus and context awareness’ problems, we have conducted a pilot study with a group of novice 3D modellers, and a series of interviews with a group of professional 3D modellers. This article presents these two studies, and their findings, which have resulted in identifying a set of focus and context awareness problems that modellers face in creating 3D models using conventional modelling software. The article also provides a review of potential solutions to these problems in the related literature

Dynamic worlds in miniature

The World in Miniature (WIM) metaphor allows users to interact and travel efficiently in virtual environments. In addition to the first-person perspective offered by typical VR applications, the WIM offers a second dynamic viewpoint through a hand-held miniature copy of the virtual environment. In the original WIM paper the miniature was a scaled down replica of the whole environment, thus limiting the technique to simple models being manipulated at a single level of scale. Several WIM extensions have been proposed where the replica shows only a part of the virtual environment. In this paper we present an improved visualization of WIM that supports arbitrarily-complex, densely-occluded scenes. In particular, we discuss algorithms for selecting the region of the virtual environment which will be covered by the miniature copy and efficient algorithms for handling 3D occlusion from an exocentric viewpoint.Peer ReviewedPostprint (author’s final draft

Task-based Adaptation of Graphical Content in Smart Visual Interfaces

To be effective visual representations must be adapted to their respective context of use, especially in so-called Smart Visual Interfaces striving to present specifically those information required for the task at hand. This thesis proposes a generic approach that facilitate the automatic generation of task-specific visual representations from suitable task descriptions. It is discussed how the approach is applied to four principal content types raster images, 2D vector and 3D graphics as well as data visualizations, and how existing display techniques can be integrated into the approach.Effektive visuelle Repräsentationen müssen an den jeweiligen Nutzungskontext angepasst sein, insbesondere in sog. Smart Visual Interfaces, welche anstreben, möglichst genau für die aktuelle Aufgabe benötigte Informationen anzubieten. Diese Arbeit entwirft einen generischen Ansatz zur automatischen Erzeugung aufgabenspezifischer Darstellungen anhand geeigneter Aufgabenbeschreibungen. Es wird gezeigt, wie dieser Ansatz auf vier grundlegende Inhaltstypen Rasterbilder, 2D-Vektor- und 3D-Grafik sowie Datenvisualisierungen anwendbar ist, und wie existierende Darstellungstechniken integrierbar sind

Physically Interacting With Four Dimensions

Thesis (Ph.D.) - Indiana University, Computer Sciences, 2009People have long been fascinated with understanding the fourth dimension. While making pictures of 4D objects by projecting them to 3D can help reveal basic geometric features, 3D graphics images by themselves are of limited value. For example, just as 2D shadows of 3D curves may have lines crossing one another in the shadow, 3D graphics projections of smooth 4D topological surfaces can be interrupted where one surface intersects another. The research presented here creates physically realistic models for simple interactions with objects and materials in a virtual 4D world. We provide methods for the construction, multimodal exploration, and interactive manipulation of a wide variety of 4D objects. One basic achievement of this research is to exploit the free motion of a computer-based haptic probe to support a continuous motion that follows the \emph{local continuity\/} of a 4D surface, allowing collision-free exploration in the 3D projection. In 3D, this interactive probe follows the full local continuity of the surface as though we were in fact \emph{physically touching\/} the actual static 4D object. Our next contribution is to support dynamic 4D objects that can move, deform, and collide with other objects as well as with themselves. By combining graphics, haptics, and collision-sensing physical modeling, we can thus enhance our 4D visualization experience. Since we cannot actually place interaction devices in 4D, we develop fluid methods for interacting with a 4D object in its 3D shadow image using adapted reduced-dimension 3D tools for manipulating objects embedded in 4D. By physically modeling the correct properties of 4D surfaces, their bending forces, and their collisions in the 3D interactive or haptic controller interface, we can support full-featured physical exploration of 4D mathematical objects in a manner that is otherwise far beyond the real-world experience accessible to human beings

Supporting Focus and Context Awareness in 3D Modeling Using Multi-Layered Displays

Although advances in computer technology over the past few decades have made it possible to create and render highly realistic 3D models these days, the process of creating these models has remained largely unchanged over the years. Modern 3D modeling software provide a range of tools to assist users with creating 3D models, but the process of creating models in virtual 3D space is nevertheless still challenging and cumbersome. This thesis, therefore, aims to investigate whether it is possible to support modelers more effectively by providing them with alternative combinations of hardware and software tools to improve their 3D modeling tasks. The first step towards achieving this goal has been to better understand the type of problems modelers face in using conventional 3D modeling software. To achieve this, a pilot study of novice 3D modelers, and a more comprehensive study of professional modelers were conducted. These studies resulted in identifying a range of focus and context awareness problems that modelers face in creating complex 3D models using conventional modeling software. These problems can be divided into four categories: maintaining position awareness, identifying and selecting objects or components of interest, recognizing the distance between objects or components, and realizing the relative position of objects or components. Based on the above categorization, five focus and context awareness techniques were developed for a multi-layer computer display to enable modelers to better maintain their focus and context awareness while performing 3D modeling tasks. These techniques are: object isolation, component segregation, peeling focus, slicing, and peeling focus and context. A user study was then conducted to compare the effectiveness of these focus and context awareness techniques with other tools provided by conventional 3D modeling software. The results of this study were used to further improve, and evaluate through a second study, the five focus and context awareness techniques. The two studies have demonstrated that some of these techniques are more effective in supporting 3D modeling tasks than other existing software tools

How Game Development Technology Can Assist in Visualization Challenges

Research has shown that product configuration is becoming an increasingly common aspect of commercial sales, where previously it would have been an advantage that sets your products apart from competitors, it’s now becoming normalised as a baseline service offered to customers. Product configurators are notoriously difficult to develop, often requiring mass amounts of content and data. Larger businesses with resources to spare can afford to take on the risks associated with developing a product configurator, but medium and smaller sized businesses often cannot, and so find themselves at an increasingly larger disadvantage. This study aims to investigate whether game technology can mitigate or remove some of the development risks associated with developing a product configurator. Building on existing work on product configurator development, it asks. Can game technology provide an advantage over traditional development solutions? And if so, how exactly does it improve on them Based on a review of the literature on product configurator development, it’s clear that traditional rendering solutions are fine if the products showcased feature low-levels of customisation. But if products exceed this customisation threshold, it’s evident that these practices cannot scale and so bottleneck production, making the investment and risk involved in continuing development too high. The results indicate that when using game technology, you can automate aspects of production, such as rendering and storage. The speed difference between traditional rendering solutions and game technology methods are considerable if implemented correctly. On this basis, it is recommended that game technology is used in place of traditional rendering solutions such as Blender, 3DS Max and other 3D rendering packages as the difference in production speed is immense and the inclusion of automating certain tasks outright removes certain tasks involved in the production of a product configurator

Flexible occlusion rendering for improved views of three-dimensional medical images

The goal of this work is to enable more rapid and accurate diagnosis of pathology from three-dimensional (3D) medical images by augmenting standard volume rendering techniques to display otherwise-occluded features within the volume. When displaying such data sets with volume rendering, appropriate selection of the transfer function is critical for determining which features of the data will be displayed. In many cases, however, no transfer function is able to produce the most useful views for diagnosis of pathology. Flexible Occlusion Rendering (FOR) is an addition to standard ray cast volume rendering that modulates accumulated color and opacity along each ray upon detecting features indicating the separation between objects of the same intensity range. For contrast-enhanced MRI and CT data, these separation features are intensity peaks. To detect these peaks, a dual-threshold method is used to reduce sensitivity to noise. To further reduce noise and enable control over the spatial scale of the features detected, a smoothed version of the original data set is used for feature detection, while rendering the original data at high resolution. Separating the occlusion feature detection from the volume rendering transfer function enables robust occlusion determination and seamless transition from occluded views to non-occluded views of surfaces during virtual fly-throughs. FOR has been applied to virtual arthroscopy of joints from MRI data. For example, survey views of entire shoulder socket surfaces have been rendered to enable rapid evaluation by automatically removing the occluding material of the humeral head. Such views are not possible with standard volume rendering. FOR has also been successfully applied to virtual ureteroscopy of the renal collecting system from CT data, and knee fracture visualization from CT data