The virtual magic lantern: an interaction metaphor for enhanced medical data inspection

In this paper we present the Virtual Magic Lantern (VML), an interaction tool tailored to facilitate volumetric data inspection. It behaves like a lantern whose virtual illumination cone provides the focal region which is visualized using a secondary transfer function or different rendering style. This may be used for simple visual inspection, surgery planning, or injure diagnosis. The VML is a particularly friendly and intuitive interaction tool suitable for an immersive Virtual Reality setup with a large screen, where the user moves a Wanda device, like a lantern pointing to the model. We show that this inspection metaphor can be efficiently and easily adapted to a GPU ray casting volume visualization algorithm. We also present the Virtual Magic Window (VMW) metaphor as an efficient collateral implementation of the VML, that can be seen as a restricted case where the lantern illuminates following the viewing direction, through a virtual window created as the intersection of the virtual lantern (guided by the Wanda device) and the bounding box of the volume.Peer ReviewedPostprint (author’s final draft

Friction surfaces: scaled ray-casting manipulation for interacting with 2D GUIs

The accommodation of conventional 2D GUIs with Virtual Environments (VEs) can greatly enhance the possibilities of many VE applications. In this paper we present a variation of the well-known ray-casting technique for fast and accurate selection of 2D widgets over a virtual window immersed into a 3D world. The main idea is to provide a new interaction mode where hand rotations are scaled down so that the ray is constrained to intersect the active virtual window. This is accomplished by changing the control-display ratio between the orientation of the user’s hand and the ray used for selection. Our technique uses a curved representation of the ray providing visual feedback of the orientation of both the input device and the selection ray. The users’ feeling is that they control a flexible ray that gets curved as it moves over a virtual friction surface defined by the 2D window. We have implemented this technique and evaluated its effectiveness in terms of accuracy and performance. Our experiments on a four-sided CAVE indicate that the proposed technique can increase the speed and accuracy of component selection in 2D GUIs immersed into 3D worlds.Peer ReviewedPostprint (author’s final draft

Motor de eventos

Dissertação de mestrado em Computação Gráfica e Ambientes Virtuais (área de especialização em Informática)“A Universidade do Minho, em particular o grupo de Gráficos, Interacção e Visão (GIV), em parceria com o Município de Ponte de Lima, desenvolveu um trabalho no âmbito do projecto ‘Ponte de Lima, Terra Rica da Humanidade’, que consistiu na modelação 3D, e apresentação sobre diversos formatos, da vila de Ponte de Lima na sua versão actual no início do século XXI, e numa possível reconstrução virtual da muralha construída no século XIV. De um ponto de vista académico e no âmbito da computação gráfica, este projecto permitiu ao grupo GIV atingir dois objectivos: a construção de um cenário virtual de grandes dimensões e um motor de renderização 3D completo e extremamente versátil.” [pl3D] A transcrição anterior refere-se ao motor gráfico de renderização 3D denominado Curitiba. Este permite visualizar modelos 3D, com capacidade para realizar múltiplos passos, possibilitando a geração de efeitos visuais complexos. Embora a demonstração do motor, no âmbito do projecto Ponte de Lima 3D, permitisse algumas animações e interactividade, estas foram de facto criadas através da alteração do próprio código, sem que houvesse na altura muito cuidado na elaboração de uma arquitectura para criar cenas dinâmicas. A interacção com alguns elementos no modelo, como por exemplo, clicando na marca que indica a subida do rio numa das grandes cheias ocorridas, provoca uma animação. Essa animação foi realizada de forma muito direccionada para o efeito desejado nesse momento, não permitindo generalizar de forma sistemática a criação de outras situações com animações diferentes, ou, mantendo animação, aplicá-la a outros modelos. Apesar de ter acoplado um motor de física e estar dotado de capacidades para facilmente ser expansível, o Curitiba permitia apenas criar um ambiente 3D estático sem interacção, a menos da movimentação da câmara no modo de primeira pessoa. Neste contexto pretende-se dotar o Curitiba da capacidade para definir e visualizar cenas dinâmicas, e com interacção, quer com o utilizador quer entre elementos da cena. É nesta perspectiva, a de acrescentar funcionalidade e interacção ao ambiente, que é proposto realizar nesta dissertação de mestrado, um estudo e implementação de uma arquitectura de software para um motor de eventos. Um motor de eventos que permita acrescentar diferentes tipos de interacção num ambiente 3D, que seja fácil de usar, genérico, e expansível, seguindo a própria filosofia do Curitiba.“Minho University, in particular the Graphics, Interaction and Vision team (GIV), in association with the council of Ponte de Lima has developed a project called ‘Ponte de Lima, a Land Rich in Humanity’, that consisted of the 3D modeling and presentation, in various formats, of the town of Ponte de Lima in its present version of the beginning of the XXIst century and in a virtual prospective reconstruction with the wall built in the XIVth century. From an academic point of view, and in the graphic computation field, this project allowed the GIV team to achieve two purposes: the construction of a large virtual scenario and a complete 3D rendering engine that is extremely versatile.” [pl3D] The previous quotation refers to a 3D rendering graphic engine called Curitiba. This allows 3D model visualization with multiple steps which make complex visual effects possible. Although the engine demonstration in the “Ponte de Lima 3D” project allowed some animation and interactivity, in fact they were achieved with the change of the engine’s own code, without taking into consideration at the time, the building of a dynamic architecture to create dynamic scenes. The interaction of some elements in the model, for instance, by clicking on a mark which represents the flooding of the river, causes an animation. That animation was achieved based on the wanted effect at the time, but it can’t be applied systematically when creating other situations with different animations or, when using that very animation on other models. Despite having engaged a physics engine and being able to be easily expandable, Curitiba, at the present moment, creates a static 3D ambience without interaction, besides the movement of the camera in first person mode. The aim of this work is to provide Curitiba with the ability to define and visualize dynamic scenes and with interaction among both the user and elements of the scene. To accomplish this goal, to add functionality and interaction to 3D environment, we propose to accomplish in this master work a software architecture study and implementation for an event engine. An event engine that permits expansion of different interaction types in 3D environment that is easy to use, generic and expandable, following the very philosophy of Curitiba

Manipulació i interacció en entorns de realitat virtual d'una aplicació mèdica

L'objectiu principal del projecte consisteix en migrar una aplicació de sobretaula que permet visualitzar reconstruccions tridimensionals d’estructures anatòmiques a un entorn de realitat virtual. S'implementaran i avaluaran diferents mètodes de manipulació i interacció que siguin intuïtius i fàcils d'usar en entorns de realitat virtual; també s’estudiarà la viabilitat i usabilitat de l’aplicació per a ser utilitzada en entorns mèdics

A cost-effective approach for developing application-control GUIs for virtual environments

In this paper we present a new approach for fast development of application-control User Interfaces (UIs) for Virtual Environments (VEs). Our approach allows developers to build sophisticated UIs containing both simple widgets (such as windows, buttons, menus and sliders) and advanced widgets (such as hierarchical views and web browsers) with minimum effort. Rather than providing a new API for defining and managing the interface components, we propose to extend current 2D toolkits such as Qt so that its full range of widgets can be displayed and manipulated either as 2D shapes on the desktop or as textured 3D objects within the virtual world. This approach allows 3D UI developers to take advantage of the increasing number of components, layout managers and graphical design tools provided by 2D UI toolkits. Resulting programs can run on platforms ranging from fully immersive systems to generic desktop workstations with little or no modification. The design of the system and the key features required on the host UI toolkit are presented and discussed. A prototype system has been implemented above Qt and evaluated on a 4-sided CAVE. The results indicate that this approach provides an efficient and cost-effective way for porting and developing application-control GUIs on VEs and thus it can greatly enhance the possibilities of many VE applications.Peer Reviewe

Enhanced Virtuality: Increasing the Usability and Productivity of Virtual Environments

Mit stetig steigender Bildschirmauflösung, genauerem Tracking und fallenden Preisen stehen Virtual Reality (VR) Systeme kurz davor sich erfolgreich am Markt zu etablieren. Verschiedene Werkzeuge helfen Entwicklern bei der Erstellung komplexer Interaktionen mit mehreren Benutzern innerhalb adaptiver virtueller Umgebungen. Allerdings entstehen mit der Verbreitung der VR-Systeme auch zusätzliche Herausforderungen: Diverse Eingabegeräte mit ungewohnten Formen und Tastenlayouts verhindern eine intuitive Interaktion. Darüber hinaus zwingt der eingeschränkte Funktionsumfang bestehender Software die Nutzer dazu, auf herkömmliche PC- oder Touch-basierte Systeme zurückzugreifen. Außerdem birgt die Zusammenarbeit mit anderen Anwendern am gleichen Standort Herausforderungen hinsichtlich der Kalibrierung unterschiedlicher Trackingsysteme und der Kollisionsvermeidung. Beim entfernten Zusammenarbeiten wird die Interaktion durch Latenzzeiten und Verbindungsverluste zusätzlich beeinflusst. Schließlich haben die Benutzer unterschiedliche Anforderungen an die Visualisierung von Inhalten, z.B. Größe, Ausrichtung, Farbe oder Kontrast, innerhalb der virtuellen Welten. Eine strikte Nachbildung von realen Umgebungen in VR verschenkt Potential und wird es nicht ermöglichen, die individuellen Bedürfnisse der Benutzer zu berücksichtigen. Um diese Probleme anzugehen, werden in der vorliegenden Arbeit Lösungen in den Bereichen Eingabe, Zusammenarbeit und Erweiterung von virtuellen Welten und Benutzern vorgestellt, die darauf abzielen, die Benutzerfreundlichkeit und Produktivität von VR zu erhöhen. Zunächst werden PC-basierte Hardware und Software in die virtuelle Welt übertragen, um die Vertrautheit und den Funktionsumfang bestehender Anwendungen in VR zu erhalten. Virtuelle Stellvertreter von physischen Geräten, z.B. Tastatur und Tablet, und ein VR-Modus für Anwendungen ermöglichen es dem Benutzer reale Fähigkeiten in die virtuelle Welt zu übertragen. Des Weiteren wird ein Algorithmus vorgestellt, der die Kalibrierung mehrerer ko-lokaler VR-Geräte mit hoher Genauigkeit und geringen Hardwareanforderungen und geringem Aufwand ermöglicht. Da VR-Headsets die reale Umgebung der Benutzer ausblenden, wird die Relevanz einer Ganzkörper-Avatar-Visualisierung für die Kollisionsvermeidung und das entfernte Zusammenarbeiten nachgewiesen. Darüber hinaus werden personalisierte räumliche oder zeitliche Modifikationen vorgestellt, die es erlauben, die Benutzerfreundlichkeit, Arbeitsleistung und soziale Präsenz von Benutzern zu erhöhen. Diskrepanzen zwischen den virtuellen Welten, die durch persönliche Anpassungen entstehen, werden durch Methoden der Avatar-Umlenkung (engl. redirection) kompensiert. Abschließend werden einige der Methoden und Erkenntnisse in eine beispielhafte Anwendung integriert, um deren praktische Anwendbarkeit zu verdeutlichen. Die vorliegende Arbeit zeigt, dass virtuelle Umgebungen auf realen Fähigkeiten und Erfahrungen aufbauen können, um eine vertraute und einfache Interaktion und Zusammenarbeit von Benutzern zu gewährleisten. Darüber hinaus ermöglichen individuelle Erweiterungen des virtuellen Inhalts und der Avatare Einschränkungen der realen Welt zu überwinden und das Erlebnis von VR-Umgebungen zu steigern