Interactive exploration of historic information via gesture recognition

Developers of interactive exhibits often struggle to �nd appropriate input devices that enable intuitive control, permitting the visitors to engage e�ectively with the content. Recently motion sensing input devices like the Microsoft Kinect or Panasonic D-Imager have become available enabling gesture based control of computer systems. These devices present an attractive input device for exhibits since the user can interact with their hands and they are not required to physically touch any part of the system. In this thesis we investigate techniques to enable the raw data coming from these types of devices to be used to control an interactive exhibit. Object recognition and tracking techniques are used to analyse the user's hand where movement and clicks are processed. To show the e�ectiveness of the techniques the gesture system is used to control an interactive system designed to inform the public about iconic buildings in the centre of Norwich, UK. We evaluate two methods of making selections in the test environment. At the time of experimentation the technologies were relatively new to the image processing environment. As a result of the research presented in this thesis, the techniques and methods used have been detailed and published [3] at the VSMM (Virtual Systems and Multimedia 2012) conference with the intention of further forwarding the area

Interaction and locomotion techniques for the exploration of massive 3D point clouds in vr environments

Emerging virtual reality (VR) technology allows immersively exploring digital 3D content on standard consumer hardware. Using in-situ or remote sensing technology, such content can be automatically derived from real-world sites. External memory algorithms allow for the non-immersive exploration of the resulting 3D point clouds on a diverse set of devices with vastly different rendering capabilities. Applications for VR environments raise additional challenges for those algorithms as they are highly sensitive towards visual artifacts that are typical for point cloud depictions (i.e., overdraw and underdraw), while simultaneously requiring higher frame rates (i.e., around 90 fps instead of 30–60 fps). We present a rendering system for the immersive exploration and inspection of massive 3D point clouds on state-of-the-art VR devices. Based on a multi-pass rendering pipeline, we combine point-based and image-based rendering techniques to simultaneously improve the rendering performance and the visual quality. A set of interaction and locomotion techniques allows users to inspect a 3D point cloud in detail, for example by measuring distances and areas or by scaling and rotating visualized data sets. All rendering, interaction and locomotion techniques can be selected and configured dynamically, allowing to adapt the rendering system to different use cases. Tests on data sets with up to 2.6 billion points show the feasibility and scalability of our approach

Architectural rendering and 3D visualization

The following thesis, “Architectural Render and 3D Visualization,” describes the process of creating, rendering, and optimizing an Interior Design using a 3D Engine as the principal tool. The tool used during the development is “Unreal Engine,” which allows rendering and interaction in real-time with the scene. At the end of the process, we can obtain an interactive scene rendered with highquality materials trying to reach a realistic real-time scene by mixing modeling, texturing, and illumination techniques. Furthermore, scripting is contemplated in the project scope, looking to optimize the environment where we will be developing the scene, and developing some tools

rigorous procedure for mapping thermal infrared images on three dimensional models of building facades

A rigorous methodology for mapping thermal and RGB images on three-dimensional (3-D) models of building facades is presented. The developed method differs from most existing approaches because it relies on the use of thermal images coupled with 3-D models derived from terrestrial laser scanning surveying. The primary issue for an accurate texturing is the coregistration of the geometric model of the facade and the thermal images in the same reference system. This task is done by using a procedure standing out from other approaches adopted in current practice, which are mainly based on the independent registration of each image on the basis of homography or space resection techniques. A rigorous photogrammetric orientation of both thermal and RGB images is computed together in a combined bundle adjustment. This solution allows one to have a better control of the quality of the results, especially to reduce errors and artifacts in areas where more images are mosaicked onto the 3-D model. Several products can be obtained: 3-D triangulated textured models or raster products like orthophotos, having the temperature as radiometric value. The proposed approach is tested on different buildings of Politecnico di Milano University. Applications demonstrated the performance of the procedure and its technical applicability in routine thermal surveys

Development and Application of Computer Graphics Techniques for the Visualization of Large Geo-Related Data-Sets

Ziel dieser Arbeit war es, Algorithmen zu entwickeln und zu verbessern, die es gestatten, grosse geographische und andere geo-bezogene Datensätze mithilfe computergraphischer Techniken visualisieren zu können. Ein Schwerpunkt war dabei die Entwicklung neuer kamera-adaptiver Datenstrukturen für digitale Höhenmodelle und Rasterbilder. In der Arbeit wird zunächst ein neuartiges Multiresolutionmodell für Höhenfelder definiert. Dieses Modell braucht nur sehr wenig zusätzlichen Speicherplatz und ist geeignet, interaktive Anpassungsraten zu gewährleisten. Weiterhin werden Ansätze zur schnellen Bestimmung sichtbarer und verdeckter Teile einer computergraphischen Szene diskutiert, um die Bewegung in grossen und ausgedehnten Szenen wie Stadtmodellen oder Gebäuden zu beschleunigen. Im Anschluss daran werden einige Problemstellungen im Zusammenhang mit Texture Mapping erörtert, so werden zum Beispiel eine neue beobachterabhängige Datenstruktur für Texturdaten und ein neuer Ansatz zur Texturfilterung vorgestellt. Die meisten dieser Algorithmen und Verfahren wurden in ein interaktives System zur Geländevisualisierung integriert, das den Projektnamen 'FlyAway' hat und im letzten Kapitel der Arbeit beschrieben wird

Transmission adaptative de modèles 3D massifs

Avec les progrès de l'édition de modèles 3D et des techniques de reconstruction 3D, de plus en plus de modèles 3D sont disponibles et leur qualité augmente. De plus, le support de la visualisation 3D sur le web s'est standardisé ces dernières années. Un défi majeur est donc de transmettre des modèles massifs à distance et de permettre aux utilisateurs de visualiser et de naviguer dans ces environnements virtuels. Cette thèse porte sur la transmission et l'interaction de contenus 3D et propose trois contributions majeures. Tout d'abord, nous développons une interface de navigation dans une scène 3D avec des signets -- de petits objets virtuels ajoutés à la scène sur lesquels l'utilisateur peut cliquer pour atteindre facilement un emplacement recommandé. Nous décrivons une étude d'utilisateurs où les participants naviguent dans des scènes 3D avec ou sans signets. Nous montrons que les utilisateurs naviguent (et accomplissent une tâche donnée) plus rapidement en utilisant des signets. Cependant, cette navigation plus rapide a un inconvénient sur les performances de la transmission : un utilisateur qui se déplace plus rapidement dans une scène a besoin de capacités de transmission plus élevées afin de bénéficier de la même qualité de service. Cet inconvénient peut être atténué par le fait que les positions des signets sont connues à l'avance : en ordonnant les faces du modèle 3D en fonction de leur visibilité depuis un signet, on optimise la transmission et donc, on diminue la latence lorsque les utilisateurs cliquent sur les signets. Deuxièmement, nous proposons une adaptation du standard de transmission DASH (Dynamic Adaptive Streaming over HTTP), très utilisé en vidéo, à la transmission de maillages texturés 3D. Pour ce faire, nous divisons la scène en un arbre k-d où chaque cellule correspond à un adaptation set DASH. Chaque cellule est en outre divisée en segments DASH d'un nombre fixe de faces, regroupant des faces de surfaces comparables. Chaque texture est indexée dans son propre adaptation set à différentes résolutions. Toutes les métadonnées (les cellules de l'arbre k-d, les résolutions des textures, etc.) sont référencées dans un fichier XML utilisé par DASH pour indexer le contenu: le MPD (Media Presentation Description). Ainsi, notre framework hérite de la scalabilité offerte par DASH. Nous proposons ensuite des algorithmes capables d'évaluer l'utilité de chaque segment de données en fonction du point de vue du client, et des politiques de transmission qui décident des segments à télécharger. Enfin, nous étudions la mise en place de la transmission et de la navigation 3D sur les appareils mobiles. Nous intégrons des signets dans notre version 3D de DASH et proposons une version améliorée de notre client DASH qui bénéficie des signets. Une étude sur les utilisateurs montre qu'avec notre politique de chargement adaptée aux signets, les signets sont plus susceptibles d'être cliqués, ce qui améliore à la fois la qualité de service et la qualité d'expérience des utilisateur

Developing a virtual zoological museum

Abstract. This is a documentation of our work developing a virtual zoological museum. Although it’s challenging to create a virtual museum that lives up to the original, Unity3D and virtual reality technology are utilized in order to provide experiences that a traditional museum cannot. As we aim to digitize the museum that once was in University of Oulu, different ways of designing an educating and engaging virtual museum visit are explored. The animals of the museum can be interacted with, being able to play back animations and audio while also providing information in text form. An interactive forest was also developed as a more natural and lively environment. Furthermore, 360° photos of local forests were added to improve the representation of nature. Virtual reality support was programmed for Oculus Rift, allowing movement and interaction as if one was there in real life. In order to achieve a comfortable experience, some performance optimization has been done to reach stable frame rates. We evaluated users’ sense of presence, experienced Game Transfer Phenomena (GTP), system usability and content quality. Based on our tests, users found the virtual museum visit enjoyable and immersive overall despite being distracted by some aspects, like the quality of the display. Users were also mostly satisfied with the environments and the quality of the animals. Experienced Game Transfer was low, however. All in all, this concept for creating a virtual museum has appeared to be successful, and it could be developed further.Virtuaalisen eläintieteellisen museon kehittäminen. Tiivistelmä. Tämä on dokumentaatio virtuaalisen eläintieteellisen museon kehittämisestä. Vaikka onkin haastavaa luoda virtuaalinen museo, joka on verrattavissa alkuperäiseen, hyödyntämällä Unity3D:tä ja virtuaalitodellisuusteknologiaa on mahdollista tarjota kokemuksia, mitä perinteinen museo ei pysty. Digitalisoidessamme sitä museota, joka Oulun Yliopistolla ennen oli, tutkimme erilaisia keinoja kehittää opetuksellinen ja kiinnostava virtuaalimuseovierailu. Museon eläimet ovat interaktiivisia, pystyen toistamaan animaatioita ja ääniä sekä antamaan tietoa tekstin muodossa. Interaktiivinen metsä luotiin tarjoamaan luonnollisemman ja elävämmän ympäristön. Lisäksi 360° kuvia paikallisista metsistä lisättiin parantaakseen luonnon edustusta. Virtuaalitodellisuustuki lisättiin Oculus Rift:ille, sallien liikkumisen ja vuorovaikuttamisen kuin olisi siellä todellisessa elämässä. Luodakseen mukavan kokemuksen, sovelluksen suorituskykyä on optimoitu saavuttaakseen vakaan kuvan päivitystaajuuden. Evaluoimme käyttäjien läsnäolon tunnetta, koettua Game Transfer -ilmiötä (GTP), järjestelmän käytettävyyttä ja sisällön laatua. Testien perusteella käyttäjät kokivat museovierailun miellyttävänä sekä immersiivisenä yleisesti ottaen, vaikka jotkin piirteet, kuten näytön laatu, häiritsivät. Käyttäjät olivat myöskin pitkälti tyytyväisiä ympäristöihin ja eläinten laatuun. Koettu Game Transfer oli kuitenkin vähäistä. Kaikenkaikkiaan tämä virtuaalimuseo konsepti vaikuttaa toimivalta, ja sitä voisi kehittää pidemmälle

Methods for Real-time Visualization and Interaction with Landforms

This thesis presents methods to enrich data modeling and analysis in the geoscience domain with a particular focus on geomorphological applications. First, a short overview of the relevant characteristics of the used remote sensing data and basics of its processing and visualization are provided. Then, two new methods for the visualization of vector-based maps on digital elevation models (DEMs) are presented. The first method uses a texture-based approach that generates a texture from the input maps at runtime taking into account the current viewpoint. In contrast to that, the second method utilizes the stencil buffer to create a mask in image space that is then used to render the map on top of the DEM. A particular challenge in this context is posed by the view-dependent level-of-detail representation of the terrain geometry. After suitable visualization methods for vector-based maps have been investigated, two landform mapping tools for the interactive generation of such maps are presented. The user can carry out the mapping directly on the textured digital elevation model and thus benefit from the 3D visualization of the relief. Additionally, semi-automatic image segmentation techniques are applied in order to reduce the amount of user interaction required and thus make the mapping process more efficient and convenient. The challenge in the adaption of the methods lies in the transfer of the algorithms to the quadtree representation of the data and in the application of out-of-core and hierarchical methods to ensure interactive performance. Although high-resolution remote sensing data are often available today, their effective resolution at steep slopes is rather low due to the oblique acquisition angle. For this reason, remote sensing data are suitable to only a limited extent for visualization as well as landform mapping purposes. To provide an easy way to supply additional imagery, an algorithm for registering uncalibrated photos to a textured digital elevation model is presented. A particular challenge in registering the images is posed by large variations in the photos concerning resolution, lighting conditions, seasonal changes, etc. The registered photos can be used to increase the visual quality of the textured DEM, in particular at steep slopes. To this end, a method is presented that combines several georegistered photos to textures for the DEM. The difficulty in this compositing process is to create a consistent appearance and avoid visible seams between the photos. In addition to that, the photos also provide valuable means to improve landform mapping. To this end, an extension of the landform mapping methods is presented that allows the utilization of the registered photos during mapping. This way, a detailed and exact mapping becomes feasible even at steep slopes

A Framework to Generate 3D Learning Experience

A Collaborative Virtual Environment (CVE) is a computer-based virtual space that supports collaborative work and social interplay. In a 3D CVE, a ‘hosting’ 3D world is the necessary ingredient: within it users provided with graphical embodiments called avatars that convey their identity (presence, location, movement etc.), can meet and interact with other users, with agents or with virtual objects. Even if graphics hardware and 3D technologies are rapidly evolving and the increased Internet connection speed allows the sharing of amounts of data and information among geographically distributed users, the development of networked three-dimensional applications is still complicated and requires expert knowledge. Although some collaborative 3D Web technologies and applications have already been developed, most of them are particularly concerned with offering a high level realistic representation of the virtual world since increasing the level of detail increases the sense of ‘virtual presence’ in the 3D world. However, these developments have not, at the same time supported a high level, non-expert authoring process and the concepts of programming flexibility and component re-use have rarely been taken into account. In this introduction, we discuss our research experience in the field of Collaborative Virtual Environments. We will outline our approach which has been based on both multi-channel integration and on high performances issues