Ubiquitous volume rendering in the web platform

176 p.The main thesis hypothesis is that ubiquitous volume rendering can be achieved using WebGL. The thesis enumerates the challenges that should be met to achieve that goal. The results allow web content developers the integration of interactive volume rendering within standard HTML5 web pages. Content developers only need to declare the X3D nodes that provide the rendering characteristics they desire. In contrast to the systems that provide specific GPU programs, the presented architecture creates automatically the GPU code required by the WebGL graphics pipeline. This code is generated directly from the X3D nodes declared in the virtual scene. Therefore, content developers do not need to know about the GPU.The thesis extends previous research on web compatible volume data structures for WebGL, ray-casting hybrid surface and volumetric rendering, progressive volume rendering and some specific problems related to the visualization of medical datasets. Finally, the thesis contributes to the X3D standard with some proposals to extend and improve the volume rendering component. The proposals are in an advance stage towards their acceptance by the Web3D Consortium

Ubiquitous volume rendering in the web platform

176 p.The main thesis hypothesis is that ubiquitous volume rendering can be achieved using WebGL. The thesis enumerates the challenges that should be met to achieve that goal. The results allow web content developers the integration of interactive volume rendering within standard HTML5 web pages. Content developers only need to declare the X3D nodes that provide the rendering characteristics they desire. In contrast to the systems that provide specific GPU programs, the presented architecture creates automatically the GPU code required by the WebGL graphics pipeline. This code is generated directly from the X3D nodes declared in the virtual scene. Therefore, content developers do not need to know about the GPU.The thesis extends previous research on web compatible volume data structures for WebGL, ray-casting hybrid surface and volumetric rendering, progressive volume rendering and some specific problems related to the visualization of medical datasets. Finally, the thesis contributes to the X3D standard with some proposals to extend and improve the volume rendering component. The proposals are in an advance stage towards their acceptance by the Web3D Consortium

Virtual Heritage: new technologies for edutainment

Cultural heritage represents an enormous amount of information and knowledge. Accessing this treasure chest allows not only to discover the legacy of physical and intangible attributes of the past but also to provide a better understanding of the present. Museums and cultural institutions have to face the problem of providing access to and communicating these cultural contents to a wide and assorted audience, meeting the expectations and interests of the reference end-users and relying on the most appropriate tools available. Given the large amount of existing tangible and intangible heritage, artistic, historical and cultural contents, what can be done to preserve and properly disseminate their heritage significance? How can these items be disseminated in the proper way to the public, taking into account their enormous heterogeneity? Answering this question requires to deal as well with another aspect of the problem: the evolution of culture, literacy and society during the last decades of 20th century. To reflect such transformations, this period witnessed a shift in the museum’s focus from the aesthetic value of museum artifacts to the historical and artistic information they encompass, and a change into the museums’ role from a mere "container" of cultural objects to a "narrative space" able to explain, describe, and revive the historical material in order to attract and entertain visitors. These developments require creating novel exhibits, able to tell stories about the objects and enabling visitors to construct semantic meanings around them. The objective that museums presently pursue is reflected by the concept of Edutainment, Education + Entertainment. Nowadays, visitors are not satisfied with ‘learning something’, but would rather engage in an ‘experience of learning’, or ‘learning for fun’, being active actors and players in their own cultural experience. As a result, institutions are faced with several new problems, like the need to communicate with people from different age groups and different cultural backgrounds, the change in people attitude due to the massive and unexpected diffusion of technology into everyday life, the need to design the visit by a personal point of view, leading to a high level of customization that allows visitors to shape their path according to their characteristics and interests. In order to cope with these issues, I investigated several approaches. In particular, I focused on Virtual Learning Environments (VLE): real-time interactive virtual environments where visitors can experience a journey through time and space, being immersed into the original historical, cultural and artistic context of the work of arts on display. VLE can strongly help archivists and exhibit designers, allowing to create new interesting and captivating ways to present cultural materials. In this dissertation I will tackle many of the different dimensions related to the creation of a cultural virtual experience. During my research project, the entire pipeline involved into the development and deployment of VLE has been investigated. The approach followed was to analyze in details the main sub-problems to face, in order to better focus on specific issues. Therefore, I first analyzed different approaches to an effective recreation of the historical and cultural context of heritage contents, which is ultimately aimed at an effective transfer of knowledge to the end-users. In particular, I identified the enhancement of the users’ sense of presence in VLE as one of the main tools to reach this objective. Presence is generally expressed as the perception of 'being there', i.e. the subjective belief of users that they are in a certain place, even if they know that the experience is mediated by the computer. Presence is related to the number of senses involved by the VLE and to the quality of the sensorial stimuli. But in a cultural scenario, this is not sufficient as the cultural presence plays a relevant role. Cultural presence is not just a feeling of 'being there' but of being - not only physically, but also socially, culturally - 'there and then'. In other words, the VLE must be able to transfer not only the appearance, but also all the significance and characteristics of the context that makes it a place and both the environment and the context become tools capable of transferring the cultural significance of a historic place. The attention that users pay to the mediated environment is another aspect that contributes to presence. Attention is related to users’ focalization and concentration and to their interests. Thus, in order to improve the involvement and capture the attention of users, I investigated in my work the adoption of narratives and storytelling experiences, which can help people making sense of history and culture, and of gamification approaches, which explore the use of game thinking and game mechanics in cultural contexts, thus engaging users while disseminating cultural contents and, why not?, letting them have fun during this process. Another dimension related to the effectiveness of any VLE is also the quality of the user experience (UX). User interaction, with both the virtual environment and its digital contents, is one of the main elements affecting UX. With respect to this I focused on one of the most recent and promising approaches: the natural interaction, which is based on the idea that persons need to interact with technology in the same way they are used to interact with the real world in everyday life. Then, I focused on the problem of presenting, displaying and communicating contents. VLE represent an ideal presentation layer, being multiplatform hypermedia applications where users are free to interact with the virtual reconstructions by choosing their own visiting path. Cultural items, embedded into the environment, can be accessed by users according to their own curiosity and interests, with the support of narrative structures, which can guide them through the exploration of the virtual spaces, and conceptual maps, which help building meaningful connections between cultural items. Thus, VLE environments can even be seen as visual interfaces to DBs of cultural contents. Users can navigate the VE as if they were browsing the DB contents, exploiting both text-based queries and visual-based queries, provided by the re-contextualization of the objects into their original spaces, whose virtual exploration can provide new insights on specific elements and improve the awareness of relationships between objects in the database. Finally, I have explored the mobile dimension, which became absolutely relevant in the last period. Nowadays, off-the-shelf consumer devices as smartphones and tablets guarantees amazing computing capabilities, support for rich multimedia contents, geo-localization and high network bandwidth. Thus, mobile devices can support users in mobility and detect the user context, thus allowing to develop a plethora of location-based services, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits and cultural or tourist sites according to visitors’ personal interest and curiosity

Analysis of Visualisation and Interaction Tools Authors

This document provides an in-depth analysis of visualization and interaction tools employed in the context of Virtual Museum. This analysis is required to identify and design the tools and the different components that will be part of the Common Implementation Framework (CIF). The CIF will be the base of the web-based services and tools to support the development of Virtual Museums with particular attention to online Virtual Museum.The main goal is to provide to the stakeholders and developers an useful platform to support and help them in the development of their projects, despite the nature of the project itself. The design of the Common Implementation Framework (CIF) is based on an analysis of the typical workflow ofthe V-MUST partners and their perceived limitations of current technologies. This document is based also on the results of the V-MUST technical questionnaire (presented in the Deliverable 4.1). Based on these two source of information, we have selected some important tools (mainly visualization tools) and services and we elaborate some first guidelines and ideas for the design and development of the CIF, that shall provide a technological foundation for the V-MUST Platform, together with the V-MUST repository/repositories and the additional services defined in the WP4. Two state of the art reports, one about user interface design and another one about visualization technologies have been also provided in this document

A Framework to Generate and Label Synthetic/Real Video Data to Feed Temporal Segment Networks

In this project, we propose an action prediction and a data generation pipeline. While, the former makes use of Deep Learning, the latter results in a pipeline that makes possible the generation of real and synthetic data. Moreover, to feed the deep learning method a large amount of annotated data is needed. For this purpose an action tagging tool is also featured. Furthermore, in order to supply the lack of data, we have also proposed a video data augmentation pipeline for action recognition purposes. While the 3DPLab team developed a photorealistic synthetic data generator called UnrealRox, we will use this system working with some sequences recorded with a mocap to generate the necessary synthetic data. We have generated a total of 5 different useful sequences with a complex setup of 3 kinects and a capture motion suit. Finally, we have deployed and tested the novel Temporal Segment Network with the state of the art Action Recognition dataset UCF-101

Design Architecture in Virtual Reality

Architectural representation has newly been introduced to Virtual Reality (VR) technology, which provides architects with a medium to showcase unbuilt designs as immersive experiences. Designers can use specialized VR headsets and equipment to provide a client or member of their design team with the illusion of being within the digital space they are presented on screen. This mode of representation is unprecedented to the architectural field, as VR is able to create the sensation of being encompassed in an environment at full scale, potentially eliciting a visceral response from users, similar to the response physical architecture produces. While this premise makes the technology highly applicable towards the architectural practice, it might not be the most practical medium to communicate design intent. Since VR’s conception, the primary software to facilitate VR content creation has been geared towards programmers rather than architects. The practicality of integrating virtual reality within a traditional architectural design workflow is often overlooked in the discussion surrounding the use of VR to represent design projects. This thesis aims to investigate the practicality of VR as part of a design methodology, through the assessment of efficacy and efficiency, while studying the integration of VR into the architectural workflow. This is done by examining the creation of stereoscopic renderings, walkthrough animations, interactive iterations and quick demonstrations as explorations of common architectural visualization techniques using VR. Experimentation with each visualization method is supplemented with a documentation of the VR scene creation process across an approximated time frame to measure efficiency, and a set of evaluation parameters to measure efficacy. Experiments either yielded the creation of a successful experience that exceeded the time constraints a common fast-paced architectural firm might allow for the task (low efficiency) or created a limiting experience where interaction and functionality were not executed to meet the required industry standards (low efficacy). This resultant impracticality based on time and effort, demonstrates that a successfully immersive VR simulation is not produced simplistically in VR; requiring a great deal of thought to be placed into design intent. Although impractical, documentation suggests that the user experience of creating VR content might be able to engage new ways of design thinking and impact the way architects conceptualize space, encouraging further research

Parametric BIM-based Design Review

This research addressed the need for a new design review technology and method to express the tangible and intangible qualities of architectural experience of parametric BIM-based design projects. The research produced an innovative presentation tool by which parametric design is presented systematically. Focus groups provided assessments of the tool to reveal the usefulness of a parametric BIM-based design review method. The way in which we visualize architecture affects the way we design and perceive architectural form and performance. Contemporary architectural forms and systems are very complex, yet most architects who use Building Information Modeling (BIM) and generative design methods still embrace the two-dimensional 15th-century Albertian representational methods to express and review design projects. However, architecture cannot be fully perceived through a set of drawings that mediate our perception and evaluation of the built environment. The systematic and conventional approach of traditional architectural representation, in paper-based and slide-based design reviews, is not able to visualize phenomenal experience nor the inherent variation and versioning of parametric models. Pre-recorded walk-throughs with high quality rendering and imaging have been in use for decades, but high verisimilitude interactive walk-throughs are not commonly used in architectural presentations. The new generations of parametric and BIM systems allow for the quick production of variations in design by varying design parameters and their relationships. However, there is a lack of tools capable of conducting design reviews that engage the advantages of parametric and BIM design projects. Given the multitude of possibilities of in-game interface design, game-engines provide an opportunity for the creation of an interactive, parametric, and performance-oriented experience of architectural projects with multi-design options. This research has produced a concept for a dynamic presentation and review tool and method intended to meet the needs of parametric design, performance-based evaluation, and optimization of multi-objective design options. The concept is illustrated and tested using a prototype (Parametric Design Review, or PDR) based upon an interactive gaming environment equipped with a novel user interface that simultaneously engages the parametric framework, object parameters, multi-objective optimized design options and their performances with diagrammatic, perspectival, and orthographic representations. The prototype was presented to representative users in multiple focus group sessions. Focus group discussion data reveal that the proposed PDR interface was perceived to be useful if used for design reviews in both academic and professional practice settings

Videopelit ja pukutaide - analogisten pukusuunnittelumetodien digitalisointi

This thesis explores ways of integrating a costume professional to the character art team in the game industry. The research suggests, that integrating costume knowledge into the character design pipeline increases the storytelling value of the characters and provides tools for the narrative. The exploration of integrating a costume professional into game character creation as a process is still rare and little information of costume in games and experiences in transferring an analogue character building skillset into a digital one can be found, therefore this research was generated to provide knowledge on the subject. The research's main emphasis is on immersion-driven AAA-games that employ 3D-graphics and human characters and are either photorealistic or represent stylized realism. Technology for depicting reality is advancing and digital industries have become aware of the extensive skills required to depict increasingly realistic worlds. Also, tools for character art are beginning to lean on actual costume construction: the pattern based cloth simulation software entitled Marvelous Designer has become the industry standard for character clothing. The material of this thesis is based on the author's experience as an intern and Costume Artist at the game company Remedy Entertainment and on data collection in the form of participant observation, conversational interviews, archival searches and assorted documents as an internal employee of the company. Therefore, an ethnographical research that applies to qualitative, descriptive, nonmathematical and naturalistic research methods is utilized in this thesis. The result of this research is a costume production pipeline for integrating a costume professional into the game character design process. It is formed by comparing costuming processes of game and film industries to explore the similarities and differences in methods to analyze the most effective combination of these two. The final pipeline introduces the costume professional’s position during the different stages of the character design process. Furthermore, the thesis categorizes aspects essential for a costume designer to internalize in order to become a functional part of the Character Art team and the skills and knowledge required to support the character design in a production. This research identifies the need for costume knowledge in realistic AAA-games. When employing a costume professional into a game production, this thesis offers tools and vocabulary for collaboration. Costume designers are Character Artists, but with different tools and skill set and costume design can be seen as a live form of character art.Tämä opinnäytetyö käsittelee pukusuunnittelua peliteollisuudessa ja pukusuunnittelijan tarvetta hahmosuunnitteluprosessissa. Tutkimuksen materiaali perustuu kirjoittajan omaan kokemukseen harjoittelijana ja pukusuunnittelijana (Costume Artist) Remedy Entertainment -peliyhtiön hahmosuunnittelutiimissä. Työ esittelee eri peliyhtiöiden hahmotiimien käyttämiä pukusuunnittelumetodeja ja arvioi, miten pukusuunnittelun tiedostaminen erillisenä osana hahmosuunnitteluprosessia ja puvun kerronnallisten elementtien tunnistaminen ja hyödyntäminen tukee uskottavan pelihahmon luomista realistisissa peleissä. Tutkimuksen lähtökohtana on ajatus, että pukusuunnittelun integroiminen hahmosuunnitteluprosessiin syventää hahmoa ja kuluttajan pelikokemusta ja näin ollen nostaa pelin arvoa tuotteena. Tutkimuksen tuloksena syntyi uusi pukusuunnittelun tuotantolinja (Costume production pipeline) peliteollisuuden käyttöön. Se esittelee pukusuunnittelijan asemoitumisen pelituotannon hahmosuunnitteluprosessiin. Teknologinen kehitys mahdollistaa yhä realistisempien digitaalisten todellisuuksien luomisen, jonka seurauksena peliteollisuus työllistää enenevässä määrin myös perinteisten alojen erityisosaajia, kuten arkkitehtejä sekä elokuva-alan valosuunnittelijoita. Myös digitaalisen hahmonluonnin työvälineet nojaavat jo tosielämän vaatetuotantoon: vaatesimulaatio-ohjelma Marvelous Designer perustuu vaatteiden kaavoihin ja on laajalti käytössä pelihahmojen suunnittelussa. Pelit, joita tutkimus tarkastelee, ovat AAA-pelejä, jotka hyödyntävät 3D-grafiikkaa ja ihmishahmoja ja jotka ovat joko photorealistisia tai edustavat tyyliteltyä realismia. Tutkimuksen materiaali on koostettu keskustelullisista, strukturoimattomista haastatteluista, kirjoittajan työpäiväkirjasta, yhtiön edellisten pelien arkistomateriaaleista, sekä yhtiön sisäisistä ohjeistuksista. Tutkimus esittelee keinoja, joilla fyysisen ihmisvartalon kanssa työskentelemään kouluttautunut pukusuunnittelija voi integroitua digitaaliseen hahmonluontiprosessiin ja osoittaa, että pukusuunnittelijat ovat analogisilla metodeilla työskenteleviä hahmosuunnittelijoita. Hahmosuunnittelun etu suhteessa esitystaiteen pukusuunnitteluun on vapaus näyttelijän fyysisen ruumiin sekä fysiikan lakien tuottamista rajoitteista. Vaatesuunnittelu keskittyy vaatteeseen ja pukusuunnittelu fyysisen ihmisruumiin ja vaatteen synnyttämään kombinaatioon ja interaktioon, mutta hahmosuunnittelu tarjoaa pukusuunnittelullisesti rajoittamattoman vapauden hahmotulkintaan