Volumetric kombat:a case study on developing a VR game with Volumetric Video

This paper presents a case study on the development of a Virtual Reality (VR) game using Volumetric Video (VV) for character animation. We delve into the potential of VV, a technology that fuses video and depth sensor data, which has progressively matured since its initial introduction in 1995. Despite its potential to deliver unmatched realism and dynamic 4D sequences, VV applications are predominantly used in non-interactive scenarios. We explore the barriers to entry such as high costs associated with large-scale VV capture systems and the lack of tools optimized for VV in modern game engines. By actively using VV to develop a VR game, we examine and overcome these constraints developing a set of tools that address these challenges. Drawing lessons from past games, we propose an open-source data processing workflow for future VV games. This case study provides insights into the opportunities and challenges of VV in game development and contributes towards making VV more accessible for creators and researchers

A General Framework for Digital Game-Based Training Systems

This dissertation introduces a general framework for developing digital game-based training systems. The framework addresses all aspects of education. In that sense, it involves both teacher and student as game players. A visual review log supports teacher-student-communication. Based on the general framework, a novel digital game-based training system for crime scene investigation training called OpenCrimeScene has been developed and implemented.Diese Dissertation führt ein allgemeines Architekturmodell für die Entwicklung von computerspielbasierten Trainingssystemen ein. Das Architekturmodell spricht alle Aspekte der Ausbildung an. In diesem Sinne bezieht es Lehrer und Schüler als Spieler ein. Ein grafisches Bewertungsprotokoll unterstützt die Lehrer-Schüler-Kommunikation. Basierend auf dem allgemeinen Architekturmodell wurde ein neues computerspielbasiertes Trainingssystem zur Tatortsicherung entwickelt und implementiert

Automation of Robotic Agents and Generation of Synthetic Data for Semantic Segmentation

Semantic segmentation is one of the most relevant techniques in the object detection field since it provides highly valuable scene information and context. Nowadays, data driven algorithms, specifically deep learning, dictate the current state-of-the-art. Although these algorithms can be extremely accurate, they require vast amounts of data. Additionally, this data needs to be highly variable, which is fundamental in order for these models to properly achieve a general solution. On top of that, obtaining such data requires human operators for both the capture and labeling process. All of these constraints present a major drawback in terms of time, cost and resources. The Sim-To-Real field offers an alternative by synthetically generating the data for these algorithms. In this thesis, we propose a modification of a data generation framework, which is aimed towards the automation of action sequences. Then, we use the generated data in order to train and evaluate the performance of semantic segmentation architectures

Presenting ancient history through serious games: a case-study in Assyriology

This thesis investigates how ancient history can be effectively presented through serious games for educational purposes, by presenting a case study in the field of Assyriology. It is anticipated that the results are applicable to other fields of history and cultural heritage. A model is presented to describe how heritage and historical content can be manifested in video games, for the design of serious games for heritage and the analysis of commercial games that present historical material. The theories of reduced fidelity constrained virtual environments are applied to serious games for heritage, to reduce required development resources. A constrained implementation of a serious game for Assyriology is tested against an equivalent 3D environment, and results indicate the constrained environment can achieve comparable levels of presence, enjoyment, quality, and interest in the subject. Based on an interview with an Assyriology field expert, a methodology for the analysis and design of serious games for heritage is presented, based on activity theory. The methodology is applied to the analysis and redesign of a serious game for Assyriology, and the development of a playable prototype. An online user-test showed the redesigned game was enjoyed by participants and was effective at achieving its learning objectives

ISS++: Image as Stepping Stone for Text-Guided 3D Shape Generation

In this paper, we present a new text-guided 3D shape generation approach (ISS++) that uses images as a stepping stone to bridge the gap between text and shape modalities for generating 3D shapes without requiring paired text and 3D data. The core of our approach is a two-stage feature-space alignment strategy that leverages a pre-trained single-view reconstruction (SVR) model to map CLIP features to shapes: to begin with, map the CLIP image feature to the detail-rich 3D shape space of the SVR model, then map the CLIP text feature to the 3D shape space through encouraging the CLIP-consistency between rendered images and the input text. Besides, to extend beyond the generative capability of the SVR model, we design a text-guided 3D shape stylization module that can enhance the output shapes with novel structures and textures. Further, we exploit pre-trained text-to-image diffusion models to enhance the generative diversity, fidelity, and stylization capability. Our approach is generic, flexible, and scalable, and it can be easily integrated with various SVR models to expand the generative space and improve the generative fidelity. Extensive experimental results demonstrate that our approach outperforms the state-of-the-art methods in terms of generative quality and consistency with the input text. Codes and models are released at https://github.com/liuzhengzhe/ISS-Image-as-Stepping-Stone-for-Text-Guided-3D-Shape-Generation.Comment: Under review of TPAM

Archaeology of Digital Environments: Tools, Methods, and Approaches

Digital archaeologists use digital tools for conducting archaeological work, but their potential also lies in applying archaeological thinking and methods to understanding digital built environments (i.e., software) as contemporary examples of human settlement, use, and abandonment. This thesis argues for digital spaces as archaeological artifacts, sites, and landscapes that can be investigated in both traditional and non-traditional ways. At the core of my research is the fundamental argument that human-occupied digital spaces can be studied archaeologically with existing and modified theory, tools, and methods to reveal that human occupation and use of synthetic worlds is similar to how people behave in the natural world. Working digitally adds new avenues of investigation into human behavior in relation to the things people make, modify, and inhabit. In order to investigate this argument, the thesis focuses on three video game case studies, each using different kinds of archaeology specifically chosen to help understand the software environments being researched: 1) epigraphy, stylometry, and text analysis for the code-artifact of Colossal Cave Adventure; 2) photogrammetry, 3D printing, GIS mapping, phenomenology, and landscape archaeology within the designed, digital heritage virtual reality game-site of Skyrim VR; 3) actual survey and excavation of 30 heritage sites for a community of displaced human players in the synthetic landscape of No Man’s Sky. My conclusions include a blended approach to conducting future archaeological fieldwork in digital built environments, one that modifies traditional approaches to archaeological sites and material in a post/transhuman landscape. As humanity continues trending towards constant digital engagement, archaeologists need to be prepared to study how digital places are settled, used, and abandoned. This thesis takes a step in that direction using the vernacular of games as a starting point

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

ISS: Image as Stepping Stone for Text-Guided 3D Shape Generation

Text-guided 3D shape generation remains challenging due to the absence of large paired text-shape data, the substantial semantic gap between these two modalities, and the structural complexity of 3D shapes. This paper presents a new framework called Image as Stepping Stone (ISS) for the task by introducing 2D image as a stepping stone to connect the two modalities and to eliminate the need for paired text-shape data. Our key contribution is a two-stage feature-space-alignment approach that maps CLIP features to shapes by harnessing a pre-trained single-view reconstruction (SVR) model with multi-view supervisions: first map the CLIP image feature to the detail-rich shape space in the SVR model, then map the CLIP text feature to the shape space and optimize the mapping by encouraging CLIP consistency between the input text and the rendered images. Further, we formulate a text-guided shape stylization module to dress up the output shapes with novel textures. Beyond existing works on 3D shape generation from text, our new approach is general for creating shapes in a broad range of categories, without requiring paired text-shape data. Experimental results manifest that our approach outperforms the state-of-the-arts and our baselines in terms of fidelity and consistency with text. Further, our approach can stylize the generated shapes with both realistic and fantasy structures and textures

What is the Avatar? Fiction and Embodiment in Avatar-Based Singleplayer Computer Games: Revised and Commented Edition

What are the characteristic features of avatar-based singleplayer videogames, from Super Mario Bros. to Grand Theft Auto? The author examines this question with a particular focus on issues of fictionality and realism, and their relation to cinema and Virtual Reality. Through close-up analysis and philosophical discussion, the author argues that avatar-based gaming is a distinctive and dominant form of virtual self-embodiment in digital culture. This book is a revised edition of Rune Klevjer's pioneering work from 2007, featuring a new introduction by the author and afterword by Stephan Günzel, Jörg Sternagel, and Dieter Mersch

Gaze-directed gameplay in first person computer games

The use of eye tracking systems in computer games is still at an early stage. Commercial eye trackers and researches have been focusing in gaze-oriented gameplay as an alternative to traditional input devices. This dissertation proposes to investigate the advantages and disadvantages of the use of these systems in computer games. For it, instead of using eye tracking as a simple direct control input, it is proposed to use it in order to control the attention of the player’s avatar (e.g., if the player notices an obstacle in the way, the avatar will notice it too and avoid it) and the game’s procedural content generation (e.g., spawn obstacles in the opposite side of the screen to where the player’s attention is focused). To demonstrate the value of this proposal, it was developed and is herein presented the first-person shooter "Zombie Runner". Tests showed that the implementation meets the stipulated technical requirements and that, although it still needs improvements in terms of precision and robustness, eye tracking technology can be successfully used to to make the player experience more immersive and challenging.A utilização de sistemas de rastreamento ocular em jogos de computador ainda se encontra numa fase embrionária. Aparelhos de rastreamento ocular comerciais e pesquisas na área têm-se focado em jogabilidade à base da atenção visual como uma alternativa a métodos de entrada tradicionais. Esta dissertação propõe-se a investigar as vantages e desvantagens do uso destes sistemas em jogos de computador. Para isso, invés de se usar rastreamento ocular apenas como um método directo de entrada, é proposto usá-lo para controlar a atenção do personagem do jogo (e.g., se o jogador reparar num obstáculo, a personagem também repara e desvia-se do mesmo) assim como afectar a geração procedimental do jogo (e.g., gerar obstáculos no lado oposto ao qual o jogador tem a sua atenção focada). Para demonstrar o valor desta proposta, foi desenvolvido e aqui apresentado o jogo de tiros em primeira pessoa "Zombie Runner". Os testes demonstraram que a implementação cumpre os requisitos técnicos estipulados e que, apesar de ainda carecer de melhorias em termos de precisão e robustez, a tecnologia para rastreamento ocular pode ser utilizada com sucesso para tornar a experiência do jogador mais imersiva e desafiante