An Actor-Centric Approach to Facial Animation Control by Neural Networks For Non-Player Characters in Video Games

Game developers increasingly consider the degree to which character animation emulates facial expressions found in cinema. Employing animators and actors to produce cinematic facial animation by mixing motion capture and hand-crafted animation is labor intensive and therefore expensive. Emotion corpora and neural network controllers have shown promise toward developing autonomous animation that does not rely on motion capture. Previous research and practice in disciplines of Computer Science, Psychology and the Performing Arts have provided frameworks on which to build a workflow toward creating an emotion AI system that can animate the facial mesh of a 3d non-player character deploying a combination of related theories and methods. However, past investigations and their resulting production methods largely ignore the emotion generation systems that have evolved in the performing arts for more than a century. We find very little research that embraces the intellectual process of trained actors as complex collaborators from which to understand and model the training of a neural network for character animation. This investigation demonstrates a workflow design that integrates knowledge from the performing arts and the affective branches of the social and biological sciences. Our workflow begins at the stage of developing and annotating a fictional scenario with actors, to producing a video emotion corpus, to designing training and validating a neural network, to analyzing the emotion data annotation of the corpus and neural network, and finally to determining resemblant behavior of its autonomous animation control of a 3d character facial mesh. The resulting workflow includes a method for the development of a neural network architecture whose initial efficacy as a facial emotion expression simulator has been tested and validated as substantially resemblant to the character behavior developed by a human actor

Gaze-directed gameplay in first person computer games

The use of eye tracking systems in computer games is still at an early stage. Commercial eye trackers and researches have been focusing in gaze-oriented gameplay as an alternative to traditional input devices. This dissertation proposes to investigate the advantages and disadvantages of the use of these systems in computer games. For it, instead of using eye tracking as a simple direct control input, it is proposed to use it in order to control the attention of the player’s avatar (e.g., if the player notices an obstacle in the way, the avatar will notice it too and avoid it) and the game’s procedural content generation (e.g., spawn obstacles in the opposite side of the screen to where the player’s attention is focused). To demonstrate the value of this proposal, it was developed and is herein presented the first-person shooter "Zombie Runner". Tests showed that the implementation meets the stipulated technical requirements and that, although it still needs improvements in terms of precision and robustness, eye tracking technology can be successfully used to to make the player experience more immersive and challenging.A utilização de sistemas de rastreamento ocular em jogos de computador ainda se encontra numa fase embrionária. Aparelhos de rastreamento ocular comerciais e pesquisas na área têm-se focado em jogabilidade à base da atenção visual como uma alternativa a métodos de entrada tradicionais. Esta dissertação propõe-se a investigar as vantages e desvantagens do uso destes sistemas em jogos de computador. Para isso, invés de se usar rastreamento ocular apenas como um método directo de entrada, é proposto usá-lo para controlar a atenção do personagem do jogo (e.g., se o jogador reparar num obstáculo, a personagem também repara e desvia-se do mesmo) assim como afectar a geração procedimental do jogo (e.g., gerar obstáculos no lado oposto ao qual o jogador tem a sua atenção focada). Para demonstrar o valor desta proposta, foi desenvolvido e aqui apresentado o jogo de tiros em primeira pessoa "Zombie Runner". Os testes demonstraram que a implementação cumpre os requisitos técnicos estipulados e que, apesar de ainda carecer de melhorias em termos de precisão e robustez, a tecnologia para rastreamento ocular pode ser utilizada com sucesso para tornar a experiência do jogador mais imersiva e desafiante

What is the Avatar? Fiction and Embodiment in Avatar-Based Singleplayer Computer Games: Revised and Commented Edition

What are the characteristic features of avatar-based singleplayer videogames, from Super Mario Bros. to Grand Theft Auto? The author examines this question with a particular focus on issues of fictionality and realism, and their relation to cinema and Virtual Reality. Through close-up analysis and philosophical discussion, the author argues that avatar-based gaming is a distinctive and dominant form of virtual self-embodiment in digital culture. This book is a revised edition of Rune Klevjer's pioneering work from 2007, featuring a new introduction by the author and afterword by Stephan Günzel, Jörg Sternagel, and Dieter Mersch

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Sense3

Sense3, at its core, is a game in which the player has to collectively use the three senses of sight, touch and hearing to primarily dodge obstacles and collect audio samples, which are vertically remixed to form music. We chose EDM themed music, coupled with a futuristic outer space theme for the art style of the game, creating an immersive environment. The game is a pseudo-endless runner, where the player has to collect all the components to assemble their music. After each play through, the game allows the player to listen to the newly assembled music before reloading. We wanted to encourage replay value as each play through yields new and different music. The concept was decided upon after a series of prototyping sessions and iterations on a select few of them

Immersion and interaction: Creating virtual 3d worlds for stage performances

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis formulates an approach towards the creation of a gesture activated and body movement controlled real time virtual 3d world in a dance performance context. It investigates immersion and navigation techniques derived from modern video games and methodologies and proposes how they can be used to further involve a performer into a virtual space as well as simultaneously offer a stimulating visual spectacle for an audience. The argument presented develops through practice-based methodology and artistic production strategies in interdisciplinary and collaborative contexts. Two choreographic performance/installations are used as cases studies to demonstrate in practice the proposed methodologies. First, the interactive dance work Suna No Onna, created in collaboration with Birringer/Danjoux and the Dap Lab, investigates the use of interactive pre-rendered animations in a real time setting and in real time by incorporating wearable sensors in the performance. Secondly, the potentials offered by the sensor technology and real time rendering engines led to the “creation scene", a key scene in the choreographic installation UKIYO (Moveable Worlds). This thesis investigates the design, creation and interaction qualities of virtual 3d spaces by exploring the potentialities offered by a shared space, between an intelligent space and a dancer in a hybrid world. The methodology applied uses as a theoretical base the phenomenological approach of Merleau-Ponty and Mark Hansen‟s mixed reality paradigm proposing the concept of the “space schema", a system which replicates and embeds proprioception, perception and motility into the space fabric offering a world which “lives”, functions and interacts with the performer. The outcome of the research is the generation of an interactive, non-linear, randomized 3d virtual space that collaborates with a technologically embedded performer in creating a 3d world which evolves and transforms, driven by the performer‟s intention and agency. This research contributes to the field of interactive performance art by making transparent the methodology, the instruments and the code used, in a non-technical terminology, making it accessible for both team members with less technological expertise as well as artists aspiring to engage interactive 3d media promoting further experimentation and conceptual discussions

Development of A First-Person Horror Game With High-Fidelity Graphics

Victims of Dead Stories is a psychological single-player horror game in the first-person perspective, with high-fidelity graphics, where the player explores the hallways of a house whileexperiencingsupernaturaloccurrences,andprogressinginthenarrativewhilelooping around the house. The major inspiration for Victims of Dead Stories is P.T. Silent Hills, developed by KojimaProductions, while theplayer is constantly looping around the house. Themainobjectiveofthis project is to design and developagamewithrealistic graphics, reaching an appearance of a AAA game, while maintaining good performance and offering optimization options to the player. This project is also proof of what two students from different areas can achieve, of what indie developers can accomplish with love and dedication to a game. As for the methodology, this document consists of detailing the design of various areas of the game- the narrative, level design, game mechanics, among others- as well as detailing the development of those design areas, by using the Unity Engine and the C# language, to implement and export the game for the PC platform.Victims of Dead Stories é um jogo de terror psicológico para um jogador na perspectiva da primeira pessoa, com gráficos de alta fidelidade, onde o jogador explora os corredores de uma casa enquanto experiencia ocorrências sobrenaturais, e progride na narrativa enquanto percorre a casa. A principal inspiração para Victims of Dead Stories é P.T. Silent Hills, desenvolvido pela Kojima Productions, onde o jogador está constantemente a experienciar loops dos corredores. O principal objectivo deste projecto é conceber e desenvolver um jogo com gráficos realistas, alcançando a aparência de um jogo AAA, mantendo ao mesmo tempo um bom desempenho e oferecendo opções de optimização ao jogador. Este projecto é também uma prova do que dois estudantes de áreas diferentes podem alcançar, do que os criadores independentes podem realizar com amor e dedicação a um jogo. Quanto à metodologia, este documento consiste em detalhar o design de várias áreas do jogo- a narrativa, o level design, as mecânicas do jogo, entre outras- assim como em detalhar o desenvolvimento dessas áreas de design, utilizando o motor de jogo Unity e a linguagem de programação C#, para implementar e exportar o jogo para a plataforma PC

What is the Avatar?

What are the characteristic features of avatar-based singleplayer videogames, from Super Mario Bros. to Grand Theft Auto? Rune Klevjer examines this question with a particular focus on issues of fictionality and realism, and their relation to cinema and Virtual Reality. Through close-up analysis and philosophical discussion, Klevjer argues that avatar-based gaming is a distinctive and dominant form of virtual self-embodiment in digital culture. This book is a revised edition of Rune Klevjer's pioneering work from 2007, featuring a new introduction by the author and afterword by Stephan Günzel, Jörg Sternagel, and Dieter Mersch