Immersion and interaction: Creating virtual 3d worlds for stage performances

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis formulates an approach towards the creation of a gesture activated and body movement controlled real time virtual 3d world in a dance performance context. It investigates immersion and navigation techniques derived from modern video games and methodologies and proposes how they can be used to further involve a performer into a virtual space as well as simultaneously offer a stimulating visual spectacle for an audience. The argument presented develops through practice-based methodology and artistic production strategies in interdisciplinary and collaborative contexts. Two choreographic performance/installations are used as cases studies to demonstrate in practice the proposed methodologies. First, the interactive dance work Suna No Onna, created in collaboration with Birringer/Danjoux and the Dap Lab, investigates the use of interactive pre-rendered animations in a real time setting and in real time by incorporating wearable sensors in the performance. Secondly, the potentials offered by the sensor technology and real time rendering engines led to the “creation scene", a key scene in the choreographic installation UKIYO (Moveable Worlds). This thesis investigates the design, creation and interaction qualities of virtual 3d spaces by exploring the potentialities offered by a shared space, between an intelligent space and a dancer in a hybrid world. The methodology applied uses as a theoretical base the phenomenological approach of Merleau-Ponty and Mark Hansen‟s mixed reality paradigm proposing the concept of the “space schema", a system which replicates and embeds proprioception, perception and motility into the space fabric offering a world which “lives”, functions and interacts with the performer. The outcome of the research is the generation of an interactive, non-linear, randomized 3d virtual space that collaborates with a technologically embedded performer in creating a 3d world which evolves and transforms, driven by the performer‟s intention and agency. This research contributes to the field of interactive performance art by making transparent the methodology, the instruments and the code used, in a non-technical terminology, making it accessible for both team members with less technological expertise as well as artists aspiring to engage interactive 3d media promoting further experimentation and conceptual discussions

Study of emotion in videogames : understanding presence and behaviour

Only when videogames are released are we able to look at them and analyse them. Nowadays, platforms to share our thoughts and opinions about a videogame, or part of it, are everywhere, with both positive and negative commentaries being shared daily. However, what makes a game be seen as a positive experience and what components satisfy and engage players in it? In this Dissertation, we aim to comprehend how players perceive videogames and what motivates and triggers emotions one has during play. We will take a look at several different concepts that all work together when playing a videogame. We will start by understanding what Interaction is and how humans behave. Afterwards, we will better investigate the widely used topic of Immersion, and its unknown and unrecognized brother Presence. From there, we will divide involvement in game play in two parts, the technological side, which relates to natural interfaces and mastery of controls, and the side of design and implementation of content, more specifically the concept of Agency and how it plays a huge part in making players feel part of the game.Só quando um videojogo é lançado é que o podemos analisar e rever. Atualmente, encontramos plataformas para partilhar a nossa opinião acerca de um videojogo, ou parte dele, em qualquer lado, com comentários positivos e negativos a serem partilhados diariamente. No entanto, o que é que faz um jogo ser visto como uma experiência positiva e quais são os componentes que satisfazem e envolvem jogadores? Nesta Dissertação, pretendemos compreender como é que jogadores percecionam um videojogo e que emoções são despoletadas que os motiva a jogar. Iremos analisar diferentes conceitos que contribuem para o jogar de um videojogo. Começaremos por ver o que é a Interação e como é que o ser humano se comporta e age. Prosseguindo, iremos analisar o já bastante usado conceito de Imersão, e o seu desconhecido e menos reconhecido irmão, Presença. Daí iremos dividir o envolvimento com um videojogo em duas partes, no lado tecnológico, relacionado com interfaces naturais e mestria de controlos, e no lado de design e implementação de conteúdo, mais especificamente no conceito de Agência e a maneira como esta integra os jogadores no jogo

Expanding Chinese Ink Painting in the 2014 Exhibition - Unscrolled: Reframing Tradition in Chinese Contemporary Art

This thesis examines installation works by three Chinese artists included in the group exhibition Unscrolled: Reframing Tradition in Chinese Contemporary Art at the Vancouver Art Gallery, November 15, 2014 to April 6, 2015: Xu Bing’s Background Story (2014), Jennifer Wen Ma’s Black Beauty: A Living Totem (2014), and Sun Xun’s Shan Shui – Cosmos (2014). These works reconfigured the traditions and conventions of Chinese landscape ink painting using new media and techniques such as video, animation, and installation, and in so doing, defied stereotypical interpretations of Chinese art. This was achieved through three main artistic strategies: imitation, materiality, and the juxtaposition of traditional and contemporary art media. The first strategy, as seen in Xu’s work, demonstrates the unconventional use of materials to simulate the appearance of a traditional Chinese landscape painting, thereby leading viewers to misrecognize the installation as an ink painting. The second strategy, as adopted by Ma, involves a display of materiality and physicality through exploration of ink aesthetics. Finally, Sun’s installation environment combines traditional ink painting with new media to reimagine new possibilities for the Chinese landscape genre. By focusing on these artists and discussing their practices in relation to the rest of the exhibition, this thesis concludes that the works subvert essentialist views of Chineseness and challenge cultural assumptions of the viewer in order to reflect on contemporary social and urban realities in modern-day China

Reconstructing plant architecture from 3D laser scanner data

En infographie, les modèles virtuels de plantes sont de plus en plus réalistes visuellement. Cependant, dans le contexte de la biologie et l'agronomie, l'acquisition de modèles précis de plantes réelles reste un problème majeur pour la construction de modèles quantitatifs du développement des plantes. Récemment, des scanners laser 3D permettent d'acquérir des images 3D avec pour chaque pixel une profondeur correspondant à la distance entre le scanner et la surface de l'objet visé. Cependant, une plante est généralement un ensemble important de petites surfaces sur lesquelles les méthodes classiques de reconstruction échouent. Dans cette thèse, nous présentons une méthode pour reconstruire des modèles virtuels de plantes à partir de scans laser. Mesurer des plantes avec un scanner laser produit des données avec différents niveaux de précision. Les scans sont généralement denses sur la surface des branches principales mais recouvrent avec peu de points les branches fines. Le cur de notre méthode est de créer itérativement un squelette de la structure de la plante en fonction de la densité locale de points. Pour cela, une méthode localement adaptative a été développée qui combine une phase de contraction et un algorithme de suivi de points. Nous présentons également une procédure d'évaluation quantitative pour comparer nos reconstructions avec des structures reconstruites par des experts de plantes réelles. Pour cela, nous explorons d'abord l'utilisation d'une distance d'édition entre arborescence. Finalement, nous formalisons la comparaison sous forme d'un problème d'assignation pour trouver le meilleur appariement entre deux structures et quantifier leurs différences. (Résumé d'auteur

Immersion and interaction : creating virtual 3D worlds for stage performances

This thesis formulates an approach towards the creation of a gesture activated and body movement controlled real time virtual 3d world in a dance performance context. It investigates immersion and navigation techniques derived from modern video games and methodologies and proposes how they can be used to further involve a performer into a virtual space as well as simultaneously offer a stimulating visual spectacle for an audience. The argument presented develops through practice-based methodology and artistic production strategies in interdisciplinary and collaborative contexts. Two choreographic performance/installations are used as cases studies to demonstrate in practice the proposed methodologies. First, the interactive dance work Suna No Onna, created in collaboration with Birringer/Danjoux and the Dap Lab, investigates the use of interactive pre-rendered animations in a real time setting and in real time by incorporating wearable sensors in the performance. Secondly, the potentials offered by the sensor technology and real time rendering engines led to the “creation scene", a key scene in the choreographic installation UKIYO (Moveable Worlds). This thesis investigates the design, creation and interaction qualities of virtual 3d spaces by exploring the potentialities offered by a shared space, between an intelligent space and a dancer in a hybrid world. The methodology applied uses as a theoretical base the phenomenological approach of Merleau-Ponty and Mark Hansen‟s mixed reality paradigm proposing the concept of the “space schema", a system which replicates and embeds proprioception, perception and motility into the space fabric offering a world which “lives”, functions and interacts with the performer. The outcome of the research is the generation of an interactive, non-linear, randomized 3d virtual space that collaborates with a technologically embedded performer in creating a 3d world which evolves and transforms, driven by the performer‟s intention and agency. This research contributes to the field of interactive performance art by making transparent the methodology, the instruments and the code used, in a non-technical terminology, making it accessible for both team members with less technological expertise as well as artists aspiring to engage interactive 3d media promoting further experimentation and conceptual discussions.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Stylised procedural animation

This thesis develops a stylised procedural paradigm for computer graphics animation. Cartoon effects animations - stylised representations of natural phenomena - have presented a long-standing, difficult challenge to computer animators. We propose a framework for achieving the intricacy of effects motion with minimal animator intervention.Our approach is to construct cartoon effects by simulating the hand-drawing process through synthetic, computational means. We create a system which emulates the stylish appearance, movements of cartoon effects in both 2D and 3D environments. Our computational models achieve this by capturing the essential characteristics common to all cartoon effects: structure modelling, dynamic controlling and stylised rendering.To validate our framework, we have implemented a cartoon effects system for a range of effects including water effects, fire, smoke, rain and snow. Each effect model has its own static structure such as how the different parts are related temporarily. The flexibility of our approach is suggested most evidently by the high-level controls on shape, colour, timing and rendering on the effects. Like their hand-drawn counterparts, they move consistently while retaining the hand-crafted look.Since the movements of cartoon effects are animated procedurally, their detailed motions need not be keyframed. This thesis therefore demonstrates a powerful approach to computer animation in which the animator plays the role of a high level controller, rather than the more conventional hand-drawing slave. Our work not only achieves cartoon effects animation of un-precedented complexity, but it also provides an interesting experimental domain for related research disciplines toward more creative and expressive image synthesis in animation

Spatialising narrative pictures : transforming 2D narrative drawing/illustration to video installations

PhD Thesis 3 DVD's of animations available in print thesis only (theses can be requested for consultation from the Newcastle University Library Search Catalogue)This Ph.D. thesis is an investigation of the processes and problems, both practical and conceptual, involved in the transformation of my small-scale two-dimensional narrative drawings into video installations. The aim of this transformation was to increase the active involvement and engagement of the viewer and to enhance and open up the narrative/s within the original drawings. I use the term ‘spatialise’ for this transformation, looking particularly at three major narrative factors - character, event and space. It became apparent through the investigation that scale and position were also crucial factors. These elements are examined through creative practice and a critical body of knowledge gained from firsthand practical experience, contextualised against the historical and theoretical backdrop relating to narrative images and how images relate to spaces. As an artist coming to this inquiry from a drawing/illustration background, the three key concerns and questions were: how to transform a two-dimensional narrative illustration into an installation without losing the drawing/painting quality? When transformed into a video installation, what changes happen to the narrative and to the audience’s engagement and self-awareness, and subsequently the audience’s understanding of the narrative? How can technology, sculpture, installation, and video projection be used to develop and enhance my drawings? This desire to search for a new medium and approach for my drawing/illustration practice is in the context of both my own artistic identity and the backdrop of dramatic social transformation in China. The research has led to new insights as well as new dialogues for me - between drawing practice, my cultural identity as an artist, the narrative content of my own hand-made drawings, and comparisons between the traditions of Western/European and Chinese art. A particularly important new element for me was the idea of an ‘open narrative’ gained through spatialisation. The research therefore contributes to the field of contemporary art practice, video installation and narrative drawing through bringing together experimental video installation and a cultural critique – and by directing the audience’s self-awareness through open narrative discourse

Designing sound : procedural audio research based on the book by Andy Farnell

In procedural media, data normally acquired by measuring something, commonly described as sampling, is replaced by a set of computational rules (procedure) that defines the typical structure and/or behaviour of that thing. Here, a general approach to sound as a definable process, rather than a recording, is developed. By analysis of their physical and perceptual qualities, natural objects or processes that produce sound are modelled by digital Sounding Objects for use in arts and entertainments. This Thesis discusses different aspects of Procedural Audio introducing several new approaches and solutions to this emerging field of Sound Design.Em Media Procedimental, os dados os dados normalmente adquiridos através da medição de algo habitualmente designado como amostragem, são substituídos por um conjunto de regras computacionais (procedimento) que definem a estrutura típica, ou comportamento, desse elemento. Neste caso é desenvolvida uma abordagem ao som definível como um procedimento em vez de uma gravação. Através da análise das suas características físicas e perceptuais , objetos naturais ou processos que produzem som, são modelados como objetos sonoros digitais para utilização nas Artes e Entretenimento. Nesta Tese são discutidos diferentes aspectos de Áudio Procedimental, sendo introduzidas várias novas abordagens e soluções para o campo emergente do Design Sonoro

Intermedial Studies

Intermedial Studies provides a concise, hands-on introduction to the analysis of a broad array of texts from a variety of media – including literature, film, music, performance, news and videogames, addressing fiction and non-fiction, mass media and social media. The detailed introduction offers a short history of the field and outlines the main theoretical approaches to the field. Part I explains the approach, examining and exemplifying the dimensions that construct every media product. The following sections offer practical examples and case studies using many examples, which will be familiar to students, from Sherlock Holmes and football, to news, vlogs and videogames. This book is the only textbook taking both a theoretical and practical approach to intermedial studies. The book will be of use to students from a variety of disciplines looking at any form of adaptation, from comparative literature to film adaptations, fan fictions and spoken performances. The book equips students with the language and understanding to confidently and competently apply their own intermedial analysis to any text

Evaluation of Synthesised Sound Effects

PhDThe current fi eld of sound synthesis research presents a range of methods and approaches for synthesising a given sound. Sounds are synthesised to facilitate interaction or control of a sound, to enable sound searching through parametric control of a sound or to allow for the creation of an arti ficial nonexistent sound. In all of these cases, the ability of a synthesis technique to reproduce a desired sound is integral. This thesis uses an audio feature representation of audio to produce a sonically inspired taxonomy, based entirely on the sonic content of sound, which enables a user to search through a large set of sounds without the need for understanding of context. This provides an approach for using audio features to compare similarity between different audio effect samples in a sound effects library. This thesis then develops approaches for evaluation of synthesised sound effects. A large scale methodic subjective evaluation of synthesised sound effects is performed, evaluating a range of different synthesis methods in a range of different sound classes or sonic contexts. It is then identi fied that there are cases where synthesised sound effects can be considered as realistic as a recorded sample. An objective evaluation approach is then presented. Audio feature vectors are used to measure the relative objective similarities between two samples, and this is correlated with a perceptual evaluation of sound similarity. These objective measures are then compared based on the perceptual evaluations. Both evaluation approaches are then demonstrated in a case study of aeroacoustic sound effects, where these subjective and objective evaluation techniques are demonstrated for a speci fic case. There is no single best approach to synthesising sound effects. More consistent and rigorous evaluation methodologies will lead to a better understanding as to the advantages and disadvantages of each method. The outcome of this research suggests that further consistent perceptual and objective evaluation within the sound effect synthesis community will lead to a better understanding as to the successes and failings of existing work and thus facilitate an enhancement of current sound synthesis technologies.This work was supported by the EPSRC grant EP/M506394/1