Moveable worlds/digital scenographies

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ Intellect Ltd 2010.The mixed reality choreographic installation UKIYO explored in this article reflects an interest in scenographic practices that connect physical space to virtual worlds and explore how performers can move between material and immaterial spaces. The spatial design for UKIYO is inspired by Japanese hanamichi and western fashion runways, emphasizing the research production company's commitment to various creative crossovers between movement languages, innovative wearable design for interactive performance, acoustic and electronic sound processing and digital image objects that have a plastic as well as an immaterial/virtual dimension. The work integrates various forms of making art in order to visualize things that are not in themselves visual, or which connect visual and kinaesthetic/tactile/auditory experiences. The ‘Moveable Worlds’ in this essay are also reflections of the narrative spaces, subtexts and auditory relationships in the mutating matrix of an installation-space inviting the audience to move around and follow its sensorial experiences, drawn near to the bodies of the dancers.Brunel University, the British Council, and the Japan Foundation

Aesthetic choices: Defining the range of aesthetic views in interactive digital media including games and 3D virtual environments (3D VEs)

Defining aesthetic choices for interactive digital media such as games is a challenging task. Objective and subjective factors such as colour, symmetry, order and complexity, and statistical features among others play an important role for defining the aesthetic properties of interactive digital artifacts. Computational approaches developed in this regard also consider objective factors such as statistical image features for the assessment of aesthetic qualities. However, aesthetics for interactive digital media, such as games, requires more nuanced consideration than simple objective and subjective factors, for choosing a range of aesthetic features. From the study it was found that the there is no one single optimum position or viewpoint with a corresponding relationship to the aesthetic considerations that influence interactive digital media. Instead, the incorporation of aesthetic features demonstrates the need to consider each component within interactive digital media as part of a range of possible features, and therefore within a range of possible camera positions. A framework, named as PCAWF, emphasized that combination of features and factors demonstrated the need to define a range of aesthetic viewpoints. This is important for improved user experience. From the framework it has been found that factors including the storyline, user state, gameplay, and application type are critical to defining the reasons associated with making aesthetic choices. The selection of a range of aesthetic features and characteristics is influenced by four main factors and sub-factors associated with the main factors. This study informs the future of interactive digital media interaction by providing clarity and reasoning behind the aesthetic decision-making inclusions that are integrated into automatically generated vision by providing a framework for choosing a range of aesthetic viewpoints in a 3D virtual environment of a game. The study identifies critical juxtapositions between photographic and cinema-based media aesthetics by incorporating qualitative rationales from experts within the interactive digital media field. This research will change the way Artificial Intelligence (AI) generated interactive digital media in the way that it chooses visual outputs in terms of camera positions, field-view, orientation, contextual considerations, and user experiences. It will impact across all automated systems to ensure that human-values, rich variations, and extensive complexity are integrated in the AI-dominated development and design of future interactive digital media production

Literacy for digital futures : Mind, body, text

The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualised, embodied, multimodal, and digitally mediated. In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes – Mind and Materiality; Body and Senses; and Texts and Digital Semiotics – to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice

From corporeality to virtual reality: theorizing literacy, bodies, and technology in the emerging media of virtual, augmented, and mixed realities

This dissertation explores the relationships between literacy, technology, and bodies in the emerging media of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). In response to the recent, rapid emergence of new media forms, questions arise as to how and why we should prepare to compose in new digital media. To interrogate the newness accorded to new media composing, I historicize the literacy practices demanded by new media by examining digital texts, such as video games and software applications, alongside analogous “antiquated” media, such as dioramas and museum exhibits. Comparative textual analysis of analogous digital and non-digital VR, AR, and MR texts reveals new media and “antiquated” media utilize common characteristics of dimensionality, layering, and absence/presence, respectively. The establishment of shared traits demonstrates how media operate on a continuum of mutually held textual practices; despite their distinctive forms, new media texts do not represent either a hierarchical or linear progression of maturing development. Such an understanding aids composing in new VR, AR, and MR media by enabling composers to make fuller use of prior knowledge in a rapidly evolving new media environment, a finding significant both for educators and communicators. As these technologies mature, we will continue to compose both traditional and new forms of texts. As such, we need literacy theory that attends to both the traditional and the new and also is comprehensive enough to encompass future acts of composing in media yet to emerge

Under construction – contemporary opera in the crossroads between new aesthetics, techniques, and technologies

Despite of its long history, opera as an art form is constantly evolving. Composers have never lost their fascination about it and keep exploring with innovative aesthetics, techniques, and modes of expression. New technologies, such as Virtual Reality (VR), Robotics and Artificial Intelligence (AI) are steadily having an impact upon the world of opera. The evolving use of performance-based software such as Ableton Live and Max/MSP has created new and exciting compositional techniques that intertwine theatrical and musical performance. This paper presents some initial work on the development of an opera using such technologies that is being composed by Kallionpää and Chamberlain. Furthermore, it presents two composition case studies by Kallionpää: “She” (2017) and puppet opera “Croak” (2018), as well as their documentation within the world's first 360° 3D VR recordings with full spatial audio in third-order Ambisonics and the application of an unmixing paradigm for focusing and isolating individual voices

Drawing from motion capture : developing visual languages of animation

The work presented in this thesis aims to explore novel approaches of combining motion capture with drawing and 3D animation. As the art form of animation matures, possibilities of hybrid techniques become more feasible, and crosses between traditional and digital media provide new opportunities for artistic expression. 3D computer animation is used for its keyframing and rendering advancements, that result in complex pipelines where different areas of technical and artistic specialists contribute to the end result. Motion capture is mostly used for realistic animation, more often than not for live-action filmmaking, as a visual effect. Realistic animated films depend on retargeting techniques, designed to preserve actors performances with a high degree of accuracy. In this thesis, we investigate alternative production methods that do not depend on retargeting, and provide animators with greater options for experimentation and expressivity. As motion capture data is a great source for naturalistic movements, we aim to combine it with interactive methods such as digital sculpting and 3D drawing. As drawing is predominately used in preproduction, in both the case of realistic animation and visual effects, we embed it instead to alternative production methods, where artists can benefit from improvisation and expression, while emerging in a three-dimensional environment. Additionally, we apply these alternative methods for the visual development of animation, where they become relevant for the creation of specific visual languages that can be used to articulate concrete ideas for storytelling in animation

Historic Costume Simulation and its Application

This study highlights the potential of new technology as a means to provide new possibility for costumes in fragile condition to be utilised. The aim of this study is to create accurate digital duplicates of costumes from historical sources, and to explore the possibility of developing them as an exhibitory and educational method applying 3D apparel CAD and new media. To achieve this, three attributes for qualities of effective digital costumes were suggested: faithful reproduction, virtual fabrication, and interactive and stereographic appreciation. Based on these qualities, digital costumes and a PC application were produced and evaluated