Moveable worlds/digital scenographies

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ Intellect Ltd 2010.The mixed reality choreographic installation UKIYO explored in this article reflects an interest in scenographic practices that connect physical space to virtual worlds and explore how performers can move between material and immaterial spaces. The spatial design for UKIYO is inspired by Japanese hanamichi and western fashion runways, emphasizing the research production company's commitment to various creative crossovers between movement languages, innovative wearable design for interactive performance, acoustic and electronic sound processing and digital image objects that have a plastic as well as an immaterial/virtual dimension. The work integrates various forms of making art in order to visualize things that are not in themselves visual, or which connect visual and kinaesthetic/tactile/auditory experiences. The ‘Moveable Worlds’ in this essay are also reflections of the narrative spaces, subtexts and auditory relationships in the mutating matrix of an installation-space inviting the audience to move around and follow its sensorial experiences, drawn near to the bodies of the dancers.Brunel University, the British Council, and the Japan Foundation

The Magic Circle: An Essential Experience Through Virtual Theatre

This thesis explores the similarities of video game design elements within the world of theatrical scenic design. Using my 2019 scenic design of Marc Camoletti’s bedroom farce Boeing Boeing as the backdrop, I discuss the core concepts of game design. I then describe the scenic process of the production and the journey from concept all the way through to a virtual reality recreation of the design. The study of game design in theatre has the potential to open a wide new world of opportunities in the scenic design industry. This paper examines how the theatre can benefit from game design in several areas, such as artist and technical design, economic and financial relief throughout production, performance, and efficiency. But the primary focus is on the dramatic and positive impact that it can have on communication and the way that theatre artists can work together in a collaborative setting. For the theatrical world, game design presents a wide and exciting realm of possibilities in not only design, but for theatre as a whole

Wearable performance

This is the post-print version of the article. The official published version can be accessed from the link below - Copyright @ 2009 Taylor & FrancisWearable computing devices worn on the body provide the potential for digital interaction in the world. A new stage of computing technology at the beginning of the 21st Century links the personal and the pervasive through mobile wearables. The convergence between the miniaturisation of microchips (nanotechnology), intelligent textile or interfacial materials production, advances in biotechnology and the growth of wireless, ubiquitous computing emphasises not only mobility but integration into clothing or the human body. In artistic contexts one expects such integrated wearable devices to have the two-way function of interface instruments (e.g. sensor data acquisition and exchange) worn for particular purposes, either for communication with the environment or various aesthetic and compositional expressions. 'Wearable performance' briefly surveys the context for wearables in the performance arts and distinguishes display and performative/interfacial garments. It then focuses on the authors' experiments with 'design in motion' and digital performance, examining prototyping at the DAP-Lab which involves transdisciplinary convergences between fashion and dance, interactive system architecture, electronic textiles, wearable technologies and digital animation. The concept of an 'evolving' garment design that is materialised (mobilised) in live performance between partners originates from DAP Lab's work with telepresence and distributed media addressing the 'connective tissues' and 'wearabilities' of projected bodies through a study of shared embodiment and perception/proprioception in the wearer (tactile sensory processing). Such notions of wearability are applied both to the immediate sensory processing on the performer's body and to the processing of the responsive, animate environment. Wearable computing devices worn on the body provide the potential for digital interaction in the world. A new stage of computing technology at the beginning of the 21st Century links the personal and the pervasive through mobile wearables. The convergence between the miniaturisation of microchips (nanotechnology), intelligent textile or interfacial materials production, advances in biotechnology and the growth of wireless, ubiquitous computing emphasises not only mobility but integration into clothing or the human body. In artistic contexts one expects such integrated wearable devices to have the two-way function of interface instruments (e.g. sensor data acquisition and exchange) worn for particular purposes, either for communication with the environment or various aesthetic and compositional expressions. 'Wearable performance' briefly surveys the context for wearables in the performance arts and distinguishes display and performative/interfacial garments. It then focuses on the authors' experiments with 'design in motion' and digital performance, examining prototyping at the DAP-Lab which involves transdisciplinary convergences between fashion and dance, interactive system architecture, electronic textiles, wearable technologies and digital animation. The concept of an 'evolving' garment design that is materialised (mobilised) in live performance between partners originates from DAP Lab's work with telepresence and distributed media addressing the 'connective tissues' and 'wearabilities' of projected bodies through a study of shared embodiment and perception/proprioception in the wearer (tactile sensory processing). Such notions of wearability are applied both to the immediate sensory processing on the performer's body and to the processing of the responsive, animate environment

Resonances: The sound of performance

It is a hot summer night in August 2013, as the audience gathers near the entrance of the large Gray Hall at the south side of the former coal mine Göttelborn (Germany). The sun has set, and there is only the gray light of dusk in the performance space inside, streaming through the large glass façade, falling onto a small array of stones laid out on the floor. Additional light from a video projector streams over the stones, and a tiny figure of a dancer is seen crawling over rocks, moving in the strange, a-syncopated rhythm of jump cuts. Slowly the sound of rocks scratching against a stone surface begins to be heard, it will remain the only sound for a while, then Japanese instrumentalist Emi Watanabe steps into the empty space with her flute

Thesis Inquiry & Process: Something About Reality

This thesis documents the employment of a system of Process and Inquiry as it serves as a structural foundation for an investigation of the manner in which Reality is represented in visual portraiture. Through a vigorous exploration of the concept of Reality and its singular nature, it is hypothesized that intrinsically unique experiences could potentially be communicated through the perceptive abilities of the emotional quotient. The effects of media, timing, complexity, abstraction, and authenticity are examined for their effect on the apparent clarity of concepts transmitted in this manner. The inquiry ultimately manifests in the daily creation of self-portraiture, as well as a multimedia exhibition inspired by the theatrical arts, that speak to the communal understanding of the Human Experience

ReliTalk: Relightable Talking Portrait Generation from a Single Video

Recent years have witnessed great progress in creating vivid audio-driven portraits from monocular videos. However, how to seamlessly adapt the created video avatars to other scenarios with different backgrounds and lighting conditions remains unsolved. On the other hand, existing relighting studies mostly rely on dynamically lighted or multi-view data, which are too expensive for creating video portraits. To bridge this gap, we propose ReliTalk, a novel framework for relightable audio-driven talking portrait generation from monocular videos. Our key insight is to decompose the portrait's reflectance from implicitly learned audio-driven facial normals and images. Specifically, we involve 3D facial priors derived from audio features to predict delicate normal maps through implicit functions. These initially predicted normals then take a crucial part in reflectance decomposition by dynamically estimating the lighting condition of the given video. Moreover, the stereoscopic face representation is refined using the identity-consistent loss under simulated multiple lighting conditions, addressing the ill-posed problem caused by limited views available from a single monocular video. Extensive experiments validate the superiority of our proposed framework on both real and synthetic datasets. Our code is released in https://github.com/arthur-qiu/ReliTalk

Intermedia Remediated & the Question of Designing Discourse

New “engines of discourse” (neural networks, algorithms and other forms of artificial intelligence, combined with the devices that record and interpret viewer actions) bring to the fore rhetorical concerns that challenge discipline-based notions of process and form. We shall focus here on the tradition of intermedial art practices to better understand the ever more complex question of how to inter-relate three aspects of digital communication: authorial “intent”, the digital sign and its interactive exploration by a “spect-actor”. We shall argue that the digital sign is an extension of intermedial thinking rooted in a pre-digital, photographic practice and esthetic. The writings of several French theorists on the subject of interactive digital design will provide a context for understanding examples of “virtual art-realities”, whose specificity is staging relationships between objects and people. Keywords: Rhetoric; Discourse; Intermedia; Interactivity; Digital Sign; Esthetics; Artificial Intelligence; Behavior-based Art.</p

Attributing scientific and technical progress: the case of holography

Holography, the three-dimensional imaging technology, was portrayed widely as a paradigm of progress during its decade of explosive expansion 1964–73, and during its subsequent consolidation for commercial and artistic uses up to the mid 1980s. An unusually seductive and prolific subject, holography successively spawned scientific insights, putative applications and new constituencies of practitioners and consumers. Waves of forecasts, associated with different sponsors and user communities, cast holography as a field on the verge of success—but with the dimensions of success repeatedly refashioned. This retargeting of the subject represented a degree of cynical marketeering, but was underpinned by implicit confidence in philosophical positivism and faith in technological progressivism. Each of its communities defined success in terms of expansion, and anticipated continual progressive increase. This paper discusses the contrasting definitions of progress in holography, and how they were fashioned in changing contexts. Focusing equally on reputed ‘failures’ of some aspects of the subject, it explores the varied attributes by which success and failure were linked with progress by different technical communities. This important case illuminates the peculiar post-World War II environment that melded the military, commercial and popular engagement with scientific and technological subjects, and the competing criteria by which they assessed the products of science