Neural Voice Puppetry: Audio-driven Facial Reenactment

We present Neural Voice Puppetry, a novel approach for audio-driven facial video synthesis. Given an audio sequence of a source person or digital assistant, we generate a photo-realistic output video of a target person that is in sync with the audio of the source input. This audio-driven facial reenactment is driven by a deep neural network that employs a latent 3D face model space. Through the underlying 3D representation, the model inherently learns temporal stability while we leverage neural rendering to generate photo-realistic output frames. Our approach generalizes across different people, allowing us to synthesize videos of a target actor with the voice of any unknown source actor or even synthetic voices that can be generated utilizing standard text-to-speech approaches. Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head. We demonstrate the capabilities of our method in a series of audio- and text-based puppetry examples. Our method is not only more general than existing works since we are generic to the input person, but we also show superior visual and lip sync quality compared to photo-realistic audio- and video-driven reenactment techniques

What I could make of mothering: matricentric puppetry and the mothers of the bestiary

Published versio

The 'Dark Continent' goes north: an exploration of intercultural theatre practice through Handspring and Sogolon Puppet Companies' production of Tall Horse

This essay explores the complexities of intercultural interaction, specifically in the context of globalization. These interactions involve not only contact with, but also negotiation of cultural representations. The debates about the processes involved in such encounters are complex and highlight tensions among aesthetics, ideology, the ethics of production, voice, and authorship. The essay begins by outlining some of the key debates and issues specifically for theatre; in particular, it looks at the tension between Brook’s transcultural approach to intercultural theatre and Rustom Bharucha’s insistence on contextualized and historicized interactions. These theoretical positions are explored against the specific example of Tall Horse (2005), an intercultural production by the South African Handspring Puppet Company, the Malian Sogolon Puppet Company, a choreographer from Benin, and a scriptwriter from New York. The essay examines both the ideological issues raised in the text and the practical issues of cross-cultural collaboration and interaction to suggest an approach that may mediate between binaries that seem to dominate cultural interaction

Roots Reloaded. Culture, Identity and Social Development in the Digital Age

This edited volume is designed to explore different perspectives of culture, identity and social development using the impact of the digital age as a common thread, aiming at interdisciplinary audiences. Cases of communities and individuals using new technology as a tool to preserve and explore their cultural heritage alongside new media as a source for social orientation ranging from language acquisition to health-related issues will be covered. Therefore, aspects such as Art and Cultural Studies, Media and Communication, Behavioral Science, Psychology, Philosophy and innovative approaches used by creative individuals are included. From the Aboriginal tribes of Australia, to the Maoris of New Zealand, to the mystical teachings of Sufi brotherhoods, the significance of the oral and written traditions and their current relation to online activities shall be discussed in the opening article. The book continues with a closer look at obesity awareness support groups and their impact on social media, Facebook usage in language learning context, smartphone addiction and internet dependency, as well as online media reporting of controversial ethical issues. The Digital progress has already left its dominating mark as the world entered the 21st century. Without a doubt, as technology continues its ascent, society will be faced with new and altering values in an effort to catch-up with this extraordinary Digitization, adapt satisfactorily in order to utilize these strong developments in everyday life

Theodora Skipitares’s textual bodies

Accepted manuscrip

Theodora Skipitares’s textual bodies

Accepted manuscrip

Emotional Voice Puppetry

The paper presents emotional voice puppetry, an audio-based facial animation approach to portray characters with vivid emotional changes. The lips motion and the surrounding facial areas are controlled by the contents of the audio, and the facial dynamics are established by category of the emotion and the intensity. Our approach is exclusive because it takes account of perceptual validity and geometry instead of pure geometric processes. Another highlight of our approach is the generalizability to multiple characters. The findings showed that training new secondary characters when the rig parameters are categorized as eye, eyebrows, nose, mouth, and signature wrinkles is significant in achieving better generalization results compared to joint training. User studies demonstrate the effectiveness of our approach both qualitatively and quantitatively. Our approach can be applicable in AR/VR and 3DUI, namely, virtual reality avatars/self-avatars, teleconferencing and in-game dialogue

The Making of Faulty Optic's Dead Wedding: Inertia, Chaos and Adaptation

An examination of Faulty Optic's creative process during the devising and construction of their show Dead Wedding. Published by Palgrave Macmillan as Chapter 3 in 'Devising in Process' edited by Alex Mermikides and Jackie Smart, 201