71 research outputs found

    The Quest for Life and Intelligence in Digital Puppets.

    Get PDF
    Performers and artists wishing to make collaborative improvisations using three-dimensional computer graphics will encounter the following difficulty: the animation process lacks the functionality required for spontaneous, serendipitous, real-time interaction. While human motion capture makes such real-time interaction and the corresponding spontaneity possible, it lacks the expressivity required for non-realistic characterisation. My practice-based research study proposes digital puppetry as a solution to this problem. My focus is on credibility as opposed to realism: the expectation is that the characters will behave in a puppet-like manner as opposed to manifesting the aesthetics – in terms of both movement and visual design – found in human actor-based motion capture and three- dimensional animated cartoons. The practical element is motivated by three imperatives: to improvise, to collaborate and to perform live. The primary question this study sets out to answer is: Is it possible to achieve the same spontaneity and animation[1] in digital puppets as it is with tangible puppets and, if so, what are the mechanisms involved? Unable to find a three-dimensional computer graphics digital puppetry software and hardware solution, I have devised what I call the GLOPPID[2] method, where GLOPPID is an acronym for Goniometric[3] Live Organic Performance Puppetry Improvisation Digitalia. The GLOPPID method comprises an artistic approach and a practical solution in the form of a Human Digital Puppetry Interface. It uses off-the-shelf three-dimensional computer animation software, which I have incorporated into a pipeline customised to suit my creative process. This pipeline is configured to transform ready-made computer graphics models into digital puppets that can be used as collaborators, thereby allowing the performer to experience the same kind of spontaneity as is possible in physical puppet performance. My thesis asserts that it is possible to improvise with digital puppets, and I have devised my own solution in order to do this. I argue that the real-time, improvised manipulation of digital puppets offers creatively advantageous opportunities for spontaneity and expressivity. My research presents the technique of digital puppetry as an expansion of what I call the pro- puppetry thesis – the idea that puppets have dramatic advantages over human actors. It also contributes to the ontological discourses surrounding the Human-Machine Interface (HMI), trans-embodiment, the post-human, the illusion of life, and cybernetics. In addition, it explores how algorithms can be used in the arts, particularly in performance (see Kleber & Trojanowska 2019, p.101). It makes a timely contribution to the pool of knowledge, because I see digital puppets as zeitgeists – apt vehicles for human hopes and fears surrounding the digital and existential angst that is part of the fabric of 21st-century life. The professional digital puppetry practice undertaken and discussed in this study requires both general, transferable human-machine interacting skills, and the specific digital puppetry skills necessary for project phases such as rigging and manipulation. This practical approach prioritises the physical, as opposed to the psycho-physical. Informed by the theories and practice of human dramatic technique practitioners such as Decroux and Lecoq, it takes the basic building blocks of movement identifiable in the segmented anatomy and rotational articulations of the actor-as-puppet, rod, glove and string puppets, and configures them in their digital counterpart in order to accentuate the odd by means of atypical combinations, economy of motion, and asymmetry. My approach is underpinned by the idea that a puppet is uninhibited or influenced by its own ego, backstory, or emotions. Nevertheless, these properties are present in a channeled, and therefore changed, form that emanates from the Human in the Loop, namely, the digital puppeteer. In digital puppetry, the protagonist is a digital puppet operated by a human who is embedded at the core of the activity, and who simultaneously witnesses their own emotional responses as they are acted out in front of them. My work demonstrates that non-realistic, expressive approaches to movement performance derived from human physical theatre techniques (including theatre clowning), combined with the use of algorithm-assisted techniques of rigging and manipulation, mean that the puppets are not under the complete control of their puppeteers. Instead, their rogue nature and irrationality enables digital puppets to satirise and subvert notions surrounding computer-generated imagery and artificial general intelligence, while avoiding exile in the “uncanny valley” (Mori 1970). In this study the term puppet is used both literally and metaphorically. Deployed literally, puppet refers to a figure or object that is manipulated in real time in the presence of a puppeteer, in both tangible and intangible material formats. The term is used metaphorically in the sense that a puppet can be seen as an analogue of a human being that acts as a mirror, reflecting aspects of the human condition or predicament. My work expands upon an understanding of the quest for ‘life’ in physical puppets and the corresponding development of their ur-narrative – as described by Kohler & Jones (2009, p.346) – and extends this to include digital puppets. By combining the concept of this quest with Rokeby’s idea of technology as a “prosthetic of philosophy” (2019, p.107) and with the notion of art as a mirror, I explore how distinctive features present in digital puppetry practice can be used to express truths about being human. The work employs a practice-as-research methodology that provides moments for reflection during the creative process, and reflection on the creative outcomes: reflection-in-action and reflection-on-action. The thesis can be expressed in the form of the following formula: Extemporising with puppets + the ur-narrative of puppets + art as a mirror + technology as a mirror = a pro-digital-puppetry thesis on HMI, AGI + what it is to be human [1] The term animation used here refers not to the techniques of incremental, frame-by-frame posing of characters, but to the act of bringing them to life. [2] ‘Gloppid’ is also the name of a glove-puppet character invented for an ecologically-based travelling show performed during the 1980s (Childs 1988). [3] Goniometric refers to the measurement of the range of motion in a joint

    “What can’t be coded can be decorded” Reading Writing Performing Finnegans Wake

    Get PDF
    This thesis examines the ways in which performances of James Joyce’s Finnegans Wake (1939) navigate the boundary between reading and writing. I consider the extent to which performances enact alternative readings of Finnegans Wake, challenging notions of competence and understanding; and by viewing performance as a form of writing I ask whether Joyce’s composition process can be remembered by its recomposition into new performances. These perspectives raise questions about authority and archivisation, and I argue that performances of Finnegans Wake challenge hierarchical and institutional forms of interpretation. By appropriating Joyce’s text through different methodologies of reading and writing I argue that these performances come into contact with a community of ghosts and traces which haunt its composition. In chapter one I argue that performance played an important role in the composition and early critical reception of Finnegans Wake and conduct an overview of various performances which challenge the notion of a ‘Joycean competence’ or encounter the text through radical recompositions of its material. In chapter two I discuss Mary Manning’s The Voice of Shem (1955) and find that its theatrical reassembling of the text served as a competent reading of the Wake’s form as an alternative to contemporary studies of the book, and that its specific ‘redistribution’ of the text accessed affective and genetic elements that were yet to be explored in Joyce scholarship. In chapter three I consider several decompositions of the Wake by John Cage (1975-1983) and find that by paying attention to the materiality of the book rather than its ‘plot’ or ‘meaning’ his performances reencountered the work concealed in Finnegans Wake’s composition. In chapter four, I document and analyse my own performance, About That Original Hen (2014), a ‘research-as-performance’ lecture which re-enacts a visit to the James Joyce Archive. By reconfiguring Finnegans Wake in relation to a marginal figure from its composition process and a contemporary act of protest within the university, this performance explores how a diachronic re-animation of archival materials can engage with the ghosts which haunt its composition and enact a political reading of the text’s production and subsequent archivisation. I conclude the thesis by arguing that these performances repeat the contingencies, misreadings and appropriations and collective acts of reading and writing that were integral to the composition of Finnegans Wake

    FACING EXPERIENCE: A PAINTER’S CANVAS IN VIRTUAL REALITY

    Get PDF
    Full version unavailable due to 3rd party copyright restrictions.This research investigates how shifts in perception might be brought about through the development of visual imagery created by the use of virtual environment technology. Through a discussion of historical uses of immersion in art, this thesis will explore how immersion functions and why immersion has been a goal for artists throughout history. It begins with a discussion of ancient cave drawings and the relevance of Plato’s Allegory of the Cave. Next it examines the biological origins of “making special.” The research will discuss how this concept, combined with the ideas of “action” and “reaction,” has reinforced the view that art is fundamentally experiential rather than static. The research emphasizes how present-day virtual environment art, in providing a space that engages visitors in computer graphics, expands on previous immersive artistic practices. The thesis examines the technical context in which the research occurs by briefly describing the use of computer science technologies, the fundamentals of visual arts practices, and the importance of aesthetics in new media and provides a description of my artistic practice. The aim is to investigate how combining these approaches can enhance virtual environments as artworks. The computer science of virtual environments includes both hardware and software programming. The resultant virtual environment experiences are technologically dependent on the types of visual displays being used, including screens and monitors, and their subsequent viewing affordances. Virtual environments fill the field of view and can be experienced with a head mounted display (HMD) or a large screen display. The sense of immersion gained through the experience depends on how tracking devices and related peripheral devices are used to facilitate interaction. The thesis discusses visual arts practices with a focus on how illusions shift our cognition and perception in the visual modalities. This discussion includes how perceptual thinking is the foundation of art experiences, how analogies are the foundation of cognitive experiences and how the two intertwine in art experiences for virtual environments. An examination of the aesthetic strategies used by artists and new media critics are presented to discuss new media art. This thesis investigates the visual elements used in virtual environments and prescribes strategies for creating art for virtual environments. Methods constituting a unique virtual environment practice that focuses on visual analogies are discussed. The artistic practice that is discussed as the basis for this research also concentrates on experiential moments and shifts in perception and cognition and references Douglas Hofstadter, Rudolf Arnheim and John Dewey. iv Virtual environments provide for experiences in which the imagery generated updates in real time. Following an analysis of existing artwork and critical writing relative to the field, the process of inquiry has required the creation of artworks that involve tracking systems, projection displays, sound work, and an understanding of the importance of the visitor. In practice, the research has shown that the visitor should be seen as an interlocutor, interacting from a first-person perspective with virtual environment events, where avatars or other instrumental intermediaries, such as guns, vehicles, or menu systems, do not to occlude the view. The aesthetic outcomes of this research are the result of combining visual analogies, real time interactive animation, and operatic performance in immersive space. The environments designed in this research were informed initially by paintings created with imagery generated in a hypnopompic state or during the moments of transitioning from sleeping to waking. The drawings often emphasize emotional moments as caricatures and/or elements of the face as seen from a number of perspectives simultaneously, in the way of some cartoons, primitive artwork or Cubist imagery. In the imagery, the faces indicate situations, emotions and confrontations which can offer moments of humour and reflective exploration. At times, the faces usurp the space and stand in representation as both face and figure. The power of the placement of the caricatures in the paintings become apparent as the imagery stages the expressive moment. The placement of faces sets the scene, establishes relationships and promotes the honesty and emotions that develop over time as the paintings are scrutinized. The development process of creating virtual environment imagery starts with hand drawn sketches of characters, develops further as paintings on “digital canvas”, are built as animated, three-dimensional models and finally incorporated into a virtual environment. The imagery is generated while drawing, typically with paper and pencil, in a stream of consciousness during the hypnopompic state. This method became an aesthetic strategy for producing a snappy straightforward sketch. The sketches are explored further as they are worked up as paintings. During the painting process, the figures become fleshed out and their placement on the page, in essence brings them to life. These characters inhabit a world that I explore even further by building them into three dimensional models and placing them in computer generated virtual environments. The methodology of developing and placing the faces/figures became an operational strategy for building virtual environments. In order to open up the range of art virtual environments, and develop operational strategies for visitors’ experience, the characters and their facial features are used as navigational strategies, signposts and methods of wayfinding in order to sustain a stream of consciousness type of navigation. Faces and characters were designed to represent those intimate moments of self-reflection and confrontation that occur daily within ourselves and with others. They sought to reflect moments of wonderment, hurt, curiosity and humour that could subsequently be relinquished for more practical or purposeful endeavours. They were intended to create conditions in which visitors might reflect upon their emotional state, v enabling their understanding and trust of their personal space, in which decisions are made and the nature of world is determined. In order to extend the split-second, frozen moment of recognition that a painting affords, the caricatures and their scenes are given new dimensions as they become characters in a performative virtual reality. Emotables, distinct from avatars, are characters confronting visitors in the virtual environment to engage them in an interactive, stream of consciousness, non-linear dialogue. Visitors are also situated with a role in a virtual world, where they were required to adapt to the language of the environment in order to progress through the dynamics of a drama. The research showed that imagery created in a context of whimsy and fantasy could bring ontological meaning and aesthetic experience into the interactive environment, such that emotables or facially expressive computer graphic characters could be seen as another brushstroke in painting a world of virtual reality

    Digital Theatre: A "Live" and Mediated Art Form Expanding Perceptions of Body, Place, and Community

    Get PDF
    This work discusses Digital Theatre, a type of performance which utilizes both "live" actors and co-present audiences along with digital media to create a hybrid art form revitalizing theatre for contemporary audiences. This work surveys a wide range of digital performances (with "live" and digital elements, limited interactivity/participation and spoken words) and identifies the group collectively as Digital Theatre, an art form with the flexibility and reach of digital data and the sense of community found in "live" theatre. I offer performance examples from Mark Reaney, David Saltz, Troika Ranch, Gertrude Stein Repertory Theatre, Flying Karamazov Brothers, Talking Birds, Yacov Sharir, Studio Z, George Coates Performance Group, and ArtGrid. (The technologies utilized in performances include: video-conferencing, media projection, MIDI control, motion capture, VR animation, and AI). Rather than looking at these productions as isolated events, I identify them as a movement and link the use of digital techniques to continuing theatrical tradition of utilizing new technologies on the stage. The work ties many of the aesthetic choices explored in theatrical past by the likes of Piscator, Svoboda, Craig, and in Bauhaus and Futurist movements. While it retains the essential qualities of public human connection and imaginative thought central to theatre, Digital Theatre can cause theatrical roles to merge as it extends the performer's body, expands our concept of place, and creates new models of global community

    Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

    Get PDF
    This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. [...] It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. [...] To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices [....] and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm

    From motion capture to interactive virtual worlds : towards unconstrained motion-capture algorithms for real-time performance-driven character animation

    Get PDF
    This dissertation takes performance-driven character animation as a representative application and advances motion capture algorithms and animation methods to meet its high demands. Existing approaches have either coarse resolution and restricted capture volume, require expensive and complex multi-camera systems, or use intrusive suits and controllers. For motion capture, set-up time is reduced using fewer cameras, accuracy is increased despite occlusions and general environments, initialization is automated, and free roaming is enabled by egocentric cameras. For animation, increased robustness enables the use of low-cost sensors input, custom control gesture definition is guided to support novice users, and animation expressiveness is increased. The important contributions are: 1) an analytic and differentiable visibility model for pose optimization under strong occlusions, 2) a volumetric contour model for automatic actor initialization in general scenes, 3) a method to annotate and augment image-pose databases automatically, 4) the utilization of unlabeled examples for character control, and 5) the generalization and disambiguation of cyclical gestures for faithful character animation. In summary, the whole process of human motion capture, processing, and application to animation is advanced. These advances on the state of the art have the potential to improve many interactive applications, within and outside virtual reality.Diese Arbeit befasst sich mit Performance-driven Character Animation, insbesondere werden Motion Capture-Algorithmen entwickelt um den hohen Anforderungen dieser Beispielanwendung gerecht zu werden. Existierende Methoden haben entweder eine geringe Genauigkeit und einen eingeschrĂ€nkten Aufnahmebereich oder benötigen teure Multi-Kamera-Systeme, oder benutzen störende Controller und spezielle AnzĂŒge. FĂŒr Motion Capture wird die Setup-Zeit verkĂŒrzt, die Genauigkeit fĂŒr Verdeckungen und generelle Umgebungen erhöht, die Initialisierung automatisiert, und BewegungseinschrĂ€nkung verringert. FĂŒr Character Animation wird die Robustheit fĂŒr ungenaue Sensoren erhöht, Hilfe fĂŒr benutzerdefinierte Gestendefinition geboten, und die AusdrucksstĂ€rke der Animation verbessert. Die wichtigsten BeitrĂ€ge sind: 1) ein analytisches und differenzierbares Sichtbarkeitsmodell fĂŒr Rekonstruktionen unter starken Verdeckungen, 2) ein volumetrisches Konturenmodell fĂŒr automatische Körpermodellinitialisierung in genereller Umgebung, 3) eine Methode zur automatischen Annotation von Posen und Augmentation von Bildern in großen Datenbanken, 4) das Nutzen von Beispielbewegungen fĂŒr Character Animation, und 5) die Generalisierung und Übertragung von zyklischen Gesten fĂŒr genaue Charakteranimation. Es wird der gesamte Prozess erweitert, von Motion Capture bis hin zu Charakteranimation. Die Verbesserungen sind fĂŒr viele interaktive Anwendungen geeignet, innerhalb und außerhalb von virtueller RealitĂ€t

    From motion capture to interactive virtual worlds : towards unconstrained motion-capture algorithms for real-time performance-driven character animation

    Get PDF
    This dissertation takes performance-driven character animation as a representative application and advances motion capture algorithms and animation methods to meet its high demands. Existing approaches have either coarse resolution and restricted capture volume, require expensive and complex multi-camera systems, or use intrusive suits and controllers. For motion capture, set-up time is reduced using fewer cameras, accuracy is increased despite occlusions and general environments, initialization is automated, and free roaming is enabled by egocentric cameras. For animation, increased robustness enables the use of low-cost sensors input, custom control gesture definition is guided to support novice users, and animation expressiveness is increased. The important contributions are: 1) an analytic and differentiable visibility model for pose optimization under strong occlusions, 2) a volumetric contour model for automatic actor initialization in general scenes, 3) a method to annotate and augment image-pose databases automatically, 4) the utilization of unlabeled examples for character control, and 5) the generalization and disambiguation of cyclical gestures for faithful character animation. In summary, the whole process of human motion capture, processing, and application to animation is advanced. These advances on the state of the art have the potential to improve many interactive applications, within and outside virtual reality.Diese Arbeit befasst sich mit Performance-driven Character Animation, insbesondere werden Motion Capture-Algorithmen entwickelt um den hohen Anforderungen dieser Beispielanwendung gerecht zu werden. Existierende Methoden haben entweder eine geringe Genauigkeit und einen eingeschrĂ€nkten Aufnahmebereich oder benötigen teure Multi-Kamera-Systeme, oder benutzen störende Controller und spezielle AnzĂŒge. FĂŒr Motion Capture wird die Setup-Zeit verkĂŒrzt, die Genauigkeit fĂŒr Verdeckungen und generelle Umgebungen erhöht, die Initialisierung automatisiert, und BewegungseinschrĂ€nkung verringert. FĂŒr Character Animation wird die Robustheit fĂŒr ungenaue Sensoren erhöht, Hilfe fĂŒr benutzerdefinierte Gestendefinition geboten, und die AusdrucksstĂ€rke der Animation verbessert. Die wichtigsten BeitrĂ€ge sind: 1) ein analytisches und differenzierbares Sichtbarkeitsmodell fĂŒr Rekonstruktionen unter starken Verdeckungen, 2) ein volumetrisches Konturenmodell fĂŒr automatische Körpermodellinitialisierung in genereller Umgebung, 3) eine Methode zur automatischen Annotation von Posen und Augmentation von Bildern in großen Datenbanken, 4) das Nutzen von Beispielbewegungen fĂŒr Character Animation, und 5) die Generalisierung und Übertragung von zyklischen Gesten fĂŒr genaue Charakteranimation. Es wird der gesamte Prozess erweitert, von Motion Capture bis hin zu Charakteranimation. Die Verbesserungen sind fĂŒr viele interaktive Anwendungen geeignet, innerhalb und außerhalb von virtueller RealitĂ€t

    Designing interactive technology for cross-cultural appreciation of intangible cultural heritage: Chinese traditional painting and puppetry

    Get PDF
    Ph. D. ThesisDigital heritage is becoming a significant component of cultural heritage, and cultural organisations are increasingly using interactive technologies to showcase and safeguard heritage assets. However, few studies focus on using interactive technology to enhance the appreciation of Intangible Cultural Heritage (ICH) amongst cross-cultural audiences. This dissertation explores the design of interactive technologies to support the cultural appreciation, learning, and experience of Chinese ICH. In addition, the research seeks to explore the value of Human-Computer Interaction (HCI) design strategies in supporting the appreciation of ICH. The research uses HCI design strategies to specifically explore how interactive technology might be effectively utilised in two case-study contexts, supporting traditional Chinese painting and traditional Chinese puppetry. To this end, in stage one of the research, a qualitative study involving interviews, workshops, and fieldwork for design was undertaken with potential cross-cultural audiences and both Chinese and international painting and puppetry practitioners. Based on the results of these studies, several suggestions were developed for safeguarding ICH across cultural boundaries. In the next stage of the research, two interactive applications were designed and deployed that supported cross-cultural audiences’ appreciation of traditional of ICH. One application explored Chinese painting, the other Chinese puppetry. Using both qualitative and quantitative methods, studies were conducted that examined the efficacy of both applications and offered suggestions for a holistic approach to cross-cultural appreciation through the use of interactive applications. The analysis focuses on the use of element-based archiving to increase aesthetic appreciation, gestural/tangible interfaces for cultural engagement, and the use of interactive access to inspire self-expression and collaborative appreciation. Finally, this research relies on practical methods to deconstruct cultural elements from the HCI perspective and enhance the cross-cultural appreciation of Chinese ICH. It thus provides a framework for assisting non-Chinese people to better understand the cultural significance of Chinese ICH. The findings have design implications for both HCI researchers and digital heritage researchers

    Masks praxis: theories and practices in modern drama

    Get PDF
    Mask Praxis is an investigation of the theories and practices behind the uses of the mask in modern drama from 1896 to 2004. The study traces the crisis in humanism through the use of idealist and materialist masks by theatre practitioners and explains how the search for a unified field was overlaid by fractured identities and a slide into dissonance. How important are the masks that people adopt on the stage for understanding their actions in society? How does the metaphorical power and perceptual ambiguity of the mask correlate with intentions of its maker and performer? What is the relationship between the mask and the face of the actor, and what does the mask do that cannot be done unaided? What are the main approaches to actor training that have used masks, and how are these training systems connected to wider belief systems? What do we learn from the act of masking about self-perception and social being, and what are the principal considerations that this gives rise to? This investigation proceeds from a consideration of major theories and practices. Chapter 1 examines mask performance theories, conventions, and typologies. Chapter 2 analyses the specificity of the mask, materials and methods, representative mask-makers and provides casebook studies on the Sartori family and the Masks for Menander Project. Chapter 3 evaluates actor-training under the mask from Copeau to Lecoq. Chapter 4 assesses the masks of idealist modernism and Chapter 5 considers the masks of materialist modernism. The final chapter is dedicated to transnational flows, multinational productions and the notion of connectivity. It brings new evidence to bear on the emergent field of masks, puppets and performing objects and sets down a major overview of the mask as a primary iconographic tool and as a liminoid instrument from which to mediate and direct the flow of power in a system
    • 

    corecore