Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.

Pinching sweaters on your phone – iShoogle : multi-gesture touchscreen fabric simulator using natural on-fabric gestures to communicate textile qualities

The inability to touch fabrics online frustrates consumers, who are used to evaluating physical textiles by engaging in complex, natural gestural interactions. When customers interact with physical fabrics, they combine cross-modal information about the fabric's look, sound and handle to build an impression of its physical qualities. But whenever an interaction with a fabric is limited (i.e. when watching clothes online) there is a perceptual gap between the fabric qualities perceived digitally and the actual fabric qualities that a person would perceive when interacting with the physical fabric. The goal of this thesis was to create a fabric simulator that minimized this perceptual gap, enabling accurate perception of the qualities of fabrics presented digitally. We designed iShoogle, a multi-gesture touch-screen sound-enabled fabric simulator that aimed to create an accurate representation of fabric qualities without the need for touching the physical fabric swatch. iShoogle uses on-screen gestures (inspired by natural on-fabric movements e.g. Crunching) to control pre-recorded videos and audio of fabrics being deformed (e.g. being Crunched). iShoogle creates an illusion of direct video manipulation and also direct manipulation of the displayed fabric. This thesis describes the results of nine studies leading towards the development and evaluation of iShoogle. In the first three studies, we combined expert and non-expert textile-descriptive words and grouped them into eight dimensions labelled with terms Crisp, Hard, Soft, Textured, Flexible, Furry, Rough and Smooth. These terms were used to rate fabric qualities throughout the thesis. We observed natural on-fabric gestures during a fabric handling study (Study 4) and used the results to design iShoogle's on-screen gestures. In Study 5 we examined iShoogle's performance and speed in a fabric handling task and in Study 6 we investigated users' preferences for sound playback interactivity. iShoogle's accuracy was then evaluated in the last three studies by comparing participants’ ratings of textile qualities when using iShoogle with ratings produced when handling physical swatches. We also described the recording and processing techniques for the video and audio content that iShoogle used. Finally, we described the iShoogle iPhone app that was released to the general public. Our evaluation studies showed that iShoogle significantly improved the accuracy of fabric perception in at least some cases. Further research could investigate which fabric qualities and which fabrics are particularly suited to be represented with iShoogle

The Translocal Event and the Polyrhythmic Diagram

This thesis identifies and analyses the key creative protocols in translocal performance practice, and ends with suggestions for new forms of transversal live and mediated performance practice, informed by theory. It argues that ontologies of emergence in dynamic systems nourish contemporary practice in the digital arts. Feedback in self-organised, recursive systems and organisms elicit change, and change transforms. The arguments trace concepts from chaos and complexity theory to virtual multiplicity, relationality, intuition and individuation (in the work of Bergson, Deleuze, Guattari, Simondon, Massumi, and other process theorists). It then examines the intersection of methodologies in philosophy, science and art and the radical contingencies implicit in the technicity of real-time, collaborative composition. Simultaneous forces or tendencies such as perception/memory, content/ expression and instinct/intellect produce composites (experience, meaning, and intuition- respectively) that affect the sensation of interplay. The translocal event is itself a diagram - an interstice between the forces of the local and the global, between the tendencies of the individual and the collective. The translocal is a point of reference for exploring the distribution of affect, parameters of control and emergent aesthetics. Translocal interplay, enabled by digital technologies and network protocols, is ontogenetic and autopoietic; diagrammatic and synaesthetic; intuitive and transductive. KeyWorx is a software application developed for realtime, distributed, multimodal media processing. As a technological tool created by artists, KeyWorx supports this intuitive type of creative experience: a real-time, translocal “jamming” that transduces the lived experience of a “biogram,” a synaesthetic hinge-dimension. The emerging aesthetics are processual – intuitive, diagrammatic and transversal

Investigation into the design of educational multimedia : video, interactivity and narrative

This study critically examines the design of multimedia in education. The study begins by reviewing existing media in education and then uses a series of empirical studies to uncover, and then examine the key issues for the design of educational multimedia.The research involves two studies. The first, a preliminary study, that identifies specific areas of interest for the research. This study looks at the existing use of an interactive video disc program for training in Price Waterhouse. The literature search, combined with the outcomes of the preliminary study, identified the areas for further research as: the use of video, forms of interaction, and the role of narrative. The main study examines these areas using two phases. The first phase analyses three treatments of the same educational text, on linear video, multimedia, and structured multimedia, each treatment maintaining the same content and narrative structure, but differing in presentation and control. The second phases builds on the results of the first with the creation and analysis of an interactive multimedia program that takes advantage of the identified strengths of multimedia, and specifically tackles problems found in the first phase. Qualitative data collection techniques are used in both phases, and form the basis of the findings.The findings are presented as implications for multimedia design, and discuss the use and development of narrative and grammar in multimedia, as well as the importance of carefully designed user interaction and goal definition

Identity in Flux: A Theoretical and Choreographic Enquiry into the Identity of The Open Dance Work

This thesis presents a work of practical scholarship which re-examines issues of identity in the context of the open dance work. The debate takes the form of a symbiotic philosophical and choreographic enquiry into the identity of the open dance work. The philosophical enquiry examines the adequacy to the open work of theories of identity derived from two distinctive strands of philosophical theory, the first from the analytic tradition, the second from contemporary French philosophy. The choreographic processes which led to the creation of two open dance works constituted a central strand of the debate. Each work interrogates the adequacy of these theories of identity to the artistic theories which underpin open dance works. The first work, Intimate Memories, problematised theories of identity which were developed by analytic philosophers in the 1970s and 80s, and subsequently adopted by dance theorists, through an examination of their applicability to the open dance work. This strand of choreographic research revealed that, although these theories allowed open dance works to be individuated, they did not fully account for the particularities of the processual character of the open dance work. This interim conclusion led to an examination of the pertinence of Deleuze and Guattari’s ontology of the event to questions concerning identity raised by the open dance work. The second dance work, Halo in Performance, which was developed in the context of a collaborative engagement between choreographic and interactive digital arts practices, is an embodiment of Deleuze and Guattari’s process-oriented ontology of the event. This work indicates that theories of identity and/or individuation which are grounded in an ontology of flux are a more appropriate model to apply to the open dance work than those grounded in an ontology of substance

Montage As A Participatory System: Interactions with the Moving Image

Full version unavailable due to 3rd party copyright restrictionsRecent developments in network culture suggest a weakening of hierarchical narratives of power and representation. Online technologies of distributed authorship appear to nurture a complex, speculative, contradictory and contingent realism. Yet there is a continuing deficit where the moving image is concerned, its very form appearing resistant to the dynamic throughputs and change models of real-time interaction. If the task is not to suspend but encourage disbelief as a condition in the user, how can this be approached as a design problem? In the attempt to build a series of design projects suggesting open architectures for the moving image, might a variety of (pre-digital) precursors from the worlds of art, architecture and film offer the designer models for inspiration or adaptation? A series of projects have been undertaken. Each investigates the composite moving image, specifically in the context of real-time computation and interaction. This arose from a desire to interrogate the qualia of the moving image within interactive systems, relative to a range of behaviours and/or observer positions, which attempt to situate users as conscious compositors. This is explored in the thesis through reflecting on a series of experimental interfaces designed for real time composition in performance, exhibition and online contexts

Globalisation, Entrepreneurship and the South Pacific: Reframing Australian Colonial Architecture, 1800-1850

In 1957, Clinton Hartley Grattan, one of Australia’s most important foreign observers, wrote of the shadow of the “urban” in legends of the Australian “bush”.1 He argued that the early frontiers of Australian settlement were frontiers of men with private capital, or entrepreneurs, and those frontiers thus carried more elements of the urban than is commonly realised. Such early colonial enterprises around Australia’s south and southeastern coasts, and across the Tasman included sealing, whaling, milling and pastoralism, as well as missionary, trading and finance ventures. In advance of official settlements in the late-eighteenth and early-nineteenth centuries, entrepreneurs mapped coastlines, pioneered trade routes and colonised lands. Backed by private capital they established colonial infrastructural architecture effecting urban expansion in the Australian colonies, New Zealand and beyond. Yet this architecture is rarely a subject of architectural histories

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

Sensorial perception: empowering dance practice embodiment through live and virtual environments

A thesis submitted for the Masters of Arts by Research At the University of Bedfordshire, United KingdomThis thesis presents a phenomenological study exploring the practice of creating movement in live and virtual environments. The title of this study is Sensory Perception: Empowering dance embodiment through live and virtual environments. The aims of this study are: to experience the sensorial embodiment within live and virtual environments; and to understand the cognitive responses to a set of visual moving images that are mediated through the visual perception of the participant. This study was conducted by the author Lucie Lee in 2012-2013 at the University of Bedfordshire. The theoretical underpinning for this study used mainly two French phenomenological philosophers Maurice Merleau-Ponty (1859-1941) and Henri Louis Bergeson (1908-1961). This thesis discusses other cultural theories, which were contextualised in theoretical and practical approaches to this study such as post-modernism in dance, Liveness defined by Philip Auslander (1999) and Embodying theory (1998) described by Sarah Rubidge. The other component of this practice led research focuses on cognitive science. This study uses the software developed by Mark Coniglio founder of Troika Ranch Dance Company, call Isadora. This software provides the level of interaction needed for this study. Although the software was developed for creative application of technology in performance, in this investigation it acts as a research tool. Through the software’s applications the explorative creative tasks were interactive and utilised in the live and virtual environments. This practice-led research adopts the methodology of practice as research and an approach developed by performance theorist Professor Robin Nelson (2006). It also draws on the improvisatory processes of two American dancers and practitioners Alma Hawkins (1991) and Anna Halprin (1995). The improvisation technique deployed in this study is directly linked to Feldenkrais Method (1972). The explorative tasks were practically undertaken by a dancer in order to explore the role of sensory perception with improvisatory context. Wassily Kandinsky’s (1866-1984) works were used as a stimulus within this method to engage the performer in the use of colours and objects within creative tasks. In conclusion, the thesis highlights the importance of the development within the practice-led process of the processes and methods undertaken by the researcher and dancer. The summary of findings of this research created several practical improvisatory short scores with ten minute durations. The future developments of this research study are outlined in this conclusion chapter

The revolution in data gathering systems

Data acquisition systems used in NASA's wind tunnels from the 1950's through the present time are summarized as a baseline for assessing the impact of minicomputers and microcomputers on data acquisition and data processing. Emphasis is placed on the cyclic evolution in computer technology which transformed the central computer system, and finally the distributed computer system. Other developments discussed include: medium scale integration, large scale integration, combining the functions of data acquisition and control, and micro and minicomputers