Tools for expressive gesture recognition and mapping in rehearsal and performance

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 97-101).As human movement is an incredibly rich mode of communication and expression, performance artists working with digital media often use performers' movement and gestures to control and shape that digital media as part of a theatrical, choreographic, or musical performance. In my own work, I have found that strong, semantically-meaningful mappings between gesture and sound or visuals are necessary to create compelling performance interactions. However, the existing systems for developing mappings between incoming data streams and output media have extremely low-level concepts of "gesture." The actual programming process focuses on low-level sensor data, such as the voltage values of a particular sensor, which limits the user in his or her thinking process, requires users to have significant programming experience, and loses the expressive, meaningful, and metaphor-rich content of the movement. To remedy these difficulties, I have created a new framework and development environment for gestural control of media in rehearsal and performance, allowing users to create clear and intuitive mappings in a simple and flexible manner by using high-level descriptions of gestures and of gestural qualities. This approach, the Gestural Media Framework, recognizes continuous gesture and translates Laban Effort Notation into the realm of technological gesture analysis, allowing for the abstraction and encapsulation of sensor data into movement descriptions. As part of the evaluation of this system, I choreographed four performance pieces that use this system throughout the performance and rehearsal process to map dancers' movements to manipulation of sound and visual elements. This work has been supported by the MIT Media Laboratory.by Elena Naomi Jessop.S.M

USING INTERACTIVE SOFTWARE AS A CONCEPTUAL TOOL: AN EXAMINATION OF COGNITION IN IMPROVISED MUSICAL PERFORMANCE

Viewing musical improvisation in the light of psychology and cognitive science, this thesis will explicate the rationale behind the development of a software based audiovisual interface for use in improvised solo instrumental performance. The evolution of the performance environment is presented along with the theories and concepts that have shaped its progress. The opening chapter will review the terms of reference used throughout the work and will set a boundary around the area of examination. Chapter two will place musical improvisation within the context of human behaviour and in so doing will draw upon theoretical discourse from the fields of evolutionary psychology and cognitive science. This chapter will explore the nature of volition and its relationship with subconscious processing, drawing upon anecdotal evidence from improvising musicians as linkage between theory and practice. Chapter 3 augments the study of the inner world of the improvising musician by encompassing the communicative functions of this activity. The boundary of this study does not embrace musical interactions between musicians in a dialogic sense, my remit here is to explore behavioural response to sensory information and the mechanism by which this may or may not manifest itself in conscious thought. Chapter 4 sees the development of a theoretical model with which to contextualise the practice of musical improvisation and to provide the foundation from which to evolve the architecture for an experimental performance environment. This leads in Chapter 5 to a discussion around the function and nature of tools as problem solving devices looking at conceptual and physical tools and the mapping of functionality. The discourse in this chapter is aimed at providing a rationale for the development of a software based tool to address some of the issues raised previously in the study. The concluding chapter will document the evolution of a software based audio-visual performance environment, mapping its various incarnations and its relationship to the theoretical model developed over the course of the pervious chapters. This chapter will refer to documentation and audio visual material on CD Rom and DVD found in Appendix l

Improvised experimental music and the construction of a collaborative aesthetic.

Western musical aesthetics places composition at the centre of its enquiry, and this is expressed forcefully through the concept of the musical work: the product of a composer realised by performers who interpret the composer's score. European 'free' or improvised experimental music (IEM) is examined because of its challenge to mainstream musical thought, since it is the product of more than one organising mind in the moment of performance. The thesis shows how IEM draws upon ideas such as the work concept, articulating an identity which is bound to the work concept even as it criticises those ideas and work with ideas from other musical traditions. Following an account of the origins of IEM in Britain (chapter 1), chapter 2 focuses upon the work concept detailing both the resistance of the concept to new kinds of practice and its influence upon new music. Chapters 3 and 4 examine the role of the individual within a collaborative context, using materials from interviews with improvisers to draw out concerns which motivate IEM. The themes of performance and play are extrapolated which respectively complement and conflict with the 'work concept. Chapter 4 examines these themes in a more abstract way exploring parallels with the philosophical critique of institutional models of democracy. Chapter 5 presents a case study of musicians acquiring improvising skills under the guidance of an experienced improviser. The musical negotiations between the members of the group are considered in light of the themes of performance and play upon the making of the group's aesthetic character. Chapter 6 summarises the main themes of the preceding chapters showing that the aesthetic identity of IEM distinguishes itself from the concerns of mainstream musical aesthetics by virtue of its emphasis on collaboration, while at the same time drawing upon the individualistic motivations of the work concept. The thesis concludes with proposals for further research arising from these conclusions

Melodic improvisation on a twelve bar blues model : an investigation of physical and historical aspects and their contribution to performance

"Anything made by man, no matter how many varieties it assumes, and how much of the superhuman it seems to contain, must reveal its secret to the close observer." Paul Hindemith (1942: 176). "Are you one of those guys who wants to put crutches under my ass?" Lonnie Johnson (Keil 1966: 35). It is the aim of this thesis to define a musical genre by showing how, in the realisation of an improvisation, two key elements - the physical layout of a musical instrument and human movement patterns - are combined to produce music. This thesis takes as a model the twelve bar blues form, and examines the above aspects in the output of two pioneering figures of the melodic improvised guitar: Lonnie Johnson and T-Bone Walker. The thesis is in divided in to three sections; the first, which considers context, is divided in to four topics. • Identification of the model - the twelve bar blues form, and an examination of the meaning of the model to black culture. • Consideration of the role of geography on the emergent style • The guitar design and development. • Identification of the pioneering figures of the genre. The second section, which is concerned with musical, physical and analytical aspects, surveys theories of scale, mode, blues scale and blue notes, and suggests that blues improvisation is inextricably linked to spatio-motor based patterning. A geographical layout of the guitar is presented to aid in the analytical process. The third section is analytical and attempts to identify the melodic 'characteristics of the blues guitar genre. A series of transcriptions were made of improvisations of early blues guitar soloists. These are analysed by reduction and expansion. The tones produced in the improvisations were reduced to a modal hierarchy of principal, secondary and incidental tones which are presented on a fretboard form of notation. The resulting mode is reduced to a scale which is compared to theoretical definitions of blues scale. This raw data is then expanded by considering the left hand gestural movement between tones. gestures are seen to be linked together to form cells. These are sub-grouped in to various types. Larger structures, motives, are then defined as comprising several cells. The location of the cells in the model is indicated. These are categorised in groups for each performer.Thus improvisation is presented as an interaction, which takes place in time and space, between left hand movement strategies of the performer and the surface of a musical instrument

The visual language of Turkish roman dance

Thesis (M.A.) University of Alaska Fairbanks, 2021Utilizing Peircean semiotics and Dell Hymes' Ethnography of Communication, this research analyzes the dance style known as Roman havasi or, Turkish Roman dance. Elements and influences under consideration include costuming, dance vocabulary, musical instruments and traditions, gender-based stylistic differences, spheres of performance, audience member and participant demographics, and notions of authenticity in embodiment and presentation. An abridged history of the Roman is also presented, including public policies that influence the rights and lifestyle of Turkish Roman, as such factors have influenced their sense of social belonging as well as their dance and musical expressions. Romani people have experienced marginalization wherever they have immigrated, and scholarship regarding them has not always been beneficial. This thesis aims to increase the visibility, recognition, and appreciation of their culture. Research data was collected via participant observation as a Turkish Roman dance student and performer in Anchorage, Alaska, and from structured interviews with instructors and performers. Video footage was also gathered for analysis and was obtained by the author or via public domain websites. This thesis is accompanied by a collection of video clips featuring various elements of Turkish Roman dance referenced herein, and is available as supplemental material.Preface: Notes for non-Anthropologists -- Why Dance Research? -- A Note on Cultural Appropriation -- A Note on Reification -- Chapter 1. Introduction to Turkish Roman Dance and Its Relevance, Research Scope, and Methods -- Chapter 2. A Brief Overview of Romani History: The Formation of a Nation without Territory -- Chapter 3. The Historical Roots of Turkish Roman Identity -- Chapter 4. The Visual Language of Turkish Roman Dance: Its Elements, Themes, and Meanings -- Chapter 5. Closing Remarks -- Works cited -- Appendices

Open, mobile and indeterminate forms

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Since the early fifties “open form” has become a generic description for many different compositional concepts having in common musical outcomes which to a certain degree are indeterminate. The introduction looks into different meanings given to “form” in music and gives a historical survey of the origins of compositional indeterminacy. Next, the concept of “open form” is elaborated into a territory which is usually not associated with it: jazz. The introduction is followed by five case studies. Folio (1952-54) by Earle Brown is considered to contain the first intentionally “open form” works. It is driven by improvisational ideas, either at the compositional stage or at the interpretative stage. Brown's affinity with jazz also offers connections to other topics of the thesis. Miles Davis' Ife (1972) may at first seem like an odd inclusion in this study, but it is not. Its only oddity could be that of all the works discussed it has no score. But it is a composition; it is recognizable throughout its various incarnations and repeatable, and its outcome is indeterminate. Adam Rudolph did not conceive Ostinatos of Circularity as an “open form” work, but it is an indeterminate composition: it does have a score the musical result of which depends on the decisions made by the composer/conductor during the performance as well as the choices made by the performers. In Peter Zummo's Experimenting with Household Chemicals the performers play the same, often ambiguous, score, moving in the same direction at their own speed and discretion. The lack of synchronicity and the ambiguous notation result in a very elastic organic form. Anne La Berge refers to her recent works as “guided improvisations”. The scores mainly consist of suggestive text materials, software preset descriptions and rudimentary verbal indications, leaving major decisions to the performers. The last chapter is about my own work. It presents seven works (the scores of which can be found in the accompanying portfolio), composed between 2007 and 2011. Each of these works uses the score as a “field” through which the performers roam

Dialogic coding: a performance practice for co-creative computer improvisation

This research project explores Dialogic Coding – a performance practice situated within the field of live computer music which works towards a dialogic relationship with the computer as a programmable musical instrument. The writing articulates a Practice-as-Research (PaR) inquiry that places my practice within specific contextual, analytical and philosophical frameworks. The point of departure is the assumption that following the concept of dialogue a more reflexive way of performing music with a computer becomes possible. This approach may produce innovative results through transformations of musical ideas, embodied interactions as well as the performer's self-concept within a situation of improvised group performance. Dialogic Coding employs the concept of nontriviality to create an independent but at the same time programmable musical agent – the apparatus – which so becomes a co-creator of the improvised music. As a context for Dialogic Coding practice serve other dialogic forms of music making such as free improvised music as well as dynamic performances of programming found in live coding practice. A dialogic approach in music performance is based on listening and the ability to speak one's voice in response to the situation. Here, listening is understood beyond the auditory domain on the level of abstract thinking and physical interaction (interface affordance). This research presents a first-hand account of a computer performance praxis and thus makes a contribution to academic knowledge. For this it makes some implicit or tacit 'knowings' contained in the practice accessible for an outside community through this writing. Dialogic Coding practice was developed through participating in free improvised music 'sessions' with other musicians as well as composing pieces in program code with which I then performed live (solo and group). This writing contextualizes the developed practice in a historic lineage, discusses it within the conceptual framework of dialogism and delineated how a dialogic approach fosters creativity, learning, surprise and flow. As a conclusion I summarise the ethical dimension of Dialogic Coding as a form of human-computer interaction (HCI)

INTERACTIVE SONIFICATION STRATEGIES FOR THE MOTION AND EMOTION OF DANCE PERFORMANCES

The Immersive Interactive SOnification Platform, or iISoP for short, is a research platform for the creation of novel multimedia art, as well as exploratory research in the fields of sonification, affective computing, and gesture-based user interfaces. The goal of the iISoP’s dancer sonification system is to “sonify the motion and emotion” of a dance performance via musical auditory display. An additional goal of this dissertation is to develop and evaluate musical strategies for adding layer of emotional mappings to data sonification. The result of the series of dancer sonification design exercises led to the development of a novel musical sonification framework. The overall design process is divided into three main iterative phases: requirement gathering, prototype generation, and system evaluation. For the first phase help was provided from dancers and musicians in a participatory design fashion as domain experts in the field of non-verbal affective communication. Knowledge extraction procedures took the form of semi-structured interviews, stimuli feature evaluation, workshops, and think aloud protocols. For phase two, the expert dancers and musicians helped create test-able stimuli for prototype evaluation. In phase three, system evaluation, experts (dancers, musicians, etc.) and novice participants were recruited to provide subjective feedback from the perspectives of both performer and audience. Based on the results of the iterative design process, a novel sonification framework that translates motion and emotion data into descriptive music is proposed and described

A search for primitive voice in post-modern theatre

This thesis examines the influences of primitive vocal delivery on Post-Modern Theatre. Beginning with a review of the work of three directors: Jerzy Grotowski, Peter Brook, and Richard Schechner; the thesis moves to a case study of a 1991 production of Macbeth where primary research is documented. Focusing on the production of a language based on primitive culture, the theatrical community is furnished with a vocabulary of sound/communication that may be implemented for Macbeth\u27s Weird Sisters. Finally, the thesis directs the use of primitive vocal scoring, like musical transcription, toward a future outlook at similar projects