Choreographic and Somatic Approaches for the Development of Expressive Robotic Systems

As robotic systems are moved out of factory work cells into human-facing environments questions of choreography become central to their design, placement, and application. With a human viewer or counterpart present, a system will automatically be interpreted within context, style of movement, and form factor by human beings as animate elements of their environment. The interpretation by this human counterpart is critical to the success of the system's integration: knobs on the system need to make sense to a human counterpart; an artificial agent should have a way of notifying a human counterpart of a change in system state, possibly through motion profiles; and the motion of a human counterpart may have important contextual clues for task completion. Thus, professional choreographers, dance practitioners, and movement analysts are critical to research in robotics. They have design methods for movement that align with human audience perception, can identify simplified features of movement for human-robot interaction goals, and have detailed knowledge of the capacity of human movement. This article provides approaches employed by one research lab, specific impacts on technical and artistic projects within, and principles that may guide future such work. The background section reports on choreography, somatic perspectives, improvisation, the Laban/Bartenieff Movement System, and robotics. From this context methods including embodied exercises, writing prompts, and community building activities have been developed to facilitate interdisciplinary research. The results of this work is presented as an overview of a smattering of projects in areas like high-level motion planning, software development for rapid prototyping of movement, artistic output, and user studies that help understand how people interpret movement. Finally, guiding principles for other groups to adopt are posited.Comment: Under review at MDPI Arts Special Issue "The Machine as Artist (for the 21st Century)" http://www.mdpi.com/journal/arts/special_issues/Machine_Artis

Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances

In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the perform- ers and audience members. By utilizing physiological data (namely Electrodermal Activity (EDA) and Heart Rate Variability (HRV)) and applying this data to musical artifacts including a robotic koto (a traditional 13-string Japanese instrument fitted with solenoids and linear actuators), a Eurorack synthesizer, and Max/MSP software, we aim to develop a new form of semi-improvisational and signif- icantly indeterminate performance practice. It has since evolved into a multi-modal methodology which honors improvisational performance practices and utilizes physiological data which of- fers both performers and audiences an ever-changing and intimate experience. In our first exploratory phase, we focused on the development of a means for controlling a bespoke robotic koto in conjunction with a Eurorack synthesizer system and Max/MSP software for controlling the incoming data. We integrated a reliance on physiological data to infuse a more directly human elements into this artifact system. This allows a significant portion of the decision-making to be directly controlled by the incoming physiological data in real-time, thereby affording a sense of performativity within this non-living system. Our aim is to continue the development of this method to strike a novel balance between intentionality and impromptu performative results

Are the robots coming? Designing with autonomy & control for musical creativity & performance

This paper1 expands upon our previous work, and starts to unpack notions of autonomy and control in musical composition and performance-based systems. The term autonomous has become synonymous with technologies such as “autonomous vehicles” and “drones”, while notions of control have mainly been raised in respect to the “control” of industrial systems and in respect to protocols. This position piece disrupts these notions and provides a platform, introducing a more radical proposition in respect to the representation of autonomy and control of features that can be used to design systems that support musical composition and performance. This paper supports a growing interest within the Design, HCI and Artificial Intelligence communities, leading us to think about Human Like Computing systems and the development of a Computational Creativity Continuum

Church Belles: An Interactive System and Composition Using Real-World Metaphors

This paper presents a brief review of current literature detailing some of the issues and trends in composition and performance with interactive music systems. Of particular interest is how musicians interact with a separate machine entity that exercises agency over the creative process. The use of real-world metaphors as a strategy for increasing audience engagement is also discussed. The composition and system Church Belles is presented, analyzed and evaluated in terms of its architecture, how it relates to existing studies of musician-machine creative interaction and how the use of a real-world metaphor can promote audience perceptions of liveness. This develops previous NIME work by offering a detailed case study of the development process of both a system and a piece for popular, non-improvisational vocal/guitar music. keywords: Interactive music systems, real-world, metaphor, physical model, popular musi

Accumulated response in live improvised dance performance

The response project and accompanying thesis aim to affirm the role of the dancer as an authority in revealing patterns and traces of accumulated lived experience, knowledge and ideas through the practice and performance of dance improvisation. The dancing body is investigated as a dynamical and complex system of research that is embedded in a process of continual response to the present. These bodily inscriptions and the process of active response form the seminal grounds for the physical and energetic exchange of improvised dance in performance

Musicians (Don't) Play Algorithms. Or: What makes a musical performance

Our private perception of listening to an individualized playlist during a jog is very different from the interaction we might experience at a live concert. We do realize that music is not necessarily a performing art, such as dancing or theater, while our demands regarding musical performances are conflicting: We expect perfect sound quality and the thrill of the immediate. We want the artist to overwhelm us with her virtuosity and we want her to struggle, just like a human. We want to engage with the musical expression and rely on visual and physical cues. Considering that the ears of today’s listeners are used to technologically mediated music, in this paper I explore the unique qualities of musical live performances and examine if our conception allows for new mechatronic inventions, in particular robotic musicians, to participate in this art form. Some of Godlovitch’s main thoughts expounded in his work on “musical performance” [11] serve as a reference and starting point for this investigation. His concept of ‘personalism’, which deprives computer-/program-based musical performances from expressive potential and creative accomplishment is an issue that I want to challenge by pointing out new approaches arising from a reflective discourse on technology, embodiment and expression. The enquiry conducted illustrates, how in reasoning about machine performers and algorithmic realization of music, we also examine the perceptual, physical and social aspects of human musicianship, reconceptualizing our understanding of a musical live performance

Listeners' and performers' shared understanding of jazz improvisations

This study explores the extent to which a large set of musically experienced listeners share understanding with a performing saxophone-piano duo, and with each other, of what happened in three improvisations on a jazz standard. In an online survey, 239 participants listened to audio recordings of three improvisations and rated their agreement with 24 specific statements that the performers and a jazz-expert commenting listener had made about them. Listeners endorsed statements that the performers had agreed upon significantly more than they endorsed statements that the performers had disagreed upon, even though the statements gave no indication of performers' levels of agreement. The findings show some support for a more-experienced-listeners-understand-more-like-performers hypothesis: Listeners with more jazz experience and with experience playing the performers' instruments endorsed the performers' statements more than did listeners with less jazz experience and experience on different instruments. The findings also strongly support a listeners-as-outsiders hypothesis: Listeners' ratings of the 24 statements were far more likely to cluster with the commenting listener's ratings than with either performer's. But the pattern was not universal; particular listeners even with similar musical backgrounds could interpret the same improvisations radically differently. The evidence demonstrates that it is possible for performers' interpretations to be shared with very few listeners, and that listeners' interpretations about what happened in a musical performance can be far more different from performers' interpretations than performers or other listeners might assume