The Speckled Cellist: Classification of Cello Bowing Styles using the Orient Specks

Cello bowing techniques are classified by applying supervised machine learning methods to sensor data from two inertial sensors called the Orient specks – one worn on the playing wrist and the other attached to the frog of the bow. Twelve different bowing techniques were considered, including variants on a single string and across multiple strings. Results are presented for the classification of these twelve techniques when played singly, and in combination during improvisational play. The results demonstrated that even when limited to two sensors, classification accuracy in excess of 95% was obtained for the individual bowing styles, with the added advantages of a minimalist approach

Interactive Tango Milonga: An Interactive Dance System for Argentine Tango Social Dance

abstract: When dancers are granted agency over music, as in interactive dance systems, the actors are most often concerned with the problem of creating a staged performance for an audience. However, as is reflected by the above quote, the practice of Argentine tango social dance is most concerned with participants internal experience and their relationship to the broader tango community. In this dissertation I explore creative approaches to enrich the sense of connection, that is, the experience of oneness with a partner and complete immersion in music and dance for Argentine tango dancers by providing agency over musical activities through the use of interactive technology. Specifically, I create an interactive dance system that allows tango dancers to affect and create music via their movements in the context of social dance. The motivations for this work are multifold: 1) to intensify embodied experience of the interplay between dance and music, individual and partner, couple and community, 2) to create shared experience of the conventions of tango dance, and 3) to innovate Argentine tango social dance practice for the purposes of education and increasing musicality in dancers.Dissertation/ThesisDoctoral Dissertation Music 201

Almost Human: The Study of Physical Processes and the Performance of a Prosthetic Digital Spine

Almost Human is an investigation of interdisciplinary performance through music that looks to the self to try to further understand subjective performance practices in expression, gesture and sonic output. This text presents experimental methods of examining and creating music through kinaesthetic and electronic-assisted means within instrumental, dance and interactive works. The extraction of affective, performative and sonic properties from these works aids in unlocking the relationship between the choreographic, physical and conceptual object. The first part of the text explores and illustrates multimodal approaches to analysing, capturing, measuring and archiving the moving musician and dancer in an assortment of performative settings. It focuses on a series of works for solo cello, as well as interdisciplinary pieces which positions movement and embodied expressivity at the forefront of the discussion. The second part is dedicated to the aesthetic, conceptual and utilitarian content of a new interactive work for cellist/mover, and a prosthetic digital spine. Here, relationships are combined to showcase the permeability of the body, as well as its expressive content. The conceptual object, The Spine, serves as a generator to help expand musical and artistic possibilities. Its inclusion in the work aids in refocusing my relationship to movement and sound for creation and performance, but also aesthetically, it adds to the growing canon of experimental ventures in conceptualising expressivity. Beyond the text, the portfolio of Almost Human includes an auditory and visual chronicle of the process between the years 2012-14, which is used to assist the reader in further understanding the performative practice and findings

Methods and Technologies for the Analysis and Interactive Use of Body Movements in Instrumental Music Performance

List of related publications: http://www.federicovisi.com/publications/A constantly growing corpus of interdisciplinary studies support the idea that music is a complex multimodal medium that is experienced not only by means of sounds but also through body movement. From this perspective, musical instruments can be seen as technological objects coupled with a repertoire of performance gestures. This repertoire is part of an ecological knowledge shared by musicians and listeners alike. It is part of the engine that guides musical experience and has a considerable expressive potential. This thesis explores technical and conceptual issues related to the analysis and creative use of music-related body movements in instrumental music performance. The complexity of this subject required an interdisciplinary approach, which includes the review of multiple theoretical accounts, quantitative and qualitative analysis of data collected in motion capture laboratories, the development and implementation of technologies for the interpretation and interactive use of motion data, and the creation of short musical pieces that actively employ the movement of the performers as an expressive musical feature. The theoretical framework is informed by embodied and enactive accounts of music cognition as well as by systematic studies of music-related movement and expressive music performance. The assumption that the movements of a musician are part of a shared knowledge is empirically explored through an experiment aimed at analysing the motion capture data of a violinist performing a selection of short musical excerpts. A group of subjects with no prior experience playing the violin is then asked to mime a performance following the audio excerpts recorded by the violinist. Motion data is recorded, analysed, and compared with the expert’s data. This is done both quantitatively through data analysis xii as well as qualitatively by relating the motion data to other high-level features and structures of the musical excerpts. Solutions to issues regarding capturing and storing movement data and its use in real-time scenarios are proposed. For the interactive use of motion-sensing technologies in music performance, various wearable sensors have been employed, along with different approaches for mapping control data to sound synthesis and signal processing parameters. In particular, novel approaches for the extraction of meaningful features from raw sensor data and the use of machine learning techniques for mapping movement to live electronics are described. To complete the framework, an essential element of this research project is the com- position and performance of études that explore the creative use of body movement in instrumental music from a Practice-as-Research perspective. This works as a test bed for the proposed concepts and techniques. Mapping concepts and technologies are challenged in a scenario constrained by the use of musical instruments, and different mapping ap- proaches are implemented and compared. In addition, techniques for notating movement in the score, and the impact of interactive motion sensor systems in instrumental music practice from the performer’s perspective are discussed. Finally, the chapter concluding the part of the thesis dedicated to practical implementations describes a novel method for mapping movement data to sound synthesis. This technique is based on the analysis of multimodal motion data collected from multiple subjects and its design draws from the theoretical, analytical, and practical works described throughout the dissertation. Overall, the parts and the diverse approaches that constitute this thesis work in synergy, contributing to the ongoing discourses on the study of musical gestures and the design of interactive music systems from multiple angles

Actor & Avatar: A Scientific and Artistic Catalog

What kind of relationship do we have with artificial beings (avatars, puppets, robots, etc.)? What does it mean to mirror ourselves in them, to perform them or to play trial identity games with them? Actor & Avatar addresses these questions from artistic and scholarly angles. Contributions on the making of "technical others" and philosophical reflections on artificial alterity are flanked by neuroscientific studies on different ways of perceiving living persons and artificial counterparts. The contributors have achieved a successful artistic-scientific collaboration with extensive visual material

MAMI Tech Toolkit Utilising Action Research to Develop a Technological Toolkit to Facilitate Access to Music-Making

Music is essential to most of us, it can light up all areas of the brain, help develop skills with communication, help to establish identity, and allow a unique path for expression. However, barriers to access or gaps in provision can restrict access to music-making and sound exploration for some people. Research has shown that technology can provide unique tools to access music-making but that technology is underused by practitioners. This action research project details the development and design of a technological toolkit called MAMI – the Modular Accessible Musical Instrument technology toolkit - in conjunction with stakeholders from four research sites. Stakeholders included music therapists, teachers, community musicians, and children and young people. The overarching aims of the research were: to explore how technology was incorporated into practices of music creation and sound exploration; to explore the issues that stakeholders had with current music technology; to create novel musical tools and tools that match criteria as specified by stakeholders, and address issues as found in a literature review; to assess the effectiveness of these novel tools with a view to improving practices; and to navigate propagation of the practices, technologies, and methods used to allow for transferability into the wider ecology. Outcomes of the research include: a set of design considerations that contribute to knowledge around the design and practical use of technological tools for music-making in special educational needs settings; a series of methodological considerations to help future researchers and developers navigate the process of using action research to create new technological tools with stakeholders; and the MAMI Tech Toolkit – a suite of four bespoke hardware tools and accompanying software - as an embodiment of the themes that emerged from: the cycles of action research; the design considerations; and a philosophical understanding of music creation that foregrounds it as an situated activity within a social context

MAMI tech toolkit: utilising action research to develop a technological toolkit to facilitate access to music-making.

Music is essential to most of us, it can light up all areas of the brain, help develop skills with communication, help to establish identity, and allow a unique path for expression. However, barriers to access or gaps in provision can restrict access to music-making and sound exploration for some people. Research has shown that technology can provide unique tools to access music-making but that technology is underused by practitioners. This action research project details the development and design of a technological toolkit called MAMI – the Modular Accessible Musical Instrument technology toolkit - in conjunction with stakeholders from four research sites. Stakeholders included music therapists, teachers, community musicians, and children and young people. The overarching aims of the research were: to explore how technology was incorporated into practices of music creation and sound exploration; to explore the issues that stakeholders had with current music technology; to create novel musical tools and tools that match criteria as specified by stakeholders, and address issues as found in a literature review; to assess the effectiveness of these novel tools with a view to improving practices; and to navigate propagation of the practices, technologies, and methods used to allow for transferability into the wider ecology. Outcomes of the research include: a set of design considerations that contribute to knowledge around the design and practical use of technological tools for music-making in special educational needs settings; a series of methodological considerations to help future researchers and developers navigate the process of using action research to create new technological tools with stakeholders; and the MAMI Tech Toolkit – a suite of four bespoke hardware tools and accompanying software - as an embodiment of the themes that emerged from: the cycles of action research; the design considerations; and a philosophical understanding of music creation that foregrounds it as an situated activity within a social context

Apps, Agents, and Improvisation: Ensemble Interaction with Touch-Screen Digital Musical Instruments

This thesis concerns the making and performing of music with new digital musical instruments (DMIs) designed for ensemble performance. While computer music has advanced to the point where a huge variety of digital instruments are common in educational, recreational, and professional music-making, these instruments rarely seek to enhance the ensemble context in which they are used. Interaction models that map individual gestures to sound have been previously studied, but the interactions of ensembles within these models are not well understood. In this research, new ensemble-focussed instruments have been designed and deployed in an ongoing artistic practice. These instruments have also been evaluated to find out whether, and if so how, they affect the ensembles and music that is made with them. Throughout this thesis, six ensemble-focussed DMIs are introduced for mobile touch-screen computers. A series of improvised rehearsals and performances leads to the identification of a vocabulary of continuous performative touch-gestures and a system for tracking these collaborative performances in real time using tools from machine learning. The tracking system is posed as an intelligent agent that can continually analyse the gestural states of performers, and trigger a response in the performers' user interfaces at appropriate moments. The hypothesis is that the agent interaction and UI response can enhance improvised performances, allowing performers to better explore creative interactions with each other, produce better music, and have a more enjoyable experience. Two formal studies are described where participants rate their perceptions of improvised performances with a variety of designs for agent-app interaction. The first, with three expert performers, informed refinements for a set of apps. The most successful interface was redesigned and investigated further in a second study with 16 non-expert participants. In the final interface, each performer freely improvised with a limited number of notes; at moments of peak gestural change, the agent presented users with the opportunity to try different notes. This interface is shown to produce performances that are longer, as well as demonstrate improved perceptions of musical structure, group interaction, enjoyment and overall quality. Overall, this research examined ensemble DMI performance in unprecedented scope and detail, with more than 150 interaction sessions recorded. Informed by the results of lab and field studies using quantitative and qualitative methods, four generations of ensemble-focussed interface have been developed and refined. The results of the most recent studies assure us that the intelligent agent interaction does enhance improvised performances