Music interaction: understanding music and human-computer interaction

We introduce and review recent research in Music and Human Computer Interaction, also known as Music Interaction. After a general overview of the discipline, we analyse the themes and issues raised by the fifteen chapters of this book, each of which presents recent research in this field. The bulk of this chapter is organised as an FAQ. This enables some FAQs to focus on cross cutting issues that appear in multiple chapters, and some chapters to feature in multiple FAQs. Broad topics include: the scope of research in Music Interaction; the role of HCI in Music Interaction; and conversely, the role of Music Interaction in HCI. High-level themes include embodied cognition, spatial cognition, evolutionary interaction, gesture, formal language, affective interaction, and methodologies from social science. Musical activities of interest include performance, composition, analysis, collaborative music making, and human and machine improvisation. Specific issues include: whether Music Interaction should be easy; what can be learned from the experience of being “in the groove”, and what can be learned from the deep commitment of musical amateurs. Broader issues include: what Music Interaction can offer traditional instruments and traditional musical activities; what relevance it has for non-musical domains; and ways in which Music Interaction can enable entirely new musical activities possible

Music and HCI

Music is an evolutionarily deep-rooted, abstract, real-time, complex, non-verbal, social activity. Consequently, interaction design in music can be a valuable source of challenges and new ideas for HCI. This workshop will reflect on the latest research in Music and HCI (Music Interaction for short), with the aim of strengthening the dialogue between the Music Interaction community and the wider HCI community. We will explore recent ideas from Music Interaction that may contribute new perspectives to general HCI practice, and conversely, recent HCI research in non-musical domains with implications for Music Interaction. We will also identify any concerns of Music Interaction that may require unique approaches. Contributors engaged in research in any area of Music Interaction or HCI who would like to contribute to a sustained widening of the dialogue between the distinctive concerns of the Music Interaction community and the wider HCI community will be welcome

Augmented Reality to Facilitate Learning of the Acoustic Guitar

[Abstract] Many people wishing to learn a musical instrument opt to learn using alternative or informal methods instead of the traditional Master–Apprentice model that requires a greater cognitive load. This paper presents an augmented reality (AR)-based application designed to teach and train guitar chords, with the novelty that it is also used to teach short melodies consisting of four chord transitions so that users have to change hand and finger positions. The app uses high-quality 3D models of an acoustic guitar and animated hand to indicate correct finger positions and the movements required when changing from one chord to another. To follow the animated instructions, the learner overlaps the 3D model onto the neck of the physical guitar and his or her own hand. A system usability scale (SUS) questionnaire was used to measure the usability of the application. A score of 82.0 was obtained, which is higher than the average of 68 points that indicates the application is good from a user experience perspective, thus satisfying the purpose for which it was created. Having analysed the data for both groups—individuals with no prior experience of playing a musical instrument versus individuals with prior experience—it was concluded that the application provided a useful learning approach for all participants involved in the study, regardless of experience. That said, those possessing prior experience of playing an instrument learnt faster. It should be noted that the research revealed significant difference in learning by gender, with male participants learning faster than female participants. Similar results have been detected in other research performed in the field of music, as well as in other fields. As this study required spatial reasoning when viewing the 3D model, the differences identified this case may well have arisen as a consequence of differences in men and women’s spatial awareness, thereby leaving open an alternative line of research

Conceptual Metaphor, Human-Computer Interaction And Music: Applying Conceptual Metaphor To The Design And Analysis Of Music Interactions

Interaction design for domains that involve complex abstractions can present significant challenges. This problem is particularly acute in domains where users lack effective means to conceptualise and articulate relevant abstractions. In this thesis, we investigate the use of domain-specific conceptual metaphors to address the challenge of presenting complex abstractions, using tonal harmony as an extended case study. This thesis presents a methodology for applying domain-specific conceptual metaphors to interactions designs for music. This domain involves complex abstractions where users with any degree of domain knowledge may have difficulty in articulating concepts. The methodology comprises several parts. Firstly, the thesis explores methods for systematically guiding conversation between musicians to elicit speech that describes music using conceptual metaphors. Recommendations for the most suitable methods are made. Secondly, the thesis presents a methodology for identifying image schemas and conceptual metaphors from transcriptions of conversations between musicians. The methodology covers rules for identifying source image schemas and extrapolating conceptual metaphors. Thirdly, the thesis presents a methodology for evaluating existing music interaction designs using domain-specific conceptual metaphors. We demonstrate that this approach can be used to identify potential areas for improvement as well as tensions in the design between certain tasks or abstractions. Fourthly, the thesis presents a case study for the development of a conceptual metaphor-influenced design process. In the case study, a set of materials are developed to be used by participants in the design process to facilitate the mapping of conceptual metaphors to elements of an interaction design without requiring knowledge of Conceptual Metaphor Theory. Finally, a pilot study is presented integrating the results of the conceptual metaphor-influenced design process into a consistent and useful prototype system. Compromises and refinements to the design proposals made during the design process are discussed and the resulting system design is detailed

Experiencing mappings on a music controller: case Ableton Push

During the past two decades, computers have taken a major role in composition and production of modern music. Various different devices are used to control modern DAWs (digital audio workstations). This diversity of controllers gives users a wide range of options to interact with a computer during the composition and creation. A deeper integration blurs boundaries between a software and controller and creates more complex modes of interaction. This study focuses on a connection between a musical controller and computer through the concept of mapping. The aim of the study was to understand, how mappings of a contemporary software controller can be conceptualised and how they affect user experience. The material for this study was collected through an online-survey, user-sessions and semi-structured interviews for users of Ableton Live software and Push controller. Collected data show that users consider the mappings as an important part of the initial usability of the controller, but there is no clear correlation with the user experience. The results stress the importance of avoiding idiosyncrasies in the mappings and reveal the versatile approaches between users. Users with previous experience from the software tend to have established use patterns that might affect the deployment of a new controller. Mappings can be considered to have an essential role in these patterns

Absent Performers, Absent Tools, and Their Role in Musical Composition: Exploring Integrated Competencies in the Extended Musical Mind.

The tools and performers which surround the composer are not simply a means to communicate musical ideas, they take part in the formation and development of those ideas. Essentially, musical composition does not only take place in the head. This is the compelling perspective provided by the application of ‘4E’ theories of cognition, and related ideas from phenomenology and cognitive science, to musical creativity, where the cognitive processes of musical composition can be seen as varyingly embodied, embedded, enacted, extended, distributed, and materially engaged. This thesis attempts to break new conceptual ground through applying these ideas to musical composition with a focus on absent tools and performers. The insights gained from compositional practice, both inherent and explicitly analysed, function alongside the other methodological approaches. A portfolio of compositions and recordings therefore forms a core part of the thesis. The process of recording the compositions is often as important to the research as the process of their composition – the recordings are not simply there to allow the pieces to be heard. The analysis of his portfolio forms an essential part of the written portion of the thesis. The thesis therefore undertakes its enquiry in two ways, through theory and through practice. The written portion of the thesis is also divided into two parts. The first part lays out the relevant, existing theories, and any existing applications thereof to music, and then applies these ideas to musical composition. This provides an understanding of the role of present tools and performers in the cognitive processes of composition. The second part builds on this understanding, through asking how we should understand processes of composition which take place while the tools, environments, and collaborators are absent (i.e., imagined, remembered, or virtual). This requires the development of a conceptual framework, ‘Integrated Tool Competency’, which is first applied to tools, and then to performers. In this way, the thesis develops a new way of understanding musical composition. It also has the potential to contribute substantially to the theories through which that understanding was reached

Studies on customisation-driven digital music instruments

From John Cage’s Prepared Piano to the turntable, the history of musical instruments is scattered with examples of musicians who deeply customised their instruments to fit personal artistic objectives, objectives that differed from the ones the instruments have been designed for. In their digital counterpart however, musical instruments are often presented in the form of closed, finalised systems with apriori symbolic rules set by their designer that leave very little room for the artists to customise the technologies for their unique art practices; in these cases the only possibility to change the mode of interaction with digital instrument is to reprogram them, a possibility available to programmers but not to musicians. This thesis presents two digital music instruments designed with the explicit goal of being highly customisable by musicians and to provide different modes of interactions, whilst keeping simplicity and immediateness of use. The first one leverages real-time gesture recognition to provide continuous feedback to users as guidance in defining the behaviour of the system and the gestures it recognises. The second one is a novel tangible user interface which allows to transform everyday objects into expressive digital music instruments, and whose sound generated strongly depends by the particular nature of the physical object selected

The composition of concert music within the Digital Audio Workstation environment.

This research project includes a portfolio of compositions and a contextual document, the subject of which is the Digital Audio Workstation as a human computer interface for the composition of concert music, intended for live performance. The portfolio of compositions consists of the music notation and audio recordings of five works, composed by the researcher within the context of the research project: Variances for jazz guitar, piano, percussion and indeterminate ensemble; Rapprochement for microtonal ensemble; Transits for bass flute, eight voices and electronics; Shutterspeed for clarinet and soundtrack; and Fossils for piano and live electronics. For each of the works, a distinct DAW-based concept and compositional approach has been developed and employed. The contextual document investigates the implications of the DAW for the composition of concert music, and discusses the five portfolio compositions against the background of practices of selected composers; the history and development of the analogue studio, early computer and DAW composition; aspects of device interaction; and the topic of 'liveness'

Should Music Interaction Be Easy?

A fundamental assumptio in the fields of human-computer interaction and usability studies is that interfaces should be designed for ease of use, with a few exceptions such as the trade-off with long-term power. In this chapter it is argued that in music interactio the situation is far more complex, with social, technical, artistic, and psychological reasons why difficulty is in some cases a good thing, and in other cases a necessary evil. Different aspects of static and time-varying difficulty in music interaction are categorised. Some specific areas in which difficulty seems to be inextricaly linked to positive aspects of music interaction are described. This is followed by discussion of some areas in which difficulty is undesirable and, perhaps, avoidable. Examples are drawn from music interaction research in general and from other chapters of this book in particular.No Full Tex