Understanding Music Interaction, and Why It Matters

This is the introductory chapter of a book dedicated to new research in, and emerging new understandings of, music and human-computer interaction—known for short as music interaction. Music interaction research plays a key role in innovative approaches to diverse musical activities, including performance, composition, education, analysis, production and collaborative music making. Music interaction is pivotal in new research directions in a range of activities, including audience participation, interaction between music and dancers, tools for algorithmic music, music video games, audio games, turntablism and live coding. More generally, music provides a powerful source of challenges and new ideas for human-computer interaction (HCI). This introductory chapter reviews the relationship between music and human-computer interaction and outlines research themes and issues that emerge from the collected work of researchers and practitioners in this book

Embodied Musical Interaction

Music is a natural partner to human-computer interaction, offering tasks and use cases for novel forms of interaction. The richness of the relationship between a performer and their instrument in expressive musical performance can provide valuable insight to human-computer interaction (HCI) researchers interested in applying these forms of deep interaction to other fields. Despite the longstanding connection between music and HCI, it is not an automatic one, and its history arguably points to as many differences as it does overlaps. Music research and HCI research both encompass broad issues, and utilize a wide range of methods. In this chapter I discuss how the concept of embodied interaction can be one way to think about music interaction. I propose how the three “paradigms” of HCI and three design accounts from the interaction design literature can serve as a lens through which to consider types of music HCI. I use this conceptual framework to discuss three different musical projects—Haptic Wave, Form Follows Sound, and BioMuse

Computational Music Systems for Emotional Health and Wellbeing: A Review

Music is a powerful stimulus, and both active and receptive methods of engaging with music provide affordances for improving physical, mental and social health. The emergence of sophisticated computational methods also underscores the potential for novel music technologies to address a wider range of wellbeing outcomes. In this review, we focus on describing the current state of the literature on computational approaches to music generation for health and wellbeing and identifying possible future directions for research in this area

Improvisation Pedagogy: An Epistemological Perspective of the 4‘E’ Model within Digital Musical Instruments

Recent years have witnessed the appearance of many new digital musical instruments (DMIs) and other interfaces for musical expression (NIME). This paper highlights a well-established music educational background theory that we believe may help DMI developers and users better understand DMIs in the context of music cognition and education. From an epistemological perspective, we present the paradigm of enactive music cognition related to improvisation in the context of the skills and needs of 21st century music learners. We hope this can lead to a deeper insertion of DMIs into music education, as well as to new DMIs to be ideated, prototyped and developed within these concepts and theories in mind. We specifically address the theory generally known as the 4E model of cognition (embodied, embedded, extended and enactive) within DMIs. The concept of autopoiesis is also described. Finally, we present some concrete cases of DMIs and NIMEs, and we describe how the experience of musical improvisation with them may be seen through the prism of such theories

La hipernormalización tecnológica de la creatividad: propuesta de evaluación del aprendizaje creativo en las interacciones tecnológicas musicales

[ES] El paradigma tecnosimbólico y positivista de la cuarta revolución industrial hipernormaliza progresivamente nuestras diversas formas de creatividad. Su estandarización y automatización afectan a ciencia, educación y salud a través del uso y la apropiación de la tecnología. La dialéctica del dataísmo, la conectividad, la inteligencia artificial, el aprendizaje automático o el pensamiento computacional promueve una onto-epistemología transhumanista y tecnócrata que acapara el debate pedagógico de la sociedad globalizada. Actualmente modelos computarizados intentan emular creatividad, aprendizaje o expresividad comunicativa, generando disonancias cognitivas producidas por la normalización y la estandarización sistémica connatural a la tecnología digital. Analizando este contexto e integrando los hallazgos y aportes alternativos de la educación y la terapia musical, esta investigación propone un modelo evaluativo alternativo respetuoso con el bienestar y salud de cada usuario, aportando información crucial para una optimización de la tecnología creativa y expresiva más humanizada, singular e individualizada, facilitando así un adecuado aprendizaje creativo e intuitivo dentro de un proceso creativo y expresivo musical. La investigación presente plantea el The Individualized Music Therapy Assessment Profile (IMTAP), como base metodológica evaluativa que incorpora, dentro de su propio marco, las formas metodológicas del rizoanálisis junto a los planteamientos de las Alfabetizaciones Múltiples (Kalantzis et al., 2020; Masny, 2014), integrando el trabajo de evaluación experimental que aportan Wanderley y Mackay (2019). Los resultados concluyen que es posible evaluar las interacciones entre humanos e interfaces tecnológicas musicales a través de la experiencia interactiva, singular, proactiva, creativa y expresiva de cada usuario tecnológico, integrando aspectos conjuntos de la educación, la salud y el bienestar de los usuarios dentro de los procesos de diseño de las tecnologías creativas y expresivas musicales. Además se aconseja la participación de arteterapeutas y musicoterapeutas profesionales dentro de los equipos de diseño de las interacciones y las interfaces cognitivas, creativas y expresivas, más allá del desarrollo de los diseños tecnológicos en etapas tempranas, y no sólo dentro de los marcos en los que se configuran las tecnologías musicales, sino durante los diferentes procesos iterativos que deben evaluar y optimizar las tecnologías ideadas en beneficio de la educación, la salud y el bienestar de cada usuario. [EN] The technosymbolic and positivist paradigm of the fourth industrial revolution progressively hypernormalizes our various forms of creativity. Its standardization and automation affect science, education and health through the use and appropriation of technology. The dialectic of dataism, connectivity, artificial intelligence, automatic learning or computational thinking promotes a transhumanist and technocratic onto-epistemology that monopolizes the pedagogical debate of the globalized society. Currently computerized models try to emulate creativity, learning or communicative expressiveness, generating cognitive dissonances produced by normalization and systemic standardization natural to digital technology. Analyzing this context and integrating the findings and alternative contributions of music education and therapy, this research proposes an alternative evaluative model respectful of the well-being and health of each user, providing crucial information for a more humanized optimization of creative and expressive technology, unique and individualized, thus facilitating adequate creative and intuitive learning within a creative and expressive musical process. The present investigation proposes The Individualized Music Therapy Assessment Profile (IMTAP), as an evaluative methodological basis that incorporates, within its own framework, the methodological forms of rhizoanalysis together with the approaches of Multiple Literacies (Kalantzis et al., 2020; Masny, 2014), integrating the experimental evaluation work provided by Wanderley and Mackay (2019). The results conclude that it is possible to evaluate the interactions between humans and musical technological interfaces through the interactive, singular, proactive, creative and expressive experience of each technological user, integrating joint aspects of education, health and well-being of users within of the design processes of creative and expressive musical technologies. In addition, the participation of professional art therapists and music therapists is advised within the design teams of cognitive, creative and expressive interactions and interfaces, beyond the development of technological designs in early stages, and not only within the frameworks in which musical technologies are configured, but during the different iterative processes that must evaluate and optimize the technologies designed for the benefit of education, health and well-being of each user

TMAP Design Cards for Technology-Mediated Audience Participation in Live Music

Historically, audiences have had various ways to participate in live music performances, including clapping, dancing, swaying, whistling, and singing. More recently, mobile and wireless devices, such as smartphones have opened up powerful new opportunities for audience participation. However, design for technology-mediated audience participation (TMAP) can be challenging: musicians and audiences have different demands, as does the coherence of the music, and group needs can vary widely. Thus, effective TMAP design requires the balancing of knowledge from diverse perspectives and must take into account the needs of diverse roles in creating and supporting performances. This chapter focuses on the process of creating and evaluating a set of design cards to support the interaction design and evaluation of TMAP systems. The cards are based on a previously created descriptive framework for supporting interaction design and evaluation in this challenging area. We discuss the conception and development of the TMAP design cards in some detail, and present an empirical study to evaluate their practical usefulness. Particular attention is paid to the ability of the cards to support finding ideas, changing ideas, and examining ideas from different perspectives

Idiomatic Patterns and Aesthetic Influence in Computer Music Languages

It is widely accepted that acoustic and digital musical instruments shape the cognitive processes of the performer on both embodied and conceptual levels, ultimately influencing the structure and aesthetics of the resulting performance. In this article we examine the ways in which computer music languages might similarly influence the aesthetic decisions of the digital music practitioner, even when those languages are designed for generality and theoretically capable of implementing any sound-producing process. We examine the basis for querying the non-neutrality of tools with a particular focus on the concept of idiomaticity: patterns of instruments or languages which are particularly easy or natural to execute in comparison to others. We then present correspondence with the developers of several major music programming languages and a survey of digital musical instrument creators examining the relationship between idiomatic patterns of the language and the characteristics of the resulting instruments and pieces. In an open-ended creative domain, asserting causal relationships is difficult and potentially inappropriate, but we find a complex interplay between language, instrument, piece and performance that suggests that the creator of the music programming language should be considered one party to a creative conversation that occurs each time a new instrument is designed.Peer reviewe

A Human-Centered Artificial Intelligence Approach to Adaptable Digital Musical Instrument Development

Advances within AI (Artificial Intelligence), particularly in relation to generative AI has led to greater inclusion of this technology within creative industries, including music. However, there are a variety of concerns pertaining to this innovation and its impact on musicians. Although there are a range of issues regarding this technology, it has the potential to enrich the current provision of adaptable, accessible digital musical instruments (ADMIs). This paper proposes the use of a human-centered AI methodology alongside users and musicians as co-creators to leverage the advantages of this technology, whilst integrating participatory practices to embed a more ethically focused approach to AI as a creative tool