Investigating the cognitive foundations of collaborative musical free improvisation: Experimental case studies using a novel application of the subsumption architecture

This thesis investigates the cognitive foundations of collaborative musical free improvisation. To explore the cognitive underpinnings of the collaborative process, a series of experimental case studies was undertaken in which expert improvisors performed with an artificial agent. The research connects ecological musicology and subsumption robotics, and builds upon insights from empirical psychology pertaining to the attribution of intentionality. A distinguishing characteristic of free improvisation is that no over-arching framework of formal musical conventions defines it, and it cannot be positively identified by sound alone, which poses difficulties for traditional musicology. Current musicological research has begun to focus on the social dimension of music, including improvisation. Ecological psychology, which focuses on the relation of cognition to agent–environment dynamics using the notion of affordances, has been shown to be a promising approach to understanding musical improvisation. This ecological approach to musicology makes it possible to address the subjective and social aspects of improvised music, as opposed to the common treatment of music as objective and neutral. The subjective dimension of musical listening has been highlighted in music cognition studies of cue abstraction, whereby listeners perceive emergent structures while listening to certain forms of music when no structures are identified in advance. These considerations informed the design of the artificial agent, Odessa, used for this study. In contrast to traditional artificial intelligence (AI), which tends to view the world as objective and neutral, behaviour-based robotics historically developed around ideas similar to those of ecological psychology, focused on agent–environment dynamics and the ability to deal with potentially rapidly changing environments. Behaviour-based systems that are designed using the subsumption architecture are robust and flexible in virtue of their modular, decentralised design comprised of simple interactions between simple mechanisms. The competence of such agents is demonstrated on the basis of their interaction with the environment and ability to cope with unknown and dynamic conditions, which suggests the concept of improvisation. This thesis documents a parsimonious subsumption design for an agent that performs musical free improvisation with human co-performers, as well as the experimental studies conducted with this agent. The empirical component examines the human experience of collaborating with the agent and, more generally, the cognitive psychology of collaborative improvisation. The design was ultimately successful, and yielded insights about cognition in collaborative improvisation, in particular, concerning the central relationship between perceived intentionality and affordances. As a novel application of the subsumption architecture, this research contributes to AI/robotics and to research on interactive improvisation systems. It also contributes to music psychology and cognition, as well as improvisation studies, through its empirical grounding of an ecological model of musical interaction

Designing and evaluating the usability of a machine learning API for rapid prototyping music technology

To better support creative software developers and music technologists' needs, and to empower them as machine learning users and innovators, the usability of and developer experience with machine learning tools must be considered and better understood. We review background research on the design and evaluation of application programming interfaces (APIs), with a focus on the domain of machine learning for music technology software development. We present the design rationale for the RAPID-MIX API, an easy-to-use API for rapid prototyping with interactive machine learning, and a usability evaluation study with software developers of music technology. A cognitive dimensions questionnaire was designed and delivered to a group of 12 participants who used the RAPID-MIX API in their software projects, including people who developed systems for personal use and professionals developing software products for music and creative technology companies. The results from the questionnaire indicate that participants found the RAPID-MIX API a machine learning API which is easy to learn and use, fun, and good for rapid prototyping with interactive machine learning. Based on these findings, we present an analysis and characterization of the RAPID-MIX API based on the cognitive dimensions framework, and discuss its design trade-offs and usability issues. We use these insights and our design experience to provide design recommendations for ML APIs for rapid prototyping of music technology. We conclude with a summary of the main insights, a discussion of the merits and challenges of the application of the CDs framework to the evaluation of machine learning APIs, and directions to future work which our research deems valuable

Sampling the past:a tactile approach to interactive musical instrument exhibits in the heritage sector

In the last decade, the heritage sector has had to adapt to a shifting cultural landscape of public expectations and attitudes towards ownership and intellectual property. One way it has done this is to focus on each visitor’s encounter and provide them with a sense of experiential authenticity.There is a clear desire by the public to engage with music collections in this way, and a sound museological rationale for providing such access, but the approach raises particular curatorial problems, specifically how do we meaningfully balance access with the duty to preserve objects for future generations?This paper charts the development of one such project. Based at Fenton House in Hampstead, and running since 2008, the project seeks to model digitally the keyboard instruments in the Benton Fletcher Collection and provide a dedicated interactive exhibit, which allows visitors to view all of the instruments in situ, and then play them through a custom-built two-manual MIDI controller with touch-screen interface.We discuss the approach to modelling, which uses high-definition sampling, and highlight the strengths and weaknesses of the exhibit as it currently stands, with particular focus on its key shortcoming: at present, there is no way to effectively model the key feel of a historic keyboard instrument.This issue is of profound importance, since the feel of any instrument is fundamental to its character, and shapes the way performers relate to it. The issue is further compounded if we are to consider a single dedicated keyboard as being the primary mode of interface for several instrument models of different classes, each with its own characteristic feel.We conclude by proposing an outline solution to this problem, detailing early work on a real-time adaptive haptic keyboard interface that changes its action in response to sampled resistance curves, measured on a key-by-key basis from the original instruments