Analysis of conceptual metaphors to improve music software: the role of prior experience in inclusive music interaction

An open challenge for interaction designers is to find ways of designing software solutions capable of enhancing the ability of users with differing levels of knowledge and capabilities to perform tasks that normally require specialised domain expertise. This challenge is particularly demanding in areas such as music analysis, where complex, abstract, domain-specific concepts and notations occur. One promising theoretical foundation for this work involves the identification of what are assumed to be conceptual metaphors and image schemas. These constructs are grounded in prior sensory-motor experiences of space, force, orientation and interactions with other bodies and are identified through analysing discourse. These theories have already been applied with some success both to musical concepts and, separately, to user interface design with the intention of creating more intuitive designs through exploiting these prior sensory-motor experiences. However, the present work appears to be the first to combine these hitherto distinct bodies of research. We hypothesise that if we can identify the conceptual metaphors and image schemas used by musical experts to structure their understanding of music, we can utilise this information to develop a general theoretical framework to aid designers in systematically improving their designs, providing a foundation for inclusive music interaction design and thus opening up software normally restricted to domain experts to a wider and more varied user population. A possible methodology for the development of such a framework is discussed along with details of a study to apply the conceptual metaphors and image schemas used by a small group of experienced musicians to an evaluation of Harmony Space

Eliciting Domain Knowledge Using Conceptual Metaphors: A Case Study from Music Interaction

Interaction design for domains that involve complex abstractions can prove challenging. This problem is particularly acute in domains where the intricate nature of domain-specific knowledge can be difficult for even the most experienced expert to conceptualise or articulate. One promising solution to the problem of representing complex domain abstractions involves the use of conceptual metaphors. Previous applications of conceptual metaphors to abstract domains have yielded encouraging results. However, the design of appropriate methods for eliciting conceptual metaphors for the purposes of informing interaction design remains an open question. In this paper, we report on a series of studies carried out to elicit conceptual metaphors from domain experts, using music as a case study, reflecting on the benefits and drawbacks of each approach

Evaluating musical software using conceptual metaphors

An open challenge for interaction designers is to find ways of designing software to enhance the ability of novices to perform tasks that normally require specialized domain expertise. This challenge is particularly demanding in areas such as music analysis, where complex, abstract, domain-specific concepts and notations occur. One promising theoretical foundation for this work involves the identification of conceptual metaphors and image schemas, found by analyzing discourse. This kind of analysis has already been applied, with some success, both to musical concepts and, separately, to user interface design. The present work appears to be the first to combine these hitherto distinct bodies of research, with the aim of devising a general method for improving user interfaces for music. Some areas where this may require extensions to existing method are noted. This paper presents the results of an exploratory evaluation of Harmony Space, a tool for playing, analysing and learning about harmony. The evaluation uses conceptual metaphors and image schemas elicited from the dialogues of experienced musicians discussing the harmonic progressions in a piece of music. Examples of where the user interface supports the conceptual metaphors, and where support could be improved, are discussed. The potential use of audio output to support conceptual metaphors and image schemas is considered

Towards a participatory approach for interaction design based on conceptual metaphor theory: a case study from music interaction

"Music Interaction" is the term for interaction design within the domain of music. In areas such as music, the ability to engage effectively in certain activities tends to be restricted to those who have acquired detailed knowledge of domain-specific theories, terminologies, concepts or processes. It can be challenging to design or enhance user interfaces for software able to support novices in these kinds of musical activities. One promising approach to this challenge involves translating musicians’ implicit domain knowledge into patterns known as conceptual metaphors, which are metaphorical ex-tensions of recurring patterns of embodied experience applied to abstract domains, and using this information to inform interaction designs for music. This approach has been applied experimentally with some success to designing user interfaces. However, to the best of our knowledge, this present work is the first to consider in detail the use of Conceptual Metaphor Theory as a key component of a partici-patory design process. In this chapter we present a participatory approach to Music Interaction design based on the principles of Conceptual Metaphor Theory. We posit that such an approach will facilitate the development of innovative and intuitive interaction designs for both novices and experts alike

Whole body interaction in abstract domains

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations.Published versio

Whole body interaction in abstract domains

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations

What can the language of musicians tell us about music interaction design?

It is difficult to create good interaction designs for music software or to substantially improve existing designs. One reason is that music involves diverse and complex concepts, entities, relationships, processes, terminologies, and notations. An open challenge for interaction designers is to find systematic ways of channeling the tacit, specialized knowledge of musicians into designs for intuitive user interfaces that can capably support musically skilled users, without excluding those with less technical musical knowledge. One promising new approach to this challenge involves the application of research from the theory of image schemas and conceptual metaphors This theory has already been applied with some success to analyzing musical concepts, mathematical concepts, general-purpose user-interface design, and sound-generation interaction designs involving simple musical parameters such as tempo, volume, and pitch. We propose that by identifying the conceptual metaphors and image schemas used by musical experts when analyzing an excerpt of music, and then assessing the extent to which these conceptual metaphors are supported by existing music-interaction designs, it is possible to identify areas where the designs do not match musicians’ understanding of the domain concepts. This process provides a principled basis for identifying points at which designs could be improved to better support musicians’ understanding and tasks, and for the provisional identification of possible improvements. We present the results of an analysis of dialogue between three musicians as they discuss an excerpt of music. A methodology for the systematic identification of image schemas and conceptual metaphors is detailed. We report on the use of the results of the dialogue analysis to illuminate the designs of two contrasting examples of music software. Areas in which the designs might be made more intuitive are identified, and corresponding suggestions for improvements are outlined. The present research appears to be the first to investigate the potential of conceptual metaphor theory for investigating and improving music-interaction designs that deal with complex musical concepts, such as harmonic progressions, modulation, and voice leading

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research