The haptic iPod: passive learning of multi-limb rhythm skills

Recent experiments showed that the use of haptic vibrotactile devices can support the learning of multi-limb rhythms [Holland et al., 2010]. These experiments centred on a tool called the Haptic Drum Kit, which uses vibrotactiles attached to wrists and ankles, together with a computer system that controls them, and a midi drum kit. The system uses haptic signals in real time, relying on human entrainment mechanisms [Clayton, Sager and Will, 2004] rather than stimulus response, to support the user in playing multi-limbed rhythms. In the present paper, we give a preliminary report on a new experiment, that aims to examine whether passive learning of multi-limb rhythms can occur through the silent playback of rhythmic stimuli via haptics when the subject is focusing on other tasks. The prototype system used for this new experiment is referred to as the Haptic iPod.Paper presented at the Workshop: When Words Fail: What can Music Interaction tell us about HCI? at BCS HCI Conference 2011, Newcastle, U

Song walker harmony space: embodied interaction design for complex musical skills

Tonal harmony is widely considered to be the most technical and complex part of music theory. Consequently harmonic skills can be hard to acquire. Furthermore, experience of the flexible manipulation of harmony in real time generally requires the ability to play an instrument. Even for those with instrumental skills, it can be difficult to gain clear insight into harmonic abstractions. The above state of affairs gives rise to substantial barriers not only for beginners but also for many experienced musicians. To address these problems, Harmony Space (Holland et al. 2009) is an interactive digital music system designed to give insight into a wide range of musical tasks in tonal harmony, ranging from performance and composition to analysis. Harmony Space employs a principled set of spatial mappings to offer fluid, precise, intuitive control of harmony. These mappings give rise to sensory-motor and music-theoretic affordances that are hard to obtain in any other way. As a result, harmonic abstractions are rendered amenable to concrete, visible control by simple spatial manipulation. In the language of conceptual metaphor theory, many relationships in tonal harmony become accessible to rapid, universal, low-level, robust human inference mechanisms using image schemata such as containment, contact, centre-periphery, and source-path-goal. This process is more rapid, and imposes far less cognitive load, than slow, abstract symbolic reasoning. Using the above principles, several versions of Harmony Space have been de-signed to exploit specific interaction styles for different purposes. We note some key variants, such as the desktop version (Holland 1994), the camera tracked version (Holland et al. 2009), while focusing principally on the most recent version, Song Walker (Holland et al. 2011), which employs whole body interaction. Preliminary results from a study of the Song Walker system are outlined, in which both beginners and expert musicians undertook a range of musical tasks involving the performance, composition and analysis of music. Finally, we offer a discussion of the limitations of the current system, and outline directions for future wor

Whole body interaction in abstract domains

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations.Published versio

Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit

This paper introduces and explores a tool known as the Haptic Drum Kit. The Haptic Drum Kit employs four computer-controlled vibrotactile devices, one attached to each limb via the wrists and ankles. In the mode of use discussed in this paper, haptic pulses are used to guide the playing, on a drum kit, of rhythmic patterns that require multi-limb co-ordination. The immediate aim is to foster rhythm skills and multi-limb coordination. A broader aim is to systematically develop skills in recognizing, identifying, memorizing, retaining, analyzing, reproducing and composing monophonic and polyphonic rhythms. We consider the implications of three different theories for this approach: the work of the music educator Dalcroze (1865-1950 [1]; the entrainment theory of human rhythm perception and production [2,3]; and sensory motor contingency theory [4]. In this paper we introduce the Haptic Drum Kit; consider the implications of the above theories for this approach; report on a design study; and identify and discuss a variety of emerging design issues. As part of the design study, audio and haptic guidance was compared for five people learning to play polyphonic drum patterns of varying complexity. The results indicate that beginning drummers are able to learn intricate drum patterns from the haptic stimuli alone, although haptic plus audio is the mode of presentation preferred by subjects

Song Walker: Embodied interaction design for harmony

Tonal Harmony is widely considered to be the most technical and complex part of music theory, and harmonic skills can be hard to acquire. Experience of the precise and flexible manipulation of harmony in real time generally requires hard-won instrumental skill. Even with instrumental skills, it can be hard to gain clear insight into harmonic abstractions. The above state of affairs gives rise to substantial barriers not only for beginners but also for many musicians. To address these problems, Harmony Space [Holland et al, 2009] is an interactive digital music system designed to give insight into a wide range of musical tasks in tonal harmony ranging from performance and composition to analysis. Harmony Space employs a principled set of spatial mappings to offer fluid, precise, intuitive control of harmony. These mappings give rise to sensory-motor, music-theoretic and information-theoretic affordances that are not readily obtainable in any other way. The result is that a wide range of harmonic abstractions are rendered amenable to concrete, visible manipulation by simple spatial means. In the language of conceptual metaphor theory, most relationships in tonal harmony become accessible, to rapid, universal, low-level, robust human inference mechanisms using image schema, such as containment, contact, centre-periphery, and source-path-goal, in place of slow, abstract symbolic reasoning. While keeping the above principles invariant, different versions of Harmony Space have been designed to exploit different detailed interaction styles for different purposes. We note some key variants, such as the desktop version [Holland, 1994], the camera tracked version [Holland et al., 2009], and the most recent whole body version, Song Walker [Holland et al., 2011]. Preliminary results from a recent study of the Song Walker system are outlined, in which both beginners and expert musicians undertook a range of solo and collaborative musical tasks involving the performance, composition and analysis of music. Finally, we offer a discussion of the limitations of the current system, and outline directions for future work.Paper presented at the workshop: When Words Fail: What can Music Interaction tell us about HCI? The 25th British Computer Society Conference on Human-Computer Interaction

Wonderful design: Applying Appraisal Theory to Procedural Level Generation

Procedural level generation for games is an active field of research with successful applications. However, how to generate content that embodies design intent is still an open research question. Level designers lack abstractions and tools for authoring generated artifacts for affecting emotion. We propose a novel pattern language for generative level design inspired by Appraisal Theory. Its patterns enable designers to add meaning, depth, and cohesiveness to the resulting content, and modify artifacts to make the content more engaging. We illustrate how these patterns can be implemented in a generative grammar for level generation for an adventure game. Formative evaluation of generated level content demonstrates the feasibility of the approach and suggests points for further improvements. Future work could focus on other elements which seem important for affecting emotions, including pacing, perception, and expectation

Towards a unified language for card game design

Card game creation is a powerful tool for game design.Using playing cards, game designers can rapidly prototype and iteratively playtest a game’s core mechanisms to explore alternatives and improve the gameplay. However, this process is time-consuming, imprecise and challenging to steer and focus. We aim to empower designers with solutions that automate game design processes. In particular, we study to what extent a unified game design language can offer the- oretical foundations, systematic techniques and practical solutions. We propose a novel approach towards a solution that leverages the expressive power of playing cards. Initially focusing on well- known card games, we illustrate the steps for creating CardScript, a formal language and toolkit that supports game design processes. The approach also has the potential to impact a wider research area. When fully developed, a unified language with a common tool set can enable reuse, and eventually support joint research agendas. We start the discussion by highlighting perspectives that relate open challenges to opportunities for future collaboration

UNLOCKing creative & innovative teaching in higher education extended abstracts

No abstract available.publishe

Graphical means for inspecting qualitative models of system behaviour

This article presents the design and evaluation of a tool for inspecting conceptual models of system behaviour. The basis for this research is the Garp framework for qualitative simulation. This framework includes modelling primitives, such as entities, quantities and causal dependencies, which are combined into model fragments and scenarios. Given a library of model fragments and a scenario describing an initial situation, the qualitative simulation engine generates predictions in the form of a state-transition graph. This rich knowledge representation has potential for educational purposes. However, communicating the contents of simulation models effectively to learners is not trivial. The predicate logic format used by Garp is not easy for non-experts to understand, and a simulation often contains so much information that it is difficult to get an overview while still having access to detailed information. To address these problems, a tool has been developed that generates graphical representations of the information contained in a qualitative simulation. This tool, named VisiGarp, incorporates a vocabulary of graphical elements for model ingredients and relationships, and combines these into interactive diagrams. VisiGarp has been evaluated by thirty students, with promising results, using a setup which included simulation results and exercises about Brazilian Cerrado ecology