Music and Language: Exploring an Artificial Music Grammar

Research regarding the brain mechanisms that underlie music and language processing supports two main interpretations: domain-specificity and domain-generality. Evidence from neuropsychology literature, specifically from amusia research, supports domain-specific mechanisms (Peretz & Coltheart, 2003) but recent neuroimaging and behavioral evidence supports overlapping mechanisms, especially for syntax processing (Patel, 2008). The present study used an artificial music grammar in order to test participants\u27 ability to learn a new music grammar as well as to observe a possible interaction between music and language syntax processing. Although participants were able to learn the artificial music grammar, a language task was not affected by errors in the new grammar as has been found with Western music-syntax errors (Sieve, Rosenberg, & Patel, 2009). Future research should consider extending exposure to the artificial grammar to allow for better learning in order for errors in the new grammar to affect the processing of language syntax

Whole body interaction in abstract domains

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations.Published versio

Musical combinatorics, tonnetz, and the CubeHarmonic

In this paper, we give an overview of some applications of combinatorics and permutations in music through the centuries. The concepts of permutation and tonnetz (spatial representation of voice leading and modulation) can be joined together in a physical device, the CubeHarmonic, a musical version of the Rubik’s cube. We finally describe a prototype of the CubeHarmonic that uses the magnetic tracking technology developed at the Tohoku University

Whole body interaction in abstract domains

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations

Negotiated Tutoring: An Approach to Interaction in Intelligent Tutoring Systems

This thesis describes a general approach to tutorial interaction in Intelligent Tutoring Systems, called "Negotiated Tutoring". Some aspects of the approach have been implemented as a computer program in the 'KANT' (Kritical Argument Negotiated Tutoring) system. Negotiated Tutoring synthesises some recent trends in Intelligent Tutoring Systems research, including interaction symmetry, use of explicit negotiation in dialogue, multiple interaction styles, and an emphasis on cognitive and metacognitive skill acquisition in domains characterised by justified belief. This combination of features has not been previously incorporated into models for intelligent tutoring dialogues. Our approach depends on modelling the high-level decision-making processes and memory representations used by a participant in dialogue. Dialogue generation is controlled by reasoning mechanisms which operate on a 'dialogue state', consisting of conversants' beliefs, a set of possible dialogue moves, and a restricted representation of the recent utterances generated by both conversants. The representation for conversants' beliefs is based on Anderson's (1983) model for semantic memory, and includes a model for dialogue focus based on spreading activation. Decisions in dialogue are based on preconditions with respect to the dialogue state, higher level educational preferences which choose between relevant alternative dialogue moves, and negotiation mechanisms designed to ensure cooperativity. The domain model for KANT was based on a cognitive model for perception of musical structures in tonal melodies, which extends the theory of Lerdahl and Jackendoff (1983). Our model ('GRAF' - GRouping Analysis with Frames) addresses a number of problems with Lerdahl and Jackendoff's theory, notably in describing how a number of unconscious processes in music cognition interact, including elements of top-down and bottom-up processing. GRAF includes a parser for musical chord functions, a mechanism for performing musical reductions, low-level feature detectors and a frame-system (Minsky 1977) for musical phrase structures

Music and language comprehension in the brain

Contains fulltext : 166652.pdf (publisher's version ) (Open Access)Radboud University, 10 februari 2017Promotor : Hagoort, P. Co-promotor : Willems, R.M.236 p