Audiovisual integration of emotional signals from others' social interactions

Audiovisual perception of emotions has been typically examined using displays of a solitary character (e.g., the face-voice and/or body-sound of one actor). However, in real life humans often face more complex multisensory social situations, involving more than one person. Here we ask if the audiovisual facilitation in emotion recognition previously found in simpler social situations extends to more complex and ecological situations. Stimuli consisting of the biological motion and voice of two interacting agents were used in two experiments. In Experiment 1, participants were presented with visual, auditory, auditory filtered/noisy, and audiovisual congruent and incongruent clips. We asked participants to judge whether the two agents were interacting happily or angrily. In Experiment 2, another group of participants repeated the same task, as in Experiment 1, while trying to ignore either the visual or the auditory information. The findings from both experiments indicate that when the reliability of the auditory cue was decreased participants weighted more the visual cue in their emotional judgments. This in turn translated in increased emotion recognition accuracy for the multisensory condition. Our findings thus point to a common mechanism of multisensory integration of emotional signals irrespective of social stimulus complexity

Computational musicology: An Artificial Life approach

Abstract — Artificial Life (A-Life) and Evolutionary Algorithms (EA) provide a variety of new techniques for making and studying music. EA have been used in different musical applications, ranging from new systems for composition and performance, to models for studying musical evolution in artificial societies. This paper starts with a brief introduction to three main fields of application of EA in Music, namely sound design, creativity and computational musicology. Then it presents our work in the field of computational musicology. Computational musicology is broadly defined as the study of Music with computational modelling and simulation. We are interested in developing A-Life-based models to study the evolution of musical cognition in an artificial society of agents. In this paper we present the main components of a model that we are developing to study the evolution of musical ontogenies, focusing on the evolution of rhythms and emotional systems. The paper concludes by suggesting that A-Life and EA provide a powerful paradigm for computational musicology. I

Software agents in music and sound art research/creative work: Current state and a possible direction

Composers, musicians and computer scientists have begun to use software-based agents to create music and sound art in both linear and non-linear (non-predetermined form and/or content) idioms, with some robust approaches now drawing on various disciplines. This paper surveys recent work: agent technology is first introduced, a theoretical framework for its use in creating music/sound art works put forward, and an overview of common approaches then given. Identifying areas of neglect in recent research, a possible direction for further work is then briefly explored. Finally, a vision for a new hybrid model that integrates non-linear, generative, conversational and affective perspectives on interactivity is proposed

Gestural control of sonic swarms: Composing with grouped sound objects

This paper outlines an alternative controller designed to diffuse and manipulate a swarm of sounds in 3- dimensional space and discusses the compositional issues that emerge from its use. The system uses an algorithm from a nature-derived model describing the spatial behavior of a swarm. The movement of the swarm is mapped in the 3- dimensional space and a series of sound transformation functions for the sonic agents are implemented. The notion of causal relationships is explored regarding the spatial movement of the swarm and sound transformation of the agents by employing the physical controller as a performance, compositional and diffusion tool

Complexity as Process: Complexity Inspired Approaches to Composition

This article examines the use of Complexity Theory as an inspiration for the creation of new musical works, and highlights problems and possible solutions associated with its application as a compositional tool. In particular it explores how the philosophy behind Complexity Theory affects notions of process-based composition, indeterminacy in music and the performer/listener/environment relationship, culminating in providing a basis for the understanding of music creation as an active process within a context. The author presents one of his own sound installations, Cross-Pollination, as an example of a composition inspired and best understood from the philosophical position as described in Complexity Theory

Key Components of Musical Discourse Analysis

Musical discourse analysis is an interdisciplinary study which is incomplete without consideration of relevant social, linguistic, psychological, visual, gestural, ritual, technical, historical and musicological aspects. In the framework of Critical Discourse Analysis, musical discourse can be interpreted as social practice: it refers to specific means of representing specific aspects of the social (musical) sphere. The article introduces a general view of contemporary musical discourse, and analyses genres from the point of ‘semiosis’, ‘social agents’, ‘social relations’, ‘social context’, and ‘text’. These components of musical discourse analysis, in their various aspects and combinations, should help thoroughly examine the context of contemporary musical art, and determine linguistic features specific to different genres of musical discourse

Filterscape: energy recycling in a creative ecosystem

This paper extends previous work in evolutionary ecosystemic approaches to generative art. Filterscape, adopts the implicit fitness specification that is fundamental to this approach and explores the use of resource recycling as a means of generating coherent sonic diversity in a generative sound work. Filterscape agents consume and deposit energy that is manifest in the simulation as sound. Resource recycling is shown to support cooperative as well as competitive survival strategies. In the context of our simulation, these strategies are recognised by their characteristic audible signatures. The model provides a novel means to generate sonic diversity through de-centralised agent interactions

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion