The Interpersonal Entrainment in Music Performance Data Collection

The Interpersonal Entrainment in Music Performance Data Collection (IEMPDC) comprises six related corpora of music research materials: Cuban Son & Salsa (CSS), European String Quartet (ESQ), Malian Jembe (MJ), North Indian Raga (NIR), Tunisian Stambeli (TS), and Uruguayan Candombe (UC). The core data for each corpus comprises media files and computationally extracted event onset timing data. Annotation of metrical structure and code used in the preparation of the collection is also shared. The collection is unprecedented in size and level of detail and represents a significant new resource for empirical and computational research in music. In this article we introduce the main features of the data collection and the methods used in its preparation. Details of technical validation procedures and notes on data visualization are available as Appendices. We also contextualize the collection in relation to developments in Open Science and Open Data, discussing important distinctions between the two related concepts

Automated Analysis of Synchronization in Human Full-body Expressive Movement

The research presented in this thesis is focused on the creation of computational models for the study of human full-body movement in order to investigate human behavior and non-verbal communication. In particular, the research concerns the analysis of synchronization of expressive movements and gestures. Synchronization can be computed both on a single user (intra-personal), e.g., to measure the degree of coordination between the joints\u2019 velocities of a dancer, and on multiple users (inter-personal), e.g., to detect the level of coordination between multiple users in a group. The thesis, through a set of experiments and results, contributes to the investigation of both intra-personal and inter-personal synchronization applied to support the study of movement expressivity, and improve the state-of-art of the available methods by presenting a new algorithm to perform the analysis of synchronization

A Practice-Based Study into the Composition and Performance of Polytemporal Music

This practice-based research explores the composition and performance of polytemporal music, culminating in ten new works in audio/visual format with accompanying commentaries and notation. Research is undertaken into concepts of rhythm and pulse in order to develop new techniques for composing music in multiple simultaneous tempi, particularly methods for managing rhythmic consonance and dissonance in the compositional process. Attention is also given to the practicalities and implications of performance, investigating issues of accessibility and ensemble in reference to the use of click tracks and headphones, as well as the form and function of notation. The approaches within this research stem from my experience as a commercial rock/studio musician fused with contemporary classical influences. As well as these musical influences, a background in visual art and design also contributes to the visual presentation of works and scores; musical works are presented in video format which is shown to enhance temporal perception, and a new form of rhythmically accurate western notation for polytemporal music is developed. Composing and performing in a strictly polytemporal setting has at the time of writing not been widely researched, and it is hoped this work displays new knowledge and approaches important for the development of composition in this area

Measuring musical interaction: analysing communication in embodied musical behaviour

This thesis addresses the ubiquity and necessity of embodied interaction to musical activity, using video analysis to observe communication in musical events. Through the specific study of classical North Indian instrumental duo performance, the thesis examines how processes of social interaction may inform human musical activity, using a combined methodology of ethnographic study and quantitative data analysis of original video-recordings. Proposing a pragmatic approach to the study of the meaningful nature of musical events, the thesis keeps sight of the generative context of the human body in social interaction, and offers a model of musical communication that privileges nonlinguistic, socially co-regulative elements in its account of human musical interaction. The socially meaningful nature of the behaviour-in-time of the musicians included in the study is investigated by means of a novel methodology. This combines the qualitative exploration of emic concepts related to the practice of North Indian classical music with an empirical analysis of video data, based on a cognitive ethological framework. The thesis draws on current notions of embodied cognition and contributes to the growing corpus of musicological literature emphasising the embodied and social nature of musical communication. The results of this exploratory study suggest that both social-interaction and music-structural factors contribute to the organisation of the musicians' communicative behaviours and that, to a certain extent, these organisational factors can be separated in analysis

The change matters! Measuring the effect of changing the leader in joint music performances

In a joint action, a group of individuals coordinate their movements to reach a shared goal. When a change - i.e., an event that affects group functioning - occurs, the group adopts strategies to face it. This paper investigates how a change involving a strategic core role in a group affects interpersonal coordination and ultimately group effectiveness in performing a joint action. Following the entrainment theory, interpersonal coordination is addressed in terms of the rhythmic cycles of the individuals and of the group and their adjustment. Music is used as an ideal ecological scenario for investigation. Results show that whereas the change of conductor had a limited significant effect on entrainment, a significant effect was found when entrainment is used as a predictor of the external ratings. Both the obtained results and the techniques developed for measuring entrainment may open novel research directions in the area of automated analysis of group behavior, and particularly of emotion in groups

Visual cues in musical synchronisation

Although music performance is generally thought of as an auditory activity in the Western tradition, the presence of continuous visual information in live music contributes to the cohesiveness of music ensembles, which presents an interesting psychological phenomenon in which audio and visual cues are presumably integrated. In order to investigate how auditory and visual sensory information are combined in the basic process of synchronising movements with music, this thesis focuses on both musicians and nonmusicians as they respond to two sources of visual information common to ensembles: the conductor, and the ancillary movements (movements that do not directly create sound; e.g. body sway or head nods) of co-performers. These visual cues were hypothesized to improve the timing of intentional synchronous action (matching a musical pulse), as well as increasing the synchrony of emergent ancillary movements between participant and stimulus. The visual cues were tested in controlled renderings of ensemble music arrangements, and were derived from real, biological motion. All three experiments employed the same basic synchronisation task: participants drummed along to the pulse of tempo-changing music while observing various visual cues. For each experiment, participants’ drum timing and upper-body movements were recorded as they completed the synchronisation task. The analyses used to quantify drum timing and ancillary movements came from theoretical approaches to movement timing and entrainment: information processing and dynamical systems. Overall, this thesis shows that basic musical timing is a common ability that is facilitated by visual cues in certain contexts, and that emergent ancillary movements and intentional synchronous movements in combination may best explain musical timing and synchronisation

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies

A breathing sonification system to reduce stress during the COVID-19 pandemic

Since sound and music are powerful forces and drivers of human behavior and physiology, we propose the use of sonification to activate healthy breathing patterns in participants to induce relaxation. Sonification is often used in the context of biofeedback as it can represent an informational, non-invasive and real-time stimulus to monitor, motivate or modify human behavior. The first goal of this study is the proposal and evaluation of a distance-based biofeedback system using a tempo- and phase-aligned sonification strategy to adapt breathing patterns and induce states of relaxation. A second goal is the evaluation of several sonification stimuli on 18 participants that were recruited online and of which we analyzed psychometric and behavioral data using, respectively questionnaires and respiration rate and ratio. Sonification stimuli consisted of filtered noise mimicking a breathing sound, nature environmental sounds and a musical phrase. Preliminary results indicated the nature stimulus as most pleasant and as leading to the most prominent decrease of respiration rate. The noise sonification had the most beneficial effect on respiration ratio. While further research is needed to generalize these findings, this study and its methodological underpinnings suggest the potential of the proposed biofeedback system to perform ecologically valid experiments at participants' homes during the COVID-19 pandemic