Multi-layer adaptation of group coordination in musical ensembles

Group coordination passes through an efficient integration of multimodal sources of information. This study examines complex non-verbal communication by recording movement kinematics from conductors and two sections of violinists of an orchestra adapting to a perturbation affecting their normal pattern of sensorimotor communication (rotation of half a turn of the first violinists’ section). We show that different coordination signals are channeled through ancillary (head kinematics) and instrumental movements (bow kinematics). Each one of them affect coordination either at the inter-group or intra-group levels, therefore tapping into different modes of cooperation: complementary versus imitative coordination. Our study suggests that the co-regulation of group behavior is based on the exchange of information across several layers, each one of them tuned to carry specific coordinative signals. Multi-layer sensorimotor communication may be the key musicians and, more generally humans, use to flexibly communicate between each other in interactive sensorimotor tasks

Multi-layer adaptation of group coordination in musical ensembles

Group coordination passes through an efficient integration of multimodal sources of information. This study examines complex non-verbal communication by recording movement kinematics from conductors and two sections of violinists of an orchestra adapting to a perturbation affecting their normal pattern of sensorimotor communication (rotation of half a turn of the first violinists\u2019 section). We show that different coordination signals are channeled through ancillary (head kinematics) and instrumental movements (bow kinematics). Each one of them affect coordination either at the inter-group or intra-group levels, therefore tapping into different modes of cooperation: complementary versus imitative coordination. Our study suggests that the co-regulation of group behavior is based on the exchange of information across several layers, each one of them tuned to carry specific coordinative signals. Multi-layer sensorimotor communication may be the key musicians and, more generally humans, use to flexibly communicate between each other in interactive sensorimotor tasks

Musical coordination in a large group without plans nor leaders

A widespread belief is that large groups engaged in joint actions that require a high level of flexibility are unable to coordinate without the introduction of additional resources such as shared plans or hierarchical organizations. Here, we put this belief to a test, by empirically investigating coordination within a large group of 16 musicians performing collective free improvisation—a genre in which improvisers aim at creating music that is as complex and unprecedented as possible without relying on shared plans or on an external conductor. We show that musicians freely improvising within a large ensemble can achieve significant levels of coordination, both at the level of their musical actions (i.e., their individual decisions to play or to stop playing) and at the level of their directional intentions (i.e., their intentions to change or to support the music produced by the group). Taken together, these results invite us to reconsider the range and scope of actions achievable by large groups, and to explore alternative organizational models that emphasize decentralized and unscripted forms of collective behavior

Participatory Sense Making in Jazz Performance: Agents’ Expressive Alignment

Relationships between musicians in Jazz performance can be understood as autonomous (turn-taking) versus simultaneous (playing joint), both assumed as social interactions that take place as to create meaning in a participatory way. To participate, in music performance, requires expressive alignment, in order to share the act of producing and perceiving sound and movement in an embodied- inter(en)acted phenomenological experience. In such context, interaction is assumed as an expressive exchange of meanings. In this work, we study a trio jazz performance from an inter(en)acted approach, applying a methodological design that combines objective/statistical measures, and subjective/phenomenological data. An experiment that tested different conditions of turn-taking and/or joint playing of a Jazz standard was conducted in a recording studio session. All the performances were registered through audio/video media, and motion capture technology. In addition, in-depth interviews before playing/after recordings were conducted. Time series data related to sound and movement were analysed to study features of expressive alignment, accounting for descriptors of participatory sense-making. A Sense Granger measure was developed from Granger Causality measures in order to describe expressive alignment between-and-within performers. Significant differences were found in situations of turn-taking, and simultaneous playing between conditions. Results show that, beyond such differences, jazz musicians sustain interactional transactions based on their phenomenological experience of ‘going together in time’. Sense-Granger measures serve to account for the ways expressive alignment evolves over time, providing significant cues that help to understand participatory sense-making in jazz performance.Trabajo publicado en Proceedings of the 25th Anniversary Conference of the European Society for the Cognitive Sciences of Music, 31 July-4 August 2017, Ghent, Belgium Van Dyck, E. (Editor).Facultad de Bellas Artes (FBA

Neural and motor basis of inter-individual interactions

The goal of my Ph.D. work was to investigate the behavioral markers and the brain activities responsible for the emergence of sensorimotor communication. Sensorimotor communication can be defined as a form of communication consisting into flexible exchanges based on bodily signals, in order to increase the efficiency of the inter-individual coordination. For instance, a soccer player carving his movements to inform another player about his intention. This form of interaction is highly dependent of the motor system and the ability to produce appropriate movements but also of the ability of the partner to decode these cues. To tackle these facets of human social interaction, we approached the complexity of the problem by splitting my research activities into two separate lines of research. First, we pursued the examination of motor-based humans\u2019 capability to perceive and \u201cread\u201d other\u2019s behaviors in focusing on single-subject experiment. The discovery of mirror neurons in monkey premotor cortex in the early nineties (di Pellegrino et al. 1992) motivated a number of human studies on this topic (Rizzolatti and Craighero 2004). The critical finding was that some ventral premotor neurons are engaged during visual presentation of actions performed by conspecifics. More importantly, those neurons were shown to encode also the actual execution of similar actions (i.e. irrespective of who the acting individual is). This phenomenon has been highly investigated in humans by using cortical and cortico-spinal measures (for review see, fMRI: Molenberghs, Cunnington, and Mattingley 2012; TMS: Naish et al. 2014; EEG: Pineda 2008). During single pulse TMS (over the primary motor cortex), the amplitude of motor evoked potentials (MEPs) provides an index of corticospinal recruitment. During action observation the modulation of this index follow the expected changes during action execution (Fadiga et al. 1995). However, dozens of studies have been published on this topic and revealed important inconsistencies. For instance, MEPs has been shown to be dependent on observed low-level motor features (e.g. kinematic features or electromyography temporal coupling; Gangitano, Mottaghy, and Pascual-Leone 2001; Borroni et al. 2005; Cavallo et al. 2012) as well as high level movement properties (e.g. action goals; Cattaneo et al. 2009; Cattaneo et al. 2013). Furthermore, MEPs modulations do not seem to be related to the observed effectors (Borroni and Baldissera 2008; Finisguerra et al. 2015; Senna, Bolognini, and Maravita 2014), suggesting their independence from low-level movement features. These contradictions call for new paradigms. Our starting hypothesis here is that the organization and function of the mirror mechanism should follow that of the motor system during action execution. Hence, we derived three action observation protocols from classical motor control theories: 1) The first study was motivated by the fact that motor redundancy in action execution do not allow the presence of a one-to-one mapping between (single) muscle activation and action goals. Based on that, we showed that the effect of action observation (observation of an actor performing a power versus a precision grasp) are variable at the single muscle level (MEPs; motor evoked potentials) but robust when evaluating the kinematic of TMS-evoked movements. Considering that movements are based on the coordination of multiple muscle activations (muscular synergies), MEPs may represent a partial picture of the real corticospinal activation. Inversely, movement kinematics is both the final functional byproduct of muscles coordination and the sole visual feedback that can be extracted from action observation (i.e. muscle recruitment is not visible). We conclude that TMS-evoked kinematics may be more reliable in representing the state of the motor system during action observation. 2) In the second study, we exploited the inter-subject variability inherent to everyday whole-body human actions, to evaluate the link between individual motor signatures (or motor styles) and other\u2019s action perception. We showed no group-level effect but a robust correlation between the individual motor signature recorded during action execution and the subsequent modulations of corticospinal excitability during action observation. However, results were at odds with a strict version of the direct matching hypothesis that would suggest the opposite pattern. In fact, the more the actor\u2019s movement was similar to the observer\u2019s individual motor signature, the smaller was the MEPs amplitude, and vice versa. These results conform to the predictive coding hypothesis, suggesting that during AO, the motor system compares our own way of doing the action (individual motor signature) with the action displayed on the screen (actor\u2019s movement). 3) In the third study, we investigated the neural mechanisms underlying the visual perception of action mistakes. According to a strict version of the direct matching hypothesis, the observer should potentially reproduce the neural activation present during the actual execution of action errors (van Schie et al. 2004). Here, instead of observing an increase of cortical inhibition, we showed an early (120 ms) decrease of intracortical inhibition (short intracortical inhibition) when a mismatch was present between the observed action (erroneous) and the observer\u2019s expectation. As proposed by the predictive coding framework, the motor system may be involved in the generation of an error signal potentially relying on an early decrease of intracortical inhibition within the corticomotor system. The second line of research aimed at the investigation of how sensorimotor communication flows between agents engaged in a complementary action coordination task. In this regard, measures of interest where related to muscle activity and/or kinematics as the recording of TMS-related indexes would be too complicated in a joint-action scenario. 1) In the first study, we exploited the known phenomenon of Anticipatory Postural Adjustments (APAs). APAs refers to postural adjustments made in anticipation of a self- or externally-generated disturbance in order to cope for the predicted perturbation and stabilize the current posture. Here we examined how observing someone else lifting an object we hold can affect our own anticipatory postural adjustments of the arm. We showed that the visual information alone (joint action condition), in the absence of efference copy (present only when the subject is unloading by himself the object situated on his hand), were not sufficient to fully deploy the needed anticipatory muscular activations. Rather, action observation elicited a dampened APA response that is later augmented by the arrival of tactile congruent feedback. 2) In a second study, we recorded the kinematic of orchestra musicians (one conductor and two lines of violinists). A manipulation was added to perturb the normal flow of information conveyed by the visual channel. The first line of violinist where rotated 180\ub0, and thus faced the second line. Several techniques were used to extract inter-group (Granger Causality method) and intra-group synchronization (PCA for musicians and autoregression for conductors). The analyses were directed to two kinematic features, hand and head movements, which are central for functionally different action. The hand is essential for instrumental actions, whereas head movements encode ancillary expressive actions. During the perturbation, we observed a complete reshaping of the whole patterns of communication going in the direction of a distribution of the leadership between conductor and violinists, especially for what regards head movements. In fact, in the perturbed condition, the second line acts as an informational hub connecting the first line to the conductor they no longer can see. This study evidences different forms of communications (coordination versus synchronization) flowing via different channels (ancillary versus instrumental) with different time-scales

Participatory Sense Making in Jazz Performance: Agents’ Expressive Alignment

Relationships between musicians in Jazz performance can be understood as autonomous (turn-taking) versus simultaneous (playing joint), both assumed as social interactions that take place as to create meaning in a participatory way. To participate, in music performance, requires expressive alignment, in order to share the act of producing and perceiving sound and movement in an embodied- inter(en)acted phenomenological experience. In such context, interaction is assumed as an expressive exchange of meanings. In this work, we study a trio jazz performance from an inter(en)acted approach, applying a methodological design that combines objective/statistical measures, and subjective/phenomenological data. An experiment that tested different conditions of turn-taking and/or joint playing of a Jazz standard was conducted in a recording studio session. All the performances were registered through audio/video media, and motion capture technology. In addition, in-depth interviews before playing/after recordings were conducted. Time series data related to sound and movement were analysed to study features of expressive alignment, accounting for descriptors of participatory sense-making. A Sense Granger measure was developed from Granger Causality measures in order to describe expressive alignment between-and-within performers. Significant differences were found in situations of turn-taking, and simultaneous playing between conditions. Results show that, beyond such differences, jazz musicians sustain interactional transactions based on their phenomenological experience of ‘going together in time’. Sense-Granger measures serve to account for the ways expressive alignment evolves over time, providing significant cues that help to understand participatory sense-making in jazz performance.Trabajo publicado en Proceedings of the 25th Anniversary Conference of the European Society for the Cognitive Sciences of Music, 31 July-4 August 2017, Ghent, Belgium Van Dyck, E. (Editor).Facultad de Bellas Artes (FBA

Participatory Sense Making in Jazz Performance: Agents’ Expressive Alignment

Relationships between musicians in Jazz performance can be understood as autonomous (turn-taking) versus simultaneous (playing joint), both assumed as social interactions that take place as to create meaning in a participatory way. To participate, in music performance, requires expressive alignment, in order to share the act of producing and perceiving sound and movement in an embodiedinter(en)acted phenomenological experience. In such context, interaction is assumed as an expressive exchange of meanings. In this work we study a trio jazz performance from an inter(en)acted approach, applying a methodological design that combines objective/statistical measures, and subjective/phenomenological data. An experiment that tested different conditions of turn-taking and/or joint playing of a Jazz standard was conducted in a recording studio session. All the performances were registered through audio/video media, and motion capture technology. In addition, in-depth interviews before playing/after recordings were conducted. Time series data related to sound and movement were analysed to study features of expressive alignment, accounting for descriptors of participatory sense-making. A Sense Granger measure was developed from Granger Causality measures in order to describe expressive alignment between-and-within performers. Significant differences were found in situations of turn-taking, and simultaneous playing between conditions. Results show that, beyond such differences, jazz musicians sustain interactional transactions based on their phenomenological experience of ‘going together in time’. Sense Granger measures serve to account for the ways expressive alignment evolves in time, providing significant cues that help understanding participatory sense making in jazz performance.Trabajo publicado en "Proceedings of the 25th Anniversary Conference of the European Society for the Cognitive Sciences of Music", 31 July-4 August 2017, Ghent, Belgium Van Dyck, E. (Editor)

Knee flexion of saxophone players anticipates tonal context of music

Music performance requires high levels of motor control. Professional musicians use body movements not only to accomplish and help technical efficiency, but to shape expressive interpretation. Here, we recorded motion and audio data of twenty participants performing four musical fragments varying in the degree of technical difficulty to analyze how knee flexion is employed by expert saxophone players. Using a computational model of the auditory periphery, we extracted emergent acoustical properties of sound to inference critical cognitive patterns of music processing and relate them to motion data. Results showed that knee flexion is causally linked to tone expectations and correlated to rhythmical density, suggesting that this gesture is associated with expressive and facilitative purposes. Furthermore, when instructed to play immobile, participants tended to microflex (>1 Hz) more frequently compared to when playing expressively, possibly indicating a natural urge to move to the music. These results underline the robustness of body movement in musical performance, providing valuable insights for the understanding of communicative processes, and development of motor learning cues.info:eu-repo/semantics/publishedVersio

Knee flexion of saxophone players anticipates tonal context of music

Music performance requires high levels of motor control. Professional musicians use body movements not only to accomplish and help technical efficiency, but to shape expressive interpretation. Here, we recorded motion and audio data of twenty participants performing four musical fragments varying in the degree of technical difficulty to analyze how knee flexion is employed by expert saxophone players. Using a computational model of the auditory periphery, we extracted emergent acoustical properties of sound to inference critical cognitive patterns of music processing and relate them to motion data. Results showed that knee flexion is causally linked to tone expectations and correlated to rhythmical density, suggesting that this gesture is associated with expressive and facilitative purposes. Furthermore, when instructed to play immobile, participants tended to microflex (>1 Hz) more frequently compared to when playing expressively, possibly indicating a natural urge to move to the music. These results underline the robustness of body movement in musical performance, providing valuable insights for the understanding of communicative processes, and development of motor learning cues.Grant 2020.05257.BD of the Portuguese Foundation for Science and TechnologyFSE programme through Programa Operacional Regional NorteFlemish Government, the project FQM-307 of the Government of Andalusia (SpainProject PID2020-113961GB-I00 of the Spanish Ministry of Science and Innovation (also supported by the FEDER programme).Consejería de Conocimiento, Investigación y Universidad, Junta de Andalucía (Spain)FEDER programme for the project A-FQM-66-UGR20IMAG-María de Maeztu grant CEX2020-001105-M/AEI/ 10.13039/50110001103