Synchronizing eye tracking and optical motion capture: How to bring them together

Both eye tracking and motion capture technologies are nowadays frequently used in human sciences, although both technologies are usually used separately. However, measuring both eye and body movements simultaneously would offer great potential for investigating cross- modal interaction in human (e.g. music and language-related) behavior. Here we combined an Ergoneers Dikablis head mounted eye tracker with a Qualisys Oqus optical motion cap- ture system. In order to synchronize the recordings of both devices, we developed a gener- alizable solution that does not rely on any (cost-intensive) ready-made / company-provided synchronization solution. At the beginning of each recording, the participant nods quickly while fixing on a target while keeping the eyes open – a motion yielding a sharp vertical displacement in both mocap and eye data. This displacement can be reliably detected with a peak-picking algorithm and used for accurately aligning the mocap and eye data. This method produces accurate synchronization results in the case of clean data and therefore provides an attractive alternative to costly plug-ins, as well as a solution in case ready-made synchronization solutions are unavailable

Brain and Body Percussion: The relationship between motor and cognitive functions

Body Percussion is a motor, rhythmic learning method within music education used to teach rhythmic structures. This activity is also used in common classrooms to improve students’ concentration and attention. Although the intervention is applied to enhance cognitive functions, effects of this method have not been studied neuroscientifically before. This study investigates embodied motor rhythmic exercises’ (e.g. Body Percussions) ability to enhance cognitive functions (planning skills). As the method has been applied successfully in music education for decades, results of this study were hypothesized to be positive.The long-term effects of Body Percussion on planning skills were studied with pre- and post-measures of an executive functions test (computed neuropsychological test Tower of London). The training period lasted 2.5 months and consisted of 10-20 minutes of weekly Body Percussion training sessions (altogether ten sessions). Twenty-four 5th graders (average age: 11 years) from a local elementary school participated in the study. After the Body Percussion training period the experiment group (N=12) performed better compared to the control group (N=12).Keywords: music education, rhythm, brain, executive function

Considerations Concerning a Methodology for Musical Robotics and Human-Robot Interaction

Robot technology is increasingly employed in artistic (musical) applications and as modeling tool for the investigation of general cognitive abilities and music related behavior in particular. Apart from the specifications of required system behavior and technological aspects of system design/implementation, problems occur concerning the evaluation of the systems. In recent approaches, techniques such as collecting informal reports, perceptual tests, video-based observational studies, or rating scales have been employed. Questions arise, however, as to the reliability and validity of these measures, and the lack of standardization diminishes the comparability of different studies. To attack these problems, we regard the paradigm of systematic observation as one suitable approach to study musical human-robot behavior. In a pilot study, observers were asked to assign labels to sequences of robot movements exhibiting features derived from expressive movements of musicians. Taking into account the implications of these remarks for the design of investigations, the generality of results may still be limited, e.g. by the choice of participants. As long as results of this kind cannot be integrated into a coherent theoretical framework, these considerations may run counter to recent attempts to use musical behavior as a testing field for principles underlying more general cognitive abilities/processes

Drumming Action and Perception: How the Movements of a Professional Drummer Influence Experiences of Tempo, Time, and Expressivity

The present study approached drumming motion from a three-fold perspective: computational movement feature analysis, perceptual evaluations of movement animations, and both in combination. We motion-captured a professional drummer performing two rhythms at two tempi with varying combinations of dynamics and amount of movement. Movement feature analysis indicated that the drummer varied their movement such that movement fluidity was higher in slow tempo compared to fast tempo, in particular for experimental conditions involving more movement and softer dynamics. Movement complexity was highest when playing slowly in the “much movement” condition. Subsequently, a perceptual online experiment was conducted using the drummer's animations with and without audio, asking observers ( n  = 114) to judge perceived passage of time, expressivity, and tempo. Results indicate that passage of time ratings were related to movement instructions, with much-movement performances being perceived to pass more quickly than little-movement ones. Furthermore, fast tempo stimuli were perceived to pass more quickly than slow tempo stimuli. Expressivity ratings were associated with the stimulus tempo as well, with fast tempo stimuli being rated as more expressive than slow tempo stimuli. Moreover, much-movement performances received higher ratings for expressivity, reinforcing the link between expressivity and movement. Including the movement features in the statistical model revealed that increased movement complexity and amount of movement related to higher perceived expressivity. While movement could be well distinguished between the different drumming conditions, the analysis regarding the other perceptual judgments (passage of time and tempo) indicated less influence of the movement characteristics, implying that participants rather focussed on other aspects when rating these. The outcomes suggest systematic links between bodily implementation of drumming and its perception related to time, tempo, and expressivity

See How It Feels to Move : Relationships Between Movement Characteristics and Perception of Emotions in Dance

Music makes humans move in ways found to relate to, for instance, musical characteristics, personality, or emotional content of the music. In this study, we investigated associations between embodiments of musical emotions and the perception thereof. After collecting motion capture data of dancers moving to emotionally distinct musical stimuli, silent stick-figure animations were rated by a set of observers regarding perceived discrete emotions, while 10 movement features were computationally extracted from the motion capture data. Results indicate kinematic profiles—emotion-specific sets of movement characteristics—that furthermore conform with dimensional models of valence and arousal, suggesting that observers rated the emotions consistently according to distinct movement features prevalent in the animations. Outcomes show commonalities and differences to a previous study that linked these movement features to auditory perception of musical emotion, providing insights into how emotional expression of music-induced movement could be conveyed and understood through auditory and visual channels, respectively.peerReviewe

MoCap Toolbox - A Matlab toolbox for computational analysis of movement data

The MoCap Toolbox is a set of functions written in Matlab for analyzing and visualizing motion capture data. It is aimed at investigating music-related movement, but can be beneficial for other research areas as well. Since the toolbox code is available as open source, users can freely adapt the functions according to their needs. Users can also make use of the additional functionality that Matlab offers, such as other toolboxes, to further analyze the features extracted with the MoCap Toolbox within the same environment. This paper describes the structure of the toolbox and its data representations, and gives an introduction to the use of the toolbox for research and analysis purposes. The examples cover basic visualization and analysis approaches, such as general data handling, creating stick-figure images and animations, kinematic and kinetic analysis, and performing Principal Component Analysis (PCA) on movement data, from which a complexity-related movement feature is derived.peerReviewe

Rhythmic recurrency in dance to music with ambiguous meter

Head movements of groups of participants moving to music allowing for both duple and triple subdivisions were analysed using Recurrence Quantification Analysis. As expected the maximum recurrence rate varied between participant, position in the music, and between groups. The lags of maximum recurrence were only partly corresponding to the metrical levels of the music and syncopated sections confused participants so that they moved to 4/4 rather than 6/8