Bossuyt, Frederick

Demey, Michiel

Leman, Marc

Vanfleteren, Jan

English

Ghent University Academic Bibliography

The Musical Synchrotron: using wireless motion sensors to study how social interaction affects synchronization with musical tempo  Michiel Demey, Marc Leman IPEM – Department of Musicology, Ghent University Blandijnberg 2 9000 Ghent, Belgium +32 (0)9 264 41 26 michiel.demey@ugent.beFrederick Bossuyt, Jan Vanfleteren TFCG Microsystems Lab, Ghent University Technologiepark Zwijnaarde 914 9052 Zwijnaarde +32 (0)9 264 53 54 jan.vanfleteren@ugent.be  ABSTRACT The Musical Synchrotron is a software interface that connects wireless motion sensors to a real-time interactive environment (Pure Data, Max/MSP). In addition to the measurement of movement, the system provides audio playback and visual feedback. The Musical Synchrotron outputs a score with the degree in which synchronization with the presented music is successful. The interface has been used to measure how people move in response to music. The system was used for experiments at public events.  Keywords Wireless sensors, tempo perception, social interaction, music and movement, embodied music cognition 1. Introduction Music is known to be a core component of the social and cultural cohesion of our society [1] [2] [3]. Yet the components of the social power of music, that is, the elements that contribute to social interaction are badly understood. In this paper, we describe a system called The Musical Synchrotron that enables the study on the relationship between music and social interaction. The Musical Synchrotron connects commercial and custom-made wireless motion sensors to a real-time interactive music environment. By integrating audio playback, kinetic monitoring and visual feedback, the system offers an interesting tool for the study of how social interaction affects synchronization with musical tempo. 2. Background Research on social aspects of musical gesture so far has been conducted mainly in the context of ethnomusicological research, using anthropological methods in combination with audio/video recording and analysis [4] [5]. Given this context, the Musical Synchrotron can be seen as a contribution to the development of a technological platform in which social embodied music interaction [6], including music perception, can be studied and explored in an ecological context. It is also related to studies that explore the relationship between music and gaming [7]. The present study thereby focuses on how social interaction affects the perception of music. This is done by asking people to move along with the music, in particular, to synchronize with musical tempo.  3. The Musical Synchrotron System The basic principle of The Musical Synchrotron is to capture movement data from wireless motion sensors together with the playback of musical stimuli. The pulse or BPM (beats per minute) value of the movement data is determined in (quasi) real time and compared to the annotated BPM value of the played musical stimuli. This comparison is used to calculate a score which can be displayed to the participants representing their performance of the synchronization to the music. The software interface is developed such that 4 persons can participate simultaneously, which enables not only the study of multiple individual performances but also the human-human interactions as a response to music.  The Musical Synchrotron consists of three main parts, which are discussed in the next sections. The first part is concerned with the data acquisition where the acceleration data from the wireless motion sensors is received. The second part is concerned with the processing of the acceleration data in terms of BPM. These values give a direct indication of the synchronization with the musical tempo. Finally, the BPM value is used to provide feedback, in the form of a user interface. The data is transmitted between the different parts of the Musical Synchrotron using the Open Sound Control (OSC) protocol [8]. This makes the design both modular and powerful since the different tasks can be distributed among different computers. When changing the sensor system one should only change the first part, all the following steps remain the same. 3.1 Data acquisition Currently two types of wireless motion sensors are in use, namely the commercially available Nintendo Wii Remote and the custom-made HOP sensors. At the time of the study there existed no standalone object in Pure Data (PD) to access the acceleration data from the Nintendo Wii. Therefore an external object for PD, called WiiSense [9], was developed. This external object enabled the readout of the 3 directional values of acceleration measured with the Wii Remote at a rate of 100 Hz. In view of more advanced studies and real-time applications, such as the use of a large number of sensors for a longer time and a more flexible attachment to different human body parts, we envisioned the development of a custom-made sensor called the HOP sensor [10].  Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NIME08, June 4-8, 2008, Genova, Italy Copyright remains with the author(s).  The HOP sensors incorporate a 3D accelerometer and a Wireless USB transceiver, which accesses the 2.4 GHz ISM band. A dedicated HUB is connected via USB to the computer and recognized as a virtual COM port. Sampling rates of 100 Hz are possible and a receiver range of 40 m is achieved. The major advantage of the HOP sensors is their size: 55 mm long, 32 mm wide and around 15 mm thick (including connectors). Each sensor is powered by a Li-Po battery, which has the same dimensions as the sensor and provides around 18 hours of operation time. This small design makes it easy to strap the sensor on the legs or arms of people using simple stretchable Velcro. 3.2 Processing In the processing of the 3D acceleration signal, the 3 directional values of acceleration are summed and filtered to a range from 0.5Hz to 4Hz. The high pass filter eliminates the constant offset in the acceleration data due to the gravitation of the earth. The low pass filter eliminates the higher frequencies that are irrelevant to human rhythm perception [11]. The calculation of the BPM value is done using an FFT with an input window of 400 samples (or 4 seconds) and an overlap of 50%, which results in an update time of 2 seconds. This external enables a real-time BPM determination of the acceleration signal.  3.3 Feedback The third part of the Musical Synchrotron consists of a user interface. This interface controls the playback of the musical stimuli and displays a direct visual feedback of the participants’ performance. The feedback is provided through a score that counts up (down) when the people are moving the sensor in the same tempo (too fast or too slow) compared to the annotated musical tempo. All data are logged to txt files containing the sensor acceleration data and the score of the participant for further offline analysis.  The software was developed such that 4 people could participate in the experiment simultaneously. As such, The Musical Synchrotron could be used in the context of social interaction where participants move along with the tempo of the music they hear. However, the Musical Synchrotron is also operational with less than 4 people. 4. Evaluation and Outlook The Musical Synchrotron has meanwhile been used in several public events, including a big fair in Ghent (ACCENTA 2007), an event on Scientific Research at Ghent University, and on the Television Broadcasting (KETNET, Betweters) where it was demonstrated in a program for children. In addition to that, the Musical Synchrotron has been used in a scientific study on synchrony behavior of children.  The results, which are described in detail in [12] [13], are promising in that they show a clear effect of the social condition (where the 4 participants see each other) in comparison with the individual condition (where the participants are blindfolded).  A major result of The Musical Synchrotron concerns the positive attitude of participants that were engaged in this interactive music-driven game. Using dedicated hardware components that are currently under development [10] we are on the way towards a platform that can highly contribute to our understanding of music-driven social interaction, using the principles of embodied music cognition [6].  5. ACKNOWLEDGMENTS Special thanks to Bart Kuyken and Wouter Verstichel and the support of the TFCG Microsystems Lab - IMEC under the guidance of Jan Vanfleteren to design and manufacture the HOP sensors. Also our gratitude goes to Prashant Vaibhav for implementing the WiiSense object in PD and to all the participants in our experiments. This work is funded with the EmcoMetecca-project. 6. REFERENCES [1] Merriam, A. The Anthropology of Music. Evanston: Northwestern University Press,1964. [2] McNeill, W. H. Keeping together in time: dance and drill in human history. Harvard University Press: Cambridge, Massachusetts, 1995. [3] Bispham, J. Rhythm in music: What is it? Who has it? And why? Music Perception, 24(2), 125-134, 2006. [4] Lortat-Jacob, B. & Olsen, M. R. Music, anthropology: A necessary marriage. Homme (171-172), 7-26, 2004. [5] Clayton, M. R. L. Observing entrainment in music performance: Video-based observational analysis of Indian musicians’ tanpura playing and beat making. Musicae Scientiae, 11(1), 27-59, 2007. [6] Leman, M. Embodied music cognition and mediation technology. Cambridge, MA: The MIT-Press, 2007. [7] Rinman, M. L., Friberg, A., Bendiksen, B., Cirotteau, D., Dahl, S., Kjellmo, I., et al. Ghost in the cave – An interactive collaborative game using non-verbal communication. Gesture-Based Communication in Human-Computer Interaction, 2915, 549-556, 2004. [8] OSC, http://opensoundcontrol.org/ [9] Vaibhav, P. http://code.google.com/p/wiisense/ [10] Kuyken, B., Verstichel, W., Demey, M. & Leman, M. The HOP sensor: wireless movement sensor, NIME 08. [11] Van Noorden, L., & Moelants, D. Resonance in the perception of musical pulse. Journal of New Music Research, 28(1), 43-66, 1999. [12] De Bruyn, L., Leman, M. & Moelants, D. Quantifying children’s embodiment of musical rhythm in individual and group settings. Accepted for publication in Proceedings of the 10th International Conference on Music Perception and Cognition. Sapporo, Japan, August 25-29 2008. [13] De Bruyn, L., Leman, M., Demey, M., Desmet, F. & Moelants, D. Measuring and quantifying the impact of social interaction in listeners’ movements to music. Accepted for publication in Springer Verlag Lecture Notes in Computer Science. 

The musical synchrotron: using wireless motion sensors to study how social interaction affects synchronization with musical tempo

https://biblio.ugent.be/publication/419834/file/6806167.pdf

The musical synchrotron: using wireless motion sensors to study how social interaction affects synchronization with musical tempo

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography