Conducting a virtual ensemble with a kinect device

This paper presents a gesture-based interaction technique for the implementation of an orchestra conductor and a virtual ensemble, using a 3D camera-based sensor to capture user’s gestures. In particular, a human-computer interface has been developed to recognize conducting gestures using a Microsoft Kinect device. The system allows the conductor to control both the tempo in the piece played as well as the dynamics of each instrument set independently. In order to modify the tempo in the playback, a time-frequency processing-based algorithmis used. Finally, an experiment was conducted to assess user’s opinion of the system as well as experimentally confirm if the features in the system were effectively improving user experience or not.This work has been funded by the Ministerio de Economia y Competitividad of the Spanish Government under Project No. TIN2010-21089-C03-02 and Project No. IPT-2011-0885-430000 and by the Junta de Andalucia under Project No. P11-TIC-7154. The work has been done at Universidad de Malaga. Campus de Excelencia Internacional Andalucia Tech

A Conceptual Framework for Motion Based Music Applications

Imaginary projections are the core of the framework for motion based music applications presented in this paper. Their design depends on the space covered by the motion tracking device, but also on the musical feature involved in the application. They can be considered a very powerful tool because they allow not only to project in the virtual environment the image of a traditional acoustic instrument, but also to express any spatially defined abstract concept. The system pipeline starts from the musical content and, through a geometrical interpretation, arrives to its projection in the physical space. Three case studies involving different motion tracking devices and different musical concepts will be analyzed. The three examined applications have been programmed and already tested by the authors. They aim respectively at musical expressive interaction (Disembodied Voices), tonal music knowledge (Harmonic Walk) and XX century music composition (Hand Composer)

Introduction to Gestural Similarity in Music. An Application of Category Theory to the Orchestra

Mathematics, and more generally computational sciences, intervene in several aspects of music. Mathematics describes the acoustics of the sounds giving formal tools to physics, and the matter of music itself in terms of compositional structures and strategies. Mathematics can also be applied to the entire making of music, from the score to the performance, connecting compositional structures to acoustical reality of sounds. Moreover, the precise concept of gesture has a decisive role in understanding musical performance. In this paper, we apply some concepts of category theory to compare gestures of orchestral musicians, and to investigate the relationship between orchestra and conductor, as well as between listeners and conductor/orchestra. To this aim, we will introduce the concept of gestural similarity. The mathematical tools used can be applied to gesture classification, and to interdisciplinary comparisons between music and visual arts.Comment: The final version of this paper has been published by the Journal of Mathematics and Musi

BitBox!:A case study interface for teaching real-time adaptive music composition for video games

Real-time adaptive music is now well-established as a popular medium, largely through its use in video game soundtracks. Commercial packages, such as fmod, make freely available the underlying technical methods for use in educational contexts, making adaptive music technologies accessible to students. Writing adaptive music, however, presents a significant learning challenge, not least because it requires a different mode of thought, and tutor and learner may have few mutual points of connection in discovering and understanding the musical drivers, relationships and structures in these works. This article discusses the creation of ‘BitBox!’, a gestural music interface designed to deconstruct and explain the component elements of adaptive composition through interactive play. The interface was displayed at the Dare Protoplay games exposition in Dundee in August 2014. The initial proof-of- concept study proved successful, suggesting possible refinements in design and a broader range of applications

Webbing and orchestration. Two interrelated views on digital tools in mathematics education

The integration of digital tools in mathematics education is considered both promising and problematic. To deal with this issue, notions of webbing and instrumental orchestration are developed. However, the two seemed to be disconnected, and having different cultural and theoretical roots. In this article, we investigate the distinct and joint journeys of these two theoretical perspectives. Taking some key moments in recent history as points of de- parture, we conclude that the two perspectives share an importance attributed to digital tools, and that initial differences, such as different views on the role of digital tools and the role of the teacher, have become more nuances. The two approaches share future chal- lenges to the organization of teachers'collaborative work and their use of digital resources.Comment: Teaching Mathematics and its Applications (2014) to be complete

The \u201cHarmonic Walk\u201d and Enactive Knowledge: an Assessment Report

The Harmonic Walk is an interactive, physical environment based on user\u2019s motion detection and devoted to the study and practice of tonal harmony. When entering the rectangular floor surface within the application\u2019s camera view, a user can actually walk inside the musical structure, causing a sound feedback depending on the occupied zone. We arranged a two masks projection set up to allow users to experience melodic segmentation and tonality harmonic space, and we planned two phase assessment sessions, submitting a 22 high school student group to various test conditions. Our findings demonstrate the high learning effectiveness of the Harmonic Walk application. Its ability to transfer abstract concepts in an enactive way, produces important improvement rates both for subjects who received explicit information and for subjects who didn\u2019t

Boston University Messiaen Project, October 12 and 13, 2007

This is the concert program of the Boston University Messiaen Project international conference on Friday, October 12, and Saturday, October 13, 2007, at the College of Fine Arts, 855 Commonwealth Avenue. The conference featured lectures by Yves Balmer, Karin Heller, Stephen Butler Murray, Martin Lee, Andrew Shenton, Mark DeVoto, Wai Ling Cheong, Luke Berryman, Thomas Peattie, Peter Bannister, Vincent Benitez, Robert Sholl, Alexandre Abdoulvaev, Robert Fallon, Adam Gustafson, Douglas Shadle, Stephen Schloesser, Alexander Rehding, Sander van Maas, Ryan W. Dohoney, and David Cannata. Works performed on the concert on Saturday, October 13, 2007 in the Concert Hall were "Fantasie pour violon et piano by Olivier Messiaen, "Un reflet dans le vent" by O. Messiaen, "Ondine" by Claude Debussy, "Les fées sont d'exquises danseuses" by C. Debussy, "Brouillards" by C. Debussy, "Feuilles mortes" by C. Debussy, "La colombe" by O. Messiaen, "Deux romances de Paul Bourget" by C. Debussy, "Deux morceaux de soir" by C. Debussy, and "Poèmes pour Mi" by O. Messiaen. Digitization for Boston University Concert Programs was supported by the Boston University Humanities Library Endowed Fund