1,502 research outputs found
Conducting a virtual ensemble with a kinect device
This paper presents a gesture-based interaction technique for the implementation of an orchestra conductor and a virtual ensemble, using a 3D camera-based sensor to capture user’s gestures. In particular, a human-computer interface has been developed to recognize conducting gestures using a Microsoft Kinect device. The system allows the conductor to control both the tempo in the piece played as well as the dynamics of each instrument set independently. In order to modify the tempo in the playback, a time-frequency
processing-based algorithmis used. Finally, an experiment was conducted to assess user’s opinion of the system as well as experimentally confirm if the features in the system were effectively improving user experience or not.This work has been funded by the Ministerio de Economia y Competitividad of the Spanish Government under Project No. TIN2010-21089-C03-02 and Project No. IPT-2011-0885-430000 and by the Junta de Andalucia under Project No. P11-TIC-7154. The work has been done at Universidad de Malaga. Campus de Excelencia Internacional Andalucia Tech
A Conceptual Framework for Motion Based Music Applications
Imaginary projections are the core of the framework for motion
based music applications presented in this paper. Their design depends
on the space covered by the motion tracking device, but also
on the musical feature involved in the application. They can be considered
a very powerful tool because they allow not only to project
in the virtual environment the image of a traditional acoustic instrument,
but also to express any spatially defined abstract concept.
The system pipeline starts from the musical content and, through a
geometrical interpretation, arrives to its projection in the physical
space. Three case studies involving different motion tracking devices
and different musical concepts will be analyzed. The three
examined applications have been programmed and already tested
by the authors. They aim respectively at musical expressive interaction
(Disembodied Voices), tonal music knowledge (Harmonic
Walk) and XX century music composition (Hand Composer)
MotionDesigner: Augmented artistic performances with kinect-based human body motion tracking
In the last two decades the use of technology in art projects has proliferated, as is the case of the interactive projections based on movement used in artistic performances and installations. However, the artists responsible for creating this work typically have to rely on computer experts to implement this type of interactive systems. The tool herein presented, MotionDesigner, intends to assist the design of these systems by providing artists with higher levels of autonomy and efficiency during the creative process, allowing them to specify the rules by which a human body interacts with both the audio and the visuals used in their interactive art work. The presented tool relies on an RGB-D camera to modulate the multimedia content according to the performer’s body motion. MotionDesigner is extensible so as to accommodate additions required by artists. The tool has been tested with dancers, choreographers, and architects. Results show that MotionDesigner is a valuable aid to artists, working as a catalyst of their creative process.info:eu-repo/semantics/acceptedVersio
Gesture, sound and place.
This paper discusses two multichannel interactive audiovisual artworks, Action A/V and SoundLabyrinth, that explore approaches to the experience of gesture, sound and place. Both works were situated in a geodesic dome frame and built within the Max, Ableton and Max for Live computer programs, and produced ostensibly similar outcomes, however the approaches taken by the two authors differ in intention, processes, and philosophy. These approaches were presented and discussed in workshops delivered at ISEA2013, on Sunday June 9 and Monday June 10 2013. In these workshops participants mprovised with the two systems, both through moving in the dome and by operating the related software, and discussed approaches and understandings of the three terms listed in the title of this paper
Virtual Conductor for String Quartet Practice
This paper presents a system that emulates an ensemble conductor for string quartets. This application has been developed as a support tool for individual and group practice, so that users of any age range can use it to further hone their skills, both for regular musicians and students
alike. The virtual conductor designed can offer similar indications to those given by a real ensemble conductor to potential users regarding beat times, dynamics, etc. The application developed allows the user to rehearse his/her
performance without the need of having an actual conductor present, and also gives access to additional tools to further support the learning/practice process, such as a tuner
or a melody evaluator. The system developed also allows for both solo practice and group practice. A set of tests were conducted to check the usefulness of the application as a practice support tool. A group of musicians from the
Chamber Orchestra of Malaga including an ensemble conductor tested the system, and reported to have found it a very useful tool within an educational environment and that it helps to address the lack of this kind of educational tools in a self-learning environment.This work has been funded by the Ministerio de Economia y Competitividad of the Spanish Government under Project No. TIN2010-21089-C03- 02 and Project No. IPT-2011-0885-430000 and by the Ministerio de Industria, Turismo y Comercio under Project No. TSI-090100-2011-25
Towards the Design of a Natural User Interface for Performing and Learning Musical Gestures
AbstractA large variety of musical instruments, either acoustical or digital, are based on a keyboard scheme. Keyboard instruments can produce sounds through acoustic means but they are increasingly used to control digital sound synthesis processes with nowadays music. Interestingly, with all the different possibilities of sonic outcomes, the input remains a musical gesture. In this paper we present the conceptualization of a Natural User Interface (NUI), named the Intangible Musical Instrument (IMI), aiming to support both learning of expert musical gestures and performing music as a unified user experience. The IMI is designed to recognize metaphors of pianistic gestures, focusing on subtle uses of fingers and upper-body. Based on a typology of musical gestures, a gesture vocabulary has been created, hierarchized from basic to complex. These piano-like gestures are finally recognized and transformed into sounds
RGB-D datasets using microsoft kinect or similar sensors: a survey
RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the state-of-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms
Motion Based Learning for Preschools Mathematics via Kinect (Counting Number)
Kids start learning numbers since they in preschools. Every kids have difficulties to
start learn about mathematics. They do not recognize and might be difficult for them
to remember the numbers at first. Teacher will help them to teach and assist them to
solve this problem and it usually happens after some stages and time. The only
problem is education system use currently is not helping much in increasing the
speed of children's learning. With current education system that sometime bored and
only use whiteboard as medium of teaching, kids cannot give their full commitment
in class. In Malaysia, traditional and conventional teaching still widely being
practiced. However, with the advancement of technology nowadays, it can help to
overcome this problem. By using current technology, it can help to complement
current education system. Education system can utilize the advancement of
technology to overcome problem such as boring and not interactive learning.
Example of technology that can be use is Kinect. Kinect is a device that can track
human body to interact with an application. It can help make learning become active,
interactive and at it also involving all body parts. The objective of introducing kinect
in education is to develop a motion based learning application for preschools'
mathematics. To achieve this objective, the author has decided to test the application
for preschools kid age from 3 to 6 years old. It will focus on teaching simple
numbers. Kinect in mathematic will cover on basic numbers among preschool kids.
Since the time given to develop this project is very limited, the methodology chosen
for this development is throwaway prototype methodology. By using throwaway
prototyping methodology this project have improved and increase user involvement.
Results showing that kids enjoy the kinect technology in mathematics and teachers
have found one of interactive ways of teaching. From data collected and analyzed,
results showed that kids enjoy this kinect technology in mathematics and at same
time teachers have found one of interactive ways of teaching
- …