Advanced Media Control Through Drawing: Using a graphics tablet to control complex audio and video data in a live context

This paper demonstrates the results of the authors’ Wacom tablet MIDI user interface. This application enables users’ drawing actions on a graphics tablet to control audio and video parameters in real-time. The programming affords five degrees (x, y, pressure, x tilt, y tilt) of concurrent control for use in any audio or video software capable of receiving and processing MIDI data. Drawing gesture can therefore form the basis of dynamic control simultaneously in the auditory and visual realms. This creates a play of connections between parameters in both mediums, and illustrates a direct correspondence between drawing action and media transformation that is immediately apparent to viewers. The paper considers the connection between drawing technique and media control both generally and specifically, postulating that dynamic drawing in a live context creates a performance mode not dissimilar to performing on a musical instrument or conducting with a baton. The use of a dynamic and physical real-time media interface re-inserts body actions into live media performance in a compelling manner. Performers can learn to “draw/play” the graphics tablet as a musical and visual “instrument”, creating a new and uniquely idiomatic form of electronic drawing. The paper also discusses how to practically program the application and presents examples of its use as a media manipulation tool

Advanced Media Control Through Drawing: Using a graphics tablet to control complex audio and video data in a live context

This paper demonstrates the results of the authors’ Wacom tablet MIDI user interface. This application enables users’ drawing actions on a graphics tablet to control audio and video parameters in real-time. The programming affords five degrees (x, y, pressure, x tilt, y tilt) of concurrent control for use in any audio or video software capable of receiving and processing MIDI data

Dalcroze meets technology : integrating music, movement and visuals with the Music Paint Machine

peer reviewedNew interactive music educational technologies are often seen as a ‘force of change’, introducing new approaches that address the shortcomings (e.g. score-based, teacher-centred and disembodied) of the so-called traditional teaching approaches. And yet, despite the growing belief in their educational potential, these new technologies have been problematised with regard to their design, reception, implementation and evaluation. A possible way to optimise the realisation of the educational potential of interactive music educational technologies is to connect their use to music educational approaches that stood the test of time and as such may inspire technologies to become a bridge between tradition and innovation. This article describes an educational technology (the Music Paint Machine) that integrates the creative use of movement and visualisation to support instrumental music teaching and learning. Next, it connects this application to such an established music educational method, the Dalcroze approach. Through the lens of a set of interconnected aspects, it is shown how the Music Paint Machine’s conceptual design aligns to the underlying principles of this approach. In this way, it is argued that integrating Dalcroze-inspired practices is a plausible way of realising the didactic potential of the system. An appendix with example exercises is provided

Towards the design of a kid’s music organizer

In this paper, we investigate how young children aged 2 to 5 interact with music and their family's music collections. By going into their homes, interviewing them and their parents and observing the children performing a range of music-related tasks, we explore the way that pre-school children select, interact with, and organize music. Additionally, drawing tasks were included in the visits to engage the children and allow them to demonstrate their thoughts in a concrete manner. Insights into the children's' music behaviours suggest design features for a music organizer / player for very young children

The ixiQuarks: merging code and GUI in one creative space

This paper reports on ixiQuarks; an environment of instruments and effects that is built on top of the audio programming language SuperCollider. The rationale of these instruments is to explore alternative ways of designing musical interaction in screen-based software, and investigate how semiotics in interface design affects the musical output. The ixiQuarks are part of external libraries available to SuperCollider through the Quarks system. They are software instruments based on a non- realist design ideology that rejects the simulation of acoustic instruments or music hardware and focuses on experimentation at the level of musical interaction. In this environment we try to merge the graphical with the textual in the same instruments, allowing the user to reprogram and change parts of them in runtime. After a short introduction to SuperCollider and the Quark system, we will describe the ixiQuarks and the philosophical basis of their design. We conclude by looking at how they can be seen as epistemic tools that influence the musician in a complex hermeneutic circle of interpretation and signification

HIEMPA: Hybrid Instruments from Electroacoustic Manipulation and Models of Pütorino and Aquascape

The HIEMPA project combined a team of people with technical, artistic, environmental and cultural expertise towards an artistic outcome aiming to extend the New Zealand sonic art tradition. The work involved collecting audio samples from the aquascape of the Ruakuri Caves and Nature Reserve in Waitomo, South Waikato, New Zealand; and samples of a variety of pütorino – a New Zealand Mäori wind instrument. Following a machine learning analysis of this audio material and an analysis of the performance material, hybrid digital instruments were built and mapped to suitable hardware triggers. The new instruments are playable in realtime, along with the electroacoustic manipulation of pütorino performances. The project takes into account the environmental and cultural significance of the source material, with the results to be released as a set of compositions. This paper discusses the background research and process of the project

Improvising with the threnoscope: integrating code, hardware, GUI, network, and graphic scores

Live coding emphasises improvisation. It is an art practice that merges the act of musical composition and performance into a public act of projected writing. This paper introduces the Threnoscope system, which includes a live coding micro-language for drone-based microtonal composition. The paper discusses the aims and objectives of the system, elucidates the design decisions, and introduces in particular the code score feature present in the Threnoscope. The code score is a novel element in the design of live coding systems allowing for improvisation through a graphic score, rendering a visual representation of past and future events in a real-time performance. The paper demonstrates how the system’s methods can be mapped ad hoc to GUI- or hardware-based control

Embodiment, sound and visualization : a multimodal perspective in music education

Recently, many studies have emphasized the role of body movements in processing, sharing and giving meaning to music. At the same time, neuroscience studies, suggest that different parts of the brain are integrated and activated by the same stimuli: sounds, for example, can be perceived by touch and can evoke imagery, energy, fluency and periodicity. This interaction of auditory, visual and motor senses can be found in the verbal descriptions of music and among children during their spontaneous games. The question to be asked is, if a more multisensory and embodied approach could redefine some of our assumptions regarding musical education. Recent research on embodiment and multimodal perception in instrumental teaching could suggest new directions in musical education. Can we consider the integration between the activities of body movement, listening, metaphor visualization, and singing, as more effective than a disembodied and fragmented approach for the process of musical understanding

Sounds of Waitakere: Using practitioner research to explore how Year 6 recorder players compose responses to visual representations of a natural environment

How might primary students utilise the stimulus of a painting in a collaborative composition drawing on a non-conventional sound palette of their own making? This practitioner research features 17 recorder players from a Year 6 class (10–11-year-olds) who attend a West Auckland primary school in New Zealand. These children were invited to experiment with the instrument to produce collectively an expanded ‘repertoire’ or ‘palette’ of sounds. In small groups, they then discussed a painting by an established New Zealand painter set in the Waitakere Ranges and attempted to formulate an interpretation in musical terms. On the basis of their interpretation, drawing on sounds from the collective palette (complemented with other sounds), they worked collaboratively to develop, refine and perform a structured composition named for their chosen painting. This case study is primarily descriptive (providing narrative accounts and rich vignettes of practice) and, secondarily, exploratory (description and analysis leading to the development of hypotheses). It has implications for a range of current educational issues, including curriculum integration and the place of composition and notation in the primary-school music programme

Cultural Engagement in California's Inland Regions

Cultural Engagement in California's Inland Regions explores patterns of cultural engagement in the San Joaquin Valley and the Inland Empire. Two major data collection efforts were undertaken. The first was a door-to-door intercept survey of more than 1,000 randomly selected households in six distinctly different neighborhoods, three in the Fresno area and three in Riverside and San Bernardino. The second was a self-administered survey of more than 5,000 residents of the two regions, promoted as the "California Cultural Census" and conducted online and through intercept work at various locations and events. It is important to note that this second data set aggregates multiple samples, including respondents who were selected at the convenience of outreach organizations. Although weighted to reduce potential biases, these data are not representative of all adults in the two regions. Results paint a detailed picture of the breadth and depth of cultural engagement in the two regions and reveal a range of activity in music, theater and drama, reading and writing, dance, and visual arts and crafts -- much of which occurs off the radar" of the traditional infrastructure of nonprofit arts organizations and facilities. The study identifies specific types of activities which, if supported at higher levels, might equitably raise participation levels and achieve higher levels of cultural vitality in millions of homes and hundreds of communities. It concludes that cultural providers and funders should look deeper into the fabric of their communities for new partners, new settings and innovative approaches to drawing residents into cultural experiences. This briefing provides a high level summary of the study's key findings, as well as discussion questions for cultural providers and funders. Comprehensive results are available at www.irvine.org, including an executive summary and detailed results by artistic discipline