BitBox!:A case study interface for teaching real-time adaptive music composition for video games

Real-time adaptive music is now well-established as a popular medium, largely through its use in video game soundtracks. Commercial packages, such as fmod, make freely available the underlying technical methods for use in educational contexts, making adaptive music technologies accessible to students. Writing adaptive music, however, presents a significant learning challenge, not least because it requires a different mode of thought, and tutor and learner may have few mutual points of connection in discovering and understanding the musical drivers, relationships and structures in these works. This article discusses the creation of ‘BitBox!’, a gestural music interface designed to deconstruct and explain the component elements of adaptive composition through interactive play. The interface was displayed at the Dare Protoplay games exposition in Dundee in August 2014. The initial proof-of- concept study proved successful, suggesting possible refinements in design and a broader range of applications

Comparison of input devices in an ISEE direct timbre manipulation task

The representation and manipulation of sound within multimedia systems is an important and currently under-researched area. The paper gives an overview of the authors' work on the direct manipulation of audio information, and describes a solution based upon the navigation of four-dimensional scaled timbre spaces. Three hardware input devices were experimentally evaluated for use in a timbre space navigation task: the Apple Standard Mouse, Gravis Advanced Mousestick II joystick (absolute and relative) and the Nintendo Power Glove. Results show that the usability of these devices significantly affected the efficacy of the system, and that conventional low-cost, low-dimensional devices provided better performance than the low-cost, multidimensional dataglove

From Autonomous to Performative Control of Timbral Spatialisation

Timbral spatialisation is one such process that requires the independent control of potentially thousands of parameters (Torchia, et al., 2003). Current research on controlling timbral spatialisation has focussed either on automated generative systems, or suggested that to design trajectories in software is to write every movement line by line (Normandeau, 2009). This research proposes that Wave Terrain Synthesis may be used as an effective bridging control structure for timbral spatialisation, enabling the performative control of large numbers of parameter sets associated with software. This methodology also allows for compact interactive mapping possibilities for a physical controller, and may also be effectively mapped gesturall

Down-Sampling coupled to Elastic Kernel Machines for Efficient Recognition of Isolated Gestures

In the field of gestural action recognition, many studies have focused on dimensionality reduction along the spatial axis, to reduce both the variability of gestural sequences expressed in the reduced space, and the computational complexity of their processing. It is noticeable that very few of these methods have explicitly addressed the dimensionality reduction along the time axis. This is however a major issue with regard to the use of elastic distances characterized by a quadratic complexity. To partially fill this apparent gap, we present in this paper an approach based on temporal down-sampling associated to elastic kernel machine learning. We experimentally show, on two data sets that are widely referenced in the domain of human gesture recognition, and very different in terms of quality of motion capture, that it is possible to significantly reduce the number of skeleton frames while maintaining a good recognition rate. The method proves to give satisfactory results at a level currently reached by state-of-the-art methods on these data sets. The computational complexity reduction makes this approach eligible for real-time applications.Comment: ICPR 2014, International Conference on Pattern Recognition, Stockholm : Sweden (2014

Gestural Control Of Wavefield synthesis

(Abstract to follow

Improvising with the threnoscope: integrating code, hardware, GUI, network, and graphic scores

Live coding emphasises improvisation. It is an art practice that merges the act of musical composition and performance into a public act of projected writing. This paper introduces the Threnoscope system, which includes a live coding micro-language for drone-based microtonal composition. The paper discusses the aims and objectives of the system, elucidates the design decisions, and introduces in particular the code score feature present in the Threnoscope. The code score is a novel element in the design of live coding systems allowing for improvisation through a graphic score, rendering a visual representation of past and future events in a real-time performance. The paper demonstrates how the system’s methods can be mapped ad hoc to GUI- or hardware-based control

Wearable and mobile devices

Information and Communication Technologies, known as ICT, have undergone dramatic changes in the last 25 years. The 1980s was the decade of the Personal Computer (PC), which brought computing into the home and, in an educational setting, into the classroom. The 1990s gave us the World Wide Web (the Web), building on the infrastructure of the Internet, which has revolutionized the availability and delivery of information. In the midst of this information revolution, we are now confronted with a third wave of novel technologies (i.e., mobile and wearable computing), where computing devices already are becoming small enough so that we can carry them around at all times, and, in addition, they have the ability to interact with devices embedded in the environment. The development of wearable technology is perhaps a logical product of the convergence between the miniaturization of microchips (nanotechnology) and an increasing interest in pervasive computing, where mobility is the main objective. The miniaturization of computers is largely due to the decreasing size of semiconductors and switches; molecular manufacturing will allow for “not only molecular-scale switches but also nanoscale motors, pumps, pipes, machinery that could mimic skin” (Page, 2003, p. 2). This shift in the size of computers has obvious implications for the human-computer interaction introducing the next generation of interfaces. Neil Gershenfeld, the director of the Media Lab’s Physics and Media Group, argues, “The world is becoming the interface. Computers as distinguishable devices will disappear as the objects themselves become the means we use to interact with both the physical and the virtual worlds” (Page, 2003, p. 3). Ultimately, this will lead to a move away from desktop user interfaces and toward mobile interfaces and pervasive computing

Sound shapes and spatial texture: Frequency-space morphology

The use of Wave Terrain Synthesis as a control mechanism is a governing system that allows the performer to create a complex and coordinated change across an existing complex parametric system. This research has focused largely on the application of Wave Terrain Synthesis for the control of Timbral Spatialisation. Various mappings of the Wave Terrain mechanism are discussed, to highlight some various ways in which frequency-space morphology may be approached with such a model. With the means of smoothly interpolating between various terrain and trajectory states allow the performer to control the evolving nature of sound shapes and spatial texture generated by the model