Evaluating the Wiimote as a musical controller

The Nintendo Wiimote is growing in popularity with mu-sicians as a controller. This mode of use is an adaptationfrom its intended use as a game controller, and requiresevaluation of its functions in a musical context in orderto understand its possibilities and limits. Drawing on Hu-man Computer Interaction methodology, we assessed thecore musical applications of the Wiimote and designeda usability experiment to test them. 17 participants tookpart, performing musical tasks in four contexts: trigger-ing; precise and expressive continuous control; and ges-ture recognition. Interviews and empirical evidence wereutilised to probe the device’s limitations and its creativestrengths. This study should help potential users to planthe Wiimote’s employment in their projects, and should beuseful as a case study in HCI evaluation of musical con-trollers

Multiparametric interfaces for fine-grained control of digital music

Digital technology provides a very powerful medium for musical creativity, and the way in which we interface and interact with computers has a huge bearing on our ability to realise our artistic aims. The standard input devices available for the control of digital music tools tend to afford a low quality of embodied control; they fail to realise our innate expressiveness and dexterity of motion. This thesis looks at ways of capturing more detailed and subtle motion for the control of computer music tools; it examines how this motion can be used to control music software, and evaluates musicians’ experience of using these systems. Two new musical controllers were created, based on a multiparametric paradigm where multiple, continuous, concurrent motion data streams are mapped to the control of musical parameters. The first controller, Phalanger, is a markerless video tracking system that enables the use of hand and finger motion for musical control. EchoFoam, the second system, is a malleable controller, operated through the manipulation of conductive foam. Both systems use machine learning techniques at the core of their functionality. These controllers are front ends to RECZ, a high-level mapping tool for multiparametric data streams. The development of these systems and the evaluation of musicians’ experience of their use constructs a detailed picture of multiparametric musical control. This work contributes to the developing intersection between the fields of computer music and human-computer interaction. The principal contributions are the two new musical controllers, and a set of guidelines for the design and use of multiparametric interfaces for the control of digital music. This work also acts as a case study of the application of HCI user experience evaluation methodology to musical interfaces. The results highlight important themes concerning multiparametric musical control. These include the use of metaphor and imagery, choreography and language creation, individual differences and uncontrol. They highlight how this style of interface can fit into the creative process, and advocate a pluralistic approach to the control of digital music tools where different input devices fit different creative scenarios

Gestural Control Of Wavefield synthesis

(Abstract to follow

Transforming musical performance: activating the audience as digital collaborators

Digital technologies have transformed the performance practice, recording and distribution technologies, economy and sonic landscape of music in a process of change that began in the early 1980s. Recent technological developments have opened up the possibility of embodied interaction between audiences and performers, reframing music performance as a collaborative improvisatory space that affords Interactive Musical Participation. The research in this practice-based thesis looks at the relationship and experience of audience members and musicians exploring Interactive Musical Participation within the wide stylistic framework of contemporary jazz. It also studies the potential for the creation of compositional, technological and performance protocols to enable successful Interactive Musical Participation. This has been achieved through a process of mapping the methodology behind the composition, technical infrastructure, performances and post-performance analysis of a series of musical artefacts. Cook (2001 and 2009) suggests that researchers in this field should “Make a piece, not an instrument or controller” and this dictum has influenced the development of the technical infrastructure for this research. Easily accessible and low-cost digital audio workstations Ableton Live (2017) and Logic Pro X (Apple, 2019) as well as the digital protocols Open Sound Control (OSC) (Opensoundcontrol.org) have been utilised to deliver the programming and networking requirements. A major innovation stemming from this project has been the development of the Deeper Love Soundpad App, a sample playback app for Apple smartphones and iPads, in collaboration with Dr. Rob Toulson. The theoretical background to this research has been informed by actornetwork theory, the sociological approach developed by Bruno Latour (2005), Michel Callon (1986) and John Law (1992). Actor-network theory (ANT) provides a framework for understanding the mechanics of power and organisation within heterogeneous non-hierarchical networks. Mapping and analysing the ANT networks and connections created by the research performances has provided valuable data in the Interactive Musical Participatio

Sound Spheres: A Design Study of the Articulacy of a Non-Contact Finger Tracking Virtual Musical Instrument

A key challenge in the design of Virtual Musical instruments (VMIs) is finding expressive, playable, learnable mappings from gesture to sound that progressively reward practice by performers. Designing such mappings can be particularly demanding in the case of non-contact musical instruments, where physical cues can be scarce. Unaided intuition works well for many instrument designers, but others may find design and evaluation heuristics useful when creating new VMIs. In this paper we gather existing criteria from the literature to assemble a simple set of design and evaluation heuristics that we dub articulacy. This paper presents a design case study in which an expressive non-contact finger-tracking VMI, Sound Spheres, is designed and evaluated with the support of the articulacy heuristics. The case study explores the extent to which articulacy usefully informs the design of a non-contact VMI, and we reflect on the usefulness or otherwise of heuristic approaches in this context

A Quantitative Evaluation of the Differences between Knobs and Sliders

This paper presents a HCI inspired evaluation of simple phys-ical interfaces used to control physical models. Specifi-cally knobs and sliders are compared in a creative and ex-ploratory framework, which simulates the natural environ-ment in which an electronic musician would normally ex-plore a new instrument. No significant difference was mea-sured between using knobs and sliders for controlling pa-rameters of a physical modeling electronic instrument. The reported difference between the tested instruments were mostly due to the sound synthesis models

A quest for the holy grail: Tactile precision, natural movement and haptic feedback in 3D virtual spaces

Three-dimensional immersive spaces such as those provided by virtual worlds, give unparalleled opportunities for learners to practically engage with simulated authentic settings that may be too expensive or too dangerous to experience in the real world. The potential afforded by these environments is severely constrained by the use of a keyboard and mouse moving in two dimensions. While most technologies have evolved rapidly in the early 21st century, the mouse and keyboard as standard navigation and interaction tools have not. However, talented teams from a range of disciplines are on serious quests to address this limitation. Their Holy Grail is to develop ways to interact with 3D immersive spaces using more natural human movements with haptic feedback. Applications would include the training of surgeons and musical conductors, training elite sports people and even physical rehabilitation. This paper reports on the cutting-edge technology projects that look most likely to provide a solution for this complex problem, including the Wiimote and the Microsoft's Project Natal. © 2009 Helen Farley and Caroline Steel

An evaluation of digital interfaces for music composition and improvisation

This PhD reports research on current representative performance paradigms using various interfaces for real time interaction with computer-based musical environments. Each device was selected to cover a particular range of interfaces. Research covers the following areas: hardware interfaces (tangible & game devices); live coding; optical devices, and hardware prototyping.The projects highlight affordances, comparative strengths and weaknesses, and provide suggestions for further improvements for each paradigm. Particular focus is given to the importance of mapping. Each project comprises corresponding software that was developed to facilitate each performance paradigm.The work is not intended to provide an exhaustive evaluation of the technology used in this research; instead, it aims to examine its feasibility for artistic and musical context. The outcomes of the examinations include a series of musical performances employing improvisation as the basis for composition. These paradigms are examined in a live context as well as fixed media that uses material originating in live performances

A Sensor-Based Interactive Digital Installation System for Virtual Painting Using MAX/MSP/Jitter

Interactive art is rapidly becoming a part of cosmopolitan society through public displays, video games, and art exhibits. It is a means of exploring the connections between our physical bodies and the virtual world. However, a sense of disconnection often exists between the users and technology because users are driving actions within an environment from which they are physically separated. This research involves the creation of a custom interactive, immersive, and real-time video-based mark-making installation as public art. Using a variety of input devices including video cameras, sensors, and special lighting, a painterly mark-making experience is contemporized, enabling the participant to immerse himself in a world he helps create. This work illustrates the potential of making the user-technology disconnection more seamless between the physical and virtual worlds. Using unobtrusive interfaces, the user?s physical interactions can be encouraged. The development of this installation progressed through improvements based on user feedback from iterative public displays of the work. This process is to serve as a guideline for other artists working in interactive media who are also exploring perceived intimacy in user interactions

Bridging The Gap Between Fun And Fitness: Instructional Techniques And Real-world Applications For Full-body Dance Games

Full-body controlled games offer the opportunity for not only entertainment, but education and exercise as well. Refined gameplay mechanics and content can boost intrinsic motivation and keep people playing over a long period of time, which is desirable for individuals who struggle with maintaining a regular exercise program. Within this gameplay genre, dance rhythm games have proven to be popular with game console owners. Yet, while other types of games utilize story mechanics that keep players engaged for dozens of hours, motion-controlled dance games are just beginning to incorporate these elements. In addition, this control scheme is still young, only becoming commercially available in the last few years. Instructional displays and clear real-time feedback remain difficult challenges. This thesis investigates the potential for full-body dance games to be used as tools for entertainment, education, and fitness. We built several game prototypes to investigate visual, aural, and tactile methods for instruction and feedback. We also evaluated the fitness potential of the game Dance Central 2 both by itself and with extra game content which unlocked based on performance. Significant contributions include a framework for running a longitudinal video game study, results indicating high engagement with some fitness potential, and informed discussion of how dance games could make exertion a more enjoyable experience