Evaluating musical software using conceptual metaphors

An open challenge for interaction designers is to find ways of designing software to enhance the ability of novices to perform tasks that normally require specialized domain expertise. This challenge is particularly demanding in areas such as music analysis, where complex, abstract, domain-specific concepts and notations occur. One promising theoretical foundation for this work involves the identification of conceptual metaphors and image schemas, found by analyzing discourse. This kind of analysis has already been applied, with some success, both to musical concepts and, separately, to user interface design. The present work appears to be the first to combine these hitherto distinct bodies of research, with the aim of devising a general method for improving user interfaces for music. Some areas where this may require extensions to existing method are noted. This paper presents the results of an exploratory evaluation of Harmony Space, a tool for playing, analysing and learning about harmony. The evaluation uses conceptual metaphors and image schemas elicited from the dialogues of experienced musicians discussing the harmonic progressions in a piece of music. Examples of where the user interface supports the conceptual metaphors, and where support could be improved, are discussed. The potential use of audio output to support conceptual metaphors and image schemas is considered

A toolkit of resource-sensitive, multimodal widgets

This thesis describes an architecture for a toolkit of user interface components which allows the presentation of the widgets to use multiple output modalities - typically, audio and visual. Previously there was no toolkit of widgets which would use the most appropriate presentational resources according to their availability and suitability. Typically the use of different forms of presentation was limited to graphical feedback with the addition of other forms of presentation, such as sound, being added in an ad hoc fashion with only limited scope for managing the use of the different resources. A review of existing auditory interfaces provided some requirements that the toolkit would need to fulfil for it to be effective. In addition, it was found that a strand of research in this area required further investigation to ensure that a full set of requirements was captured. It was found that no formal evaluation of audio being used to provide background information has been undertaken. A sonically-enhanced progress indicator was designed and evaluated showing that audio feedback could be used as a replacement for visual feedback rather than simply as an enhancement. The experiment also completed the requirements capture for the design of the toolkit of multimodal widgets. A review of existing user interface architectures and systems, with particular attention paid to the way they manage multiple output modalities presented some design guidelines for the architecture of the toolkit. Building on these guidelines a design for the toolkit which fulfils all the previously captured requirements is presented. An implementation of this design is given, with an evaluation of the implementation showing that it fulfils all the requirements of the desig

Caring, sharing widgets: a toolkit of sensitive widgets

Although most of us communicate using multiple sensory modalities in our lives, and many of our computers are similarly capable of multi-modal interaction, most human-computer interaction is predominantly in the visual mode. This paper describes a toolkit of widgets that are capable of presenting themselves in multiple modalities, but further are capapble of adapting their presentation to suit the contexts and environments in which they are used. This is of increasing importance as the use of mobile devices becomes ubiquitous

A toolkit of mechanism and context independent widgets

Most human-computer interfaces are designed to run on a static platform (e.g. a workstation with a monitor) in a static environment (e.g. an office). However, with mobile devices becoming ubiquitous and capable of running applications similar to those found on static devices, it is no longer valid to design static interfaces. This paper describes a user-interface architecture which allows interactors to be flexible about the way they are presented. This flexibility is defined by the different input and output mechanisms used. An interactor may use different mechanisms depending upon their suitability in the current context, user preference and the resources available for presentation using that mechanism

Latency Performance for Real-Time Audio on BeagleBone Black

In this paper we present a set of tests aimed at evaluating the responsiveness of a BeagleBone Black board in real-time interactive audio applications. The default Angstrom Linux distribution was tested without modifying the underlying kernel. Latency measurements and audio quality were compared across the combination of different audio interfaces and audio synthesis models. Data analysis shows that the board is generally characterised by a remarkably high responsiveness; most of the tested configurations are affected by less than 7ms of latency and under-run activity proved to be contained using the correct optimisation techniques

Lost Oscillations: Exploring a City’s Space and Time With an Interactive Auditory Art Installation

Presented at the 22nd International Conference on Auditory Display (ICAD-2016)Lost Oscillations is a spatio-temporal sound art installation that allows users to explore the past and present of a city's soundscape. Participants are positioned in the center of an octophonic speaker array; situated in the middle of the array is a touch-sensitive user interface. The user interface is a stylized representation of a map of Christchurch, New Zealand, with electrodes placed throughout the map. Upon touching an electrode, one of many sound recordings made at the electrode's real-world location is chosen and played; users must stay in contact with the electrodes in order for the sounds to continue playing, requiring commitment from users in order to explore the soundscape. The sound recordings have been chosen to represent Christchurch's development throughout its history, allowing participants to explore the evolution of the city from the early 20th Century through to its post-earthquake reconstruction. This paper discusses the motivations for Lost Oscillations before presenting the installation's design, development, and presentation

Sonically-enhanced widgets: comments on Brewster and Clarke, ICAD 1997

This paper presents a review of the research surrounding the paper “The Design and Evaluation of a Sonically Enhanced Tool Palette” by Brewster and Clarke from ICAD 1997. A historical perspective is given followed by a discussion of how this work has fed into current developments in the area

Sensing and mapping for interactive performance

This paper describes a trans-domain mapping (TDM) framework for translating meaningful activities from one creative domain onto another. The multi-disciplinary framework is designed to facilitate an intuitive and non-intrusive interactive multimedia performance interface that offers the users or performers real-time control of multimedia events using their physical movements. It is intended to be a highly dynamic real-time performance tool, sensing and tracking activities and changes, in order to provide interactive multimedia performances. From a straightforward definition of the TDM framework, this paper reports several implementations and multi-disciplinary collaborative projects using the proposed framework, including a motion and colour-sensitive system, a sensor-based system for triggering musical events, and a distributed multimedia server for audio mapping of a real-time face tracker, and discusses different aspects of mapping strategies in their context. Plausible future directions, developments and exploration with the proposed framework, including stage augmenta tion, virtual and augmented reality, which involve sensing and mapping of physical and non-physical changes onto multimedia control events, are discussed