Audio mixing interfaces (AMIs) commonly conform to a small number of paradigms. These paradigms have

significant shortcomings. Data visualisation first principles should be employed to consider alternatives. Existing AMI

paradigms are discussed and concepts of image theory and elementary perceptual elements outlined. AMIs should be evaluated by usability experiments however performing these properly is time-consuming. There are many data visualisation options and combinations. Collaboration with others would enable a greater range to be explored. Better understanding data visualisation will benefit audio and music interface development in general

Dewey, Christopher

Wakefield, Jonathan P.

University of Huddersfield Repository

University of Huddersfield RepositoryDewey, Christopher and Wakefield, Jonathan P.Audio interfaces should be designed based on data visualisation first principlesOriginal CitationDewey, Christopher and Wakefield, Jonathan P. (2016) Audio interfaces should be designed based on data visualisation first principles. In: Proceedings of the 2nd AES Workshop on Intelligent Music Production. Audio Engineering Society. This version is available at http://eprints.hud.ac.uk/32149/The University Repository is a digital collection of the research output of theUniversity, available on Open Access. Copyright and Moral Rights for the itemson this site are retained by the individual author and/or other copyright owners.Users may access full items free of charge; copies of full text items generallycan be reproduced, displayed or performed and given to third parties in anyformat or medium for personal research or study, educational or not­for­profitpurposes without prior permission or charge, provided:• The authors, title and full bibliographic details is credited in any copy;• A hyperlink and/or URL is included for the original metadata page; and• The content is not changed in any way.For more information, including our policy and submission procedure, pleasecontact the Repository Team at: E.mailbox@hud.ac.uk.http://eprints.hud.ac.uk/Proceedings of the 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016  Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). AUDIO INTERFACES SHOULD BE DESIGNED BASED ON DATA VISUALISATION FIRST PRINCIPLES Christopher Dewey and Jonathan P. Wakefield University of Huddersfield {c.dewey,j.p.wakefield}@hud.ac.uk ABSTRACT Audio mixing interfaces (AMIs) commonly conform to a small number of paradigms. These paradigms have significant shortcomings. Data visualisation first principles should be employed to consider alternatives. Existing AMI paradigms are discussed and concepts of image theory and elementary perceptual elements outlined. AMIs should be evaluated by usability experiments however performing these properly is time-consuming. There are many data visualisation options and combinations. Collaboration with others would enable a greater range to be explored. Better understanding data visualisation will benefit audio and music interface development in general.  1. INTRODUCTION Our current research focuses on audio mixing interface (AMI) design. This paper considers the relative strengths and weaknesses of current AMI paradigms. The paper moves on to our approach to AMI design which is primarily concerned with effective data visualisation and its direct manipulation. 2. AMI PARADIGMS Originally, the layout of the AMI was dictated by its underlying analogue electronic components leading to a one-to-one mapping of controls. Since the 1970’s most AMIs have continued to follow this layout despite evolving from mainly analogue to mainly digital and software solutions and the original implementation restrictions no longer existing. This design is termed the channel strip paradigm (CSP) (see figure 1). Researchers [1] have questioned whether this commercially established paradigm meets the needs of the user and have proposed alternative designs based on psychoacoustic principles that correlate with sound localization in humans [2]. These AMIs are termed the stage paradigm.   Figure 1: Channel Strip Paradigm (CSP)   The concept behind this paradigm is that each audio channel is represented on a graphical representation of a stage by an icon/node. The position of each icon/node on the stage represents its level and pan. In contrast to the CSP the stage paradigm adopts a ‘depth mixing’ approach [3] with regard to channel level with the icons/nodes closest to the user having the highest level. Although few commercial embodiments of this paradigm exist [5], it has been muted as a possible alternative to the CSP in academic literature given its psychoacoustic advantages. Ratcliffe [2] helps define this paradigm further by distinguishing those solutions that feature a 3 dimensional stage and those that feature a two dimensional stage.   The three-dimensional stage paradigm (3DSP) was the first attempt to present an alternative to the CSP and features a virtual cuboid stage with individual audio channels represented as coloured spheres as shown in figure 2.   Figure 2: Three-dimensional stage paradigm (3DSP) The two-dimensional stage paradigm (2DSP) is shown in figure 3. In contrast to the 3DSP the 2DSP graphically presents a listening position aligned centrally at the bottom of the stage. The relative distance of each circle from this listening position relates to the channel’s level with those closer to the listening point being louder than those further away. The relative angle of each circle from the listening point defines the channel’s pan position.    Figure 3: Two-dimensional stage paradigm (2DSP) Ratcliffe [2] argues that whilst the one-to-one mapping of parameters in the CSP offers precise control over many Proceedings of the 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016  Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). mix parameters, this paradigm offers no direct way to visualise the stereo distribution of audio channels as the user must scrutinize each channel’s pan knob position to assemble a mental image. Furthermore a channel to the left of the console may well be panned to the right potentially causing cognitive confusion. This assertion is reinforced by Mycroft et al [3] who argue that this visual task places an undue cognitive load on the user, detracting from their performance of the auditory tasks.   The 2DSP and 3DSP represent a significant improvement over CSP in enabling the user to visualise the absolute and relative spatial distribution between audio channels. Unfortunately these visualisations can become cluttered in real-world scenarios [4]. This is because channels with similar pan positions and level will overlap each other on the display (as illustrated by channels 2 and 4 overlapping in figures 2 and 3). This represents a deficiency with this paradigm.   The role of the mixing console in music production has changed with the distinction between what is a digital musical instrument (DMI) and an AMI blurred. AMIs should be considered as DMIs and the interface should provide the user with information about the mix elements so they can make informed creative decisions when producing music. 3. DATA VISUALISATION FIRST PRINCIPLES Shneiderman [6] asserts that interface designers are increasingly using data visualisations to display dynamic information because visual displays take advantage of the users’ cognitive ability to detect changes in colour, shape, size and texture.   Bertin’s [7] ‘Image Theory' supports the creation of effective data visualisations. Bertin defines an image as the fundamental perceptual unit of any visualisation with each image consisting of two parts termed components and invariants. A component is the concept conveyed to the user and an invariant relates these components together. Ideally, one image should be presented to the user for simplicity. Bertin advises that an optimum of three visual variables can be perceived and understood by the user in each image. These visual variables are classified as either planar or retinal. Planar variables exist as spatial dimensions and retinal variables include size, colour, shape, orientation or texture.   Cleveland et al [8] identify ten ‘elementary perceptual tasks’, which closely relate to these visual variables and suggest that we perform multiple elementary perceptual tasks when abstracting information from any visualisation. These tasks can be ordered in terms of accuracy through experimentation.  Bertin and Cleveland’s work provides a set of guidelines that can be used when designing new AMIs. There are many experiments to be conducted to comprehensively cover all data visualisation options and even greater number of combinations. AMIs should only be evaluated by usability experiments [9] which are are time consuming. Consequently, collaboration with others would enable a greater range of options and combinations to be explored. 4. CONCLUSIONS AMIs should be designed based on data visualisation principles in conjunction with interaction requirements. Furthermore data visualisation and interaction style need to be considered simultaneously as existing work has often focused on one or the other.     By considering the planar and retinal variables defined by Bertin and the elementary perceptual tasks defined by Cleveland in the context of AMI we can identify the most intuitive graphical visualisations possible and overcome the short-comings of the current AMI paradigms. Furthermore, this is applicable more widely in designing better audio and music interfaces.   Usability evaluation should be rigorously undertaken to determine whether a design is successful. For example, it is not sufficient to assume that a design is good because it is accepted or based on human perception. Furthermore evaluation contexts must be realistic and not overly simplified.  Tools used in music production should be considered as DMIs as they are often used creatively and expressively. Consequently DMI design can learn from AMI design and vice-versa. 5. REFERENCES [1] Mycroft, J. et al. Audio Mixing Displays: Influence of Overviews on Information Search and Critical Listening. [2] Ratcliffe, J. (2014). Hand motion-controlled audio mixing interface. NIME. [3] Mycroft, J. et al (2013). The influence of graphical user interface design on critical listening skills. SMC. [4] Gelineck, S. et al (2013). Towards a more flexible and creative music mixing interface. CHI'13 Extended Abstracts on Human Factors in Computing Systems. [5] Bourne, H. et al (2013). Evaluating iBall—An Intuitive Interface and Assistive Audio Mixing Algorithm for Live Football Events. 135th AES Conv. [6] Shneiderman, S. B. et al  (2005). Designing the user interface. Pearson [7] Bertin, J. (1983). Semiology of graphics: Diagrams, maps networks. University of Wisconsin Press. [8] Cleveland, W. S. et al (1984). Graphical perception: Theory, experimentation, and application to the development of graphical methods. JASA. [9] Dewey, C. et al (2014). A guide to the design and evaluation of new user interfaces for the audio industry. 135th AES Conv. 

Audio interfaces should be designed based on data visualisation first principles

Abstract

Similar works

Full text

Available Versions

University of Huddersfield Repository