Moving sounds and sonic moves : exploring interaction quality of embodied music mediation technologies through a user-centered perspective

This research project deals with the user-experience related to embodied music mediation technologies. More specifically, adoption and policy problems surrounding new media (art) are considered, which arise from the usability issues that to date pervade new interfaces for musical expression. Since the emergence of new wireless mediators and control devices for musical expression, there is an explicit aspiration of the creative industries and various research centers to embed such technologies into different areas of the cultural industries. The number of applications and their uses have exponentially increased over the last decade. Conversely, many of the applications to date still suffer from severe usability problems, which not only hinder the adoption by the cultural sector, but also make culture participants take a rather cautious, hesitant, or even downright negative stance towards these technologies. Therefore, this thesis takes a vantage point that is in part sociological in nature, yet has a link to cultural studies as well. It combines this with a musicological frame of reference to which it introduces empirical user-oriented approaches, predominantly taken from the field of human-computer-interaction studies. This interdisciplinary strategy is adopted to cope with the complex nature of digital embodied music controlling technologies. Within the Flanders cultural (and creative) industries, opportunities of systems affiliated with embodied interaction are created and examined. This constitutes an epistemological jigsaw that looks into 1) “which stakeholders require what various levels of involvement, what interactive means and what artistic possibilities?”, 2) “the way in which artistic aspirations, cultural prerequisites and operational necessities of (prospective) users can be defined?”, 3) “how functional, artistic and aesthetic requirements can be accommodated?”, and 4) “how quality of use and quality of experience can be achieved, quantified, evaluated and, eventually, improved?”. Within this multi-facetted problem, the eventual aim is to assess the applicability of the foresaid technology, both from a theoretically and empirically sound basis, and to facilitate widening and enhancing the adoption of said technologies. Methodologically, this is achieved by 1) applied experimentation, 2) interview techniques, 3) self-reporting and survey research, 4) usability evaluation of existing devices, and 5) human-computer interaction methods applied – and attuned – to the specific case of embodied music mediation technologies. Within that scope, concepts related to usability, flow, presence, goal assessment and game enjoyment are scrutinized and applied, and both task- and experience-oriented heuristics and metrics are developed and tested. In the first part, covering three chapters, the general context of the thesis is given. In the first chapter, an introduction to the topic is offered and the current problems are enumerated. In the second chapter, a broader theoretical background is presented of the concepts that underpin the project, namely 1) the paradigm of embodiment and its connection to musicology, 2) a state of the arts concerning new interfaces for musical expression, 3) an introduction into HCI-usability and its application domain in systematic musicology, 4) an insight into user-centered digital design procedures, and 5) the challenges brought about by e-culture and digitization for the cultural-creative industries. In the third chapter, the state of the arts concerning the available methodologies related to the thesis’ endeavor is discussed, a set of literature-based design guidelines are enumerated and from this a conceptual model is deduced which is gradually presented throughout the thesis, and fully deployed in the “SoundField”-project (as described in Chapter 9). The following chapters, contained in the second part of the thesis, give a quasi-chronological overview of how methodological concepts have been applied throughout the empirical case studies, aimed specifically at the exploration of the various aspects of the complex status quaestionis. In the fourth chapter, a series of application-based tests, predominantly revolving around interface evaluation, illustrate the complex relation between gestural interfaces and meaningful musical expression, advocating a more user-centered development approach to be adopted. In the fifth chapter, a multi-purpose questionnaire dubbed “What Moves You” is discussed, which aimed at creating a survey of the (prospective) end-users of embodied music mediation technologies. Therefore, it primarily focused on cultural background, musical profile and preferences, views on embodied interaction, literacy of and attitudes towards new technology and participation in digital culture. In the sixth chapter, the ethnographical studies that accompanied the exhibition of two interactive art pieces, entitled "Heart as an Ocean" & "Lament", are discussed. In these studies, the use of interview and questionnaire methodologies together with the presentation and reception of interactive art pieces, are probed. In the seventh chapter, the development of the collaboratively controlled music-game “Sync-In-Team” is presented, in which interface evaluation, presence, game enjoyment and goal assessment are the pivotal topics. In the eighth chapter, two usability studies are considered, that were conducted on prototype systems/interfaces, namely a heuristic evaluation of the “Virtual String” and a usability metrics evaluation on the “Multi-Level Sonification Tool”. The findings of these two studies in conjunction with the exploratory studies performed in association with the interactive art pieces, finally gave rise to the “SoundField”-project, which is recounted in full throughout the ninth chapter. The integrated participatory design and evaluation method, presented in the conceptual model is fully applied over the course of the “SoundField”-project, in which technological opportunities and ecological validity and applicability are investigated through user-informed development of numerous use cases. The third and last part of the thesis renders the final conclusions of this research project. The tenth chapter sets out with an epilogue in which a brief overview is given on how the state of the arts has evolved since the end of the project (as the research ended in 2012, but the research field has obviously moved on), and attempts to consolidate the implications of the research studies with some of the realities of the Flemish cultural-creative industries. Chapter eleven continues by discussing the strengths and weaknesses of the conceptual model throughout the various stages of the project. Also, it comprises the evaluation of the hypotheses, how the assumptions that were made held up, and how the research questions eventually could be assessed. Finally, the twelfth and last chapter concludes with the most important findings of the project. Also, it discusses some of the implications on cultural production, artistic research policy and offers an outlook on future research beyond the scope of the “SoundField” project

An introduction to interactive sonification

The research field of sonification, a subset of the topic of auditory display, has developed rapidly in recent decades. It brings together interests from the areas of data mining, exploratory data analysis, human–computer interfaces, and computer music. Sonification presents information by using sound (particularly nonspeech), so that the user of an auditory display obtains a deeper understanding of the data or processes under investigation by listening

Proceedings of the Third Annual Virginia Tech Center for Human-Computer Interaction Research Experience for Undergraduates (REU) Symposium

Virginia Tech's Center for Human-Computer Interaction presents the project abstracts for the REU ’08 symposium. The REU (Research Experience for Undergraduates) program provides undergraduate students from various universities with the opportunity to spend eight weeks at Virginia Tech, working with our faculty and graduate students on research projects using the state-of-the-art technology and laboratories assembled here. The REU program is sponsored by a National Science Foundation grant IIS-0552732

GpsTunes: controlling navigation via audio feedback

We combine the functionality of a mobile Global Positioning System (GPS) with that of an MP3 player, implemented on a PocketPC, to produce a handheld system capable of guiding a user to their desired target location via continuously adapted music feedback. We illustrate how the approach to presentation of the audio display can benefit from insights from control theory, such as predictive 'browsing' elements to the display, and the appropriate representation of uncertainty or ambiguity in the display. The probabilistic interpretation of the navigation task can be generalised to other context-dependent mobile applications. This is the first example of a completely handheld location- aware music player. We discuss scenarios for use of such systems

Survey on geographic visual display techniques in epidemiology: Taxonomy and characterization

Many works have been done on the topic of Geographic Visual Display with different objectives and approaches. There are studies to compare the traditional cartography techniques (the traditional term of Geographic Visual Display (GVD) without Human-Computer Interaction (HCI)) to Modern GIS which are also known as Geo-visualization, some literature differentiates and highlight the commonalities of features and architectures of different Geographic Visual Display tools (from layers and clusters to dot and color and more). Furthermore, with the existence of more advanced tools which support data exploration, few tasks are done to evaluate how those tools are used to handle complex and multivariate spatial-temporal data. Several test on usability and interactivity of tools toward user's needs or preferences, some even develop frameworks that address user's concern in a wide array of tasks, and others prove how these tools are able to stimulate the visual thought process and help in decision making or event prediction amongst decision-makers. This paper surveyed and categorized these research articles into 2 categories: Traditional Cartography (TC) and Geo-visualization (G). This paper will classify each category by their techniques and tasks that contribute to the significance of data representation in Geographic Visual Display and develop perspectives of each area and evaluating trends of Geographic Visual Display Techniques. Suggestions and ideas on what mechanisms can be used to improve and diversify Geographic Visual Display Techniques are provided at the end of this survey

Tangible auditory interfaces : combining auditory displays and tangible interfaces

Bovermann T. Tangible auditory interfaces : combining auditory displays and tangible interfaces. Bielefeld (Germany): Bielefeld University; 2009.Tangible Auditory Interfaces (TAIs) investigates into the capabilities of the interconnection of Tangible User Interfaces and Auditory Displays. TAIs utilise artificial physical objects as well as soundscapes to represent digital information. The interconnection of the two fields establishes a tight coupling between information and operation that is based on the human's familiarity with the incorporated interrelations. This work gives a formal introduction to TAIs and shows their key features at hand of seven proof of concept applications

Usability inspection for sonification applications

Bonification is the representation of data using mainly non-speech sound for the purpose of communication and interpretation. The process and technique of converting the data into sound is called the sonification technique. One or more techniques might be required by a sonification application. However, sonification techniques are not generally suitable for all kinds of data, and often custom techniques are used - where the design is tailored to the domain and nature of the data as well as the users' required tasks within the application. Therefore, it is important to assure the usability of the technique for the specific domain application being developed. In previously reported research, most designers of sonification applications have needed to develop at least a prototype for user testing. The result are interpreted and analysed to look for potential problems and solutions to improve the design. This dissertation has developed a new systematic usability inspection approach called the Task Interpretation Walkthrough (TIW) for the design of sonification application before they go to the initial development phase. It is hypothesized that designers of sonification applications will be able to detect significantly more important potential usability problems before the implementation phase by analysing the interaction between the user and the application as well as paying attention to the different stages of how the data is transformed into sound. It uses two new models - the Sonification Application (SA) model and the User Interpretation Construction (UIC) model. Four experiments with human subjects were carried out to study the feasibility and effectiveness of Task Interpretation Walkthrough inspection by comparing it against two widely used techniques; Heuristic Evaluation and Cognitive Walkthrough. The sonification designs being inspected were a Mobile Phone Joystick Text-Entry with Sound (Experiments I and II), a Diagnosis Tool for Analysis of The Motion and Usage of a Patient's Arm (Experiment III); and an Audio-Visual Analysis Tool of Cervical Sample Slides (Experiment IV). The participants included sound researchers (Experiment II); and students with a background in music technology and software engineering (Experiments I, III and IV), acting either individually or in 2-person groups. The results have shown that the research hypothesis is supported, where the significantly important usability problems were able to be detected before the implementation phase. From the inspection method comparison study, results showed the Task Interpretation Walkthrough to be more effective than the existing techniques (Heuristic Evaluation and Cognitive Walkthrough)