Mobile-Based Interactive Music for Public Spaces

With the emergence of modern mobile devices equipped with various types of built-in sensors, interactive art has become easily accessible to everyone, musicians and non-musicians alike. These efficient computers are able to analyze human activity, location, gesture, etc., and based on this information dynamically change, or create an artwork in realtime. This thesis presents an interactive mobile system that solely uses the standard embedded sensors available in current typical smart devices such as phones, and tablets to create an audio-only augmented reality for a singled out public space in order to explore the potential for social-musical interaction, without the need for any significant external infrastructure

The User Experience of Mobile Music Making: An Ethnographic Exploration of Music Production and Performance in Practice

The research herein describes the investigation of usability of software and hardware tools for musicians. Through an ethnographic approach, the aim is to broaden the scope of investigation and measure the usability of tools for musicians in a real world setting. Six musicians are observed through the planning and preparation stages, rehearsals, performing and post-performance in order to better understand the tools that they use and how these tools could be improved. This work builds on previous investigations into more traditional production environments. This investigation also explores how requirements and tools have changed. The research highlights main areas of usability problems in navigation, clarity of expression, problems in understanding flow and a mismatch between requirements and software tools that currently exist. The results highlight strengths in the flexibility of such systems and identify where they solve traditional, hardware based problems. The paper culminates in a discussion regarding the values, strengths and weaknesses of hard and soft tools and points to potential future directions of research

INSAM Journal of Contemporary Music, Art and Technology 10 (I/2023)

Having in mind the foundational idea not only of our Journal but also the INSAM Institute itself, the main theme of this issue is titled “Technological Aspects of Contemporary Artistic and Scientific Research”. This theme was recognized as important, timely, and necessary by a number of authors coming from various disciplines. The (Inter)Views section brings us three diverse pieces; the issue is opened by Aida Adžović’s interview with the legendary Slovene act Laibach regarding their performance of the Wir sing das Volk project at the Sarajevo National Theater on May 9, 2023. Following this, Marija Mitrović prepared an interview with media artist Leon Eckard, concerning this artist’s views on contemporary art and the interaction between technology and human sensitivity. An essay by Alexander Liebermann on the early 20th-century composer Erwin Schulhoff, whose search for a unique personal voice could be encouraging in any given period, closes this rubric. The Main theme section contains seven scientific articles. In the first one, Filipa Magalhães, Inês Filipe, Mariana Silva and Henrique Carvalho explore the process and details of technological and artistic challenges of reviving the music theater work FE...DE...RI...CO... (1987) by Constança Capdeville. The second article, written by Milan Milojković, is dedicated to the analysis of historical composer Vojislav Vučković and his ChatGPT-generated doppelganger and opera. The fictional narrative woven around the actual historical figure served as an example of the current possibilities of AI in the domain of musicological work. In the next paper, Luís Arandas, Miguel Carvalhais and Mick Grierson expand on their work on the film Irreplaceable Biography, which was created via language-guided generative models in audiovisual production. Thomas Moore focuses on the Belgium-based Nadar Ensemble and discusses the ways in which the performers of the ensemble understand the concept of the integrated concert and distinguish themselves from it, specifying the broadening of performers’ competencies and responsibilities. In her paper, Dana Papachristou contributes to the discussion on the politics of connectivity based on the examination of three projects: the online project Xenakis Networked Performance Marathon 2022, 2023Eleusis Mystery 91_Magnetic Dance in Elefsina European Capital of Culture, and Spaces of Reflection offline PirateBox network in the 10th Berlin Biennale. The penultimate article in the section is written by Kenrick Ho and presents us with the author’s composition Flou for solo violin through the prism of the relationship between (historically present) algorithmic processes, the composer, and the performer. Finally, Rijad Kaniža adds to the critical discourse on the reshaping of the musical experience via technology and the understanding of said technology using the example of musique concrète. In the final Review section, Bakir Memišević gives an overview of the 13th International Symposium “Music in Society” that was held in Sarajevo in December 2022

Epistemic tools: the phenomenology of digital musical instruments

Digital music technologies, and instruments in particular, are the result of specific systems of thought that define and enframe the user’s creative options. Distinctive divisions between digital and acoustic instruments can be traced, contrasting the conceptually based design of software with the affordances and constraints of physical artefacts. Having lost the world’s gift of physical properties, the digital instrument builder becomes more than a mere luthier. The process of designing and building the instrument is transformed into a process of composition, for it typically contains a greater degree of classification and music theory than its acoustic counterpart. Part I of this thesis begins by framing musical systems in the context of the philosophy of technology. Here technological conditions are questioned and theories introduced that will assist the investigation into the relationship between creativity and technology. After this general grounding, the ramifications digital technologies pose to the human body are explored in the context of human expression through tool use. The human-machine relationship is described from phenomenological perspectives and relevant theories of cognitive science. This analysis serves as a foundation for the concept of epistemic tools, defined as the mechanism whereby techno-cultural models are inscribed into technological artefacts. The cultural element of tool use and tool origins is therefore emphasised, an aspect that is highly relevant in musical technologies. Part I thus frames the material properties of acoustic and digital instruments in relation to human culture, cognition, performance and epistemology. Part II contextualises these theories in practice. The ixiQuarks, the live improvisation musical environment that resulted from the current research, are presented as a system addressing some of the vital problems of musical performance with digital systems (such as the question of embodiment and theoretical inscriptions), proposing an innovative interaction model for screen-based musical instruments. The concept of virtual embodiment is introduced and framed in the context of the ixi interaction model. Two extensive user studies are described that support the report on ixiQuarks. Furthermore, comparative surveys on the relationship between expression and technology are presented: a) the phenomenology of musical instruments, where the divergence between the acoustic and the digital is investigated; b) the question of expressive freedom versus time constraints in musical environments is explored with practitioners in the field; and c) the key players in the design of audio programming environments explain the rationale and philosophy behind their work. These are the first major surveys of this type conducted to date, and the results interweave smoothly with the observations and findings in the chapters on the nature and the design of digital instruments that make up the majority of Part II. This interdisciplinary research investigates the nature of making creative tools in the digital realm, through an active, philosophically framed and ethnographically inspired study, of both practical and theoretical engagement. It questions the nature of digital musical instruments, particularly in comparison with acoustic instruments. Through a survey of material epistemologies, the dichotomy between the acoustic and the digital is employed to illustrate the epistemic nature of digital artefacts, necessitating a theory of epistemic tools. Consequently virtual embodiment is presented as a definition of the specific interaction mode constituting human relations with digital technologies. It is demonstrated that such interactions are indeed embodied, contrasting common claims that interaction with software is a disembodied activity. The role of cultural context in such design is emphasised, through an analysis of how system design is always an intricate process of analyses, categorisations, normalisations, abstractions, and constructions; where the design paths taken are often defined by highly personal, culturally conditioned and often arbitrary reasons. The dissertation therefore dissects the digital musical instrument from the perspectives of ontology, phenomenology and epistemology. Respective sections in Part I and Part II deal with these views. The practical outcome of this research – the ixiQuarks – embodies many of the theoretical points made on these pages. The software itself, together with the theoretical elucidation of its context, should therefore be viewed as equal contributions to the field of music technology. The thesis closes by considering what has been achieved through these investigations of the technological context, software development, user studies, surveys, and the phenomenological and epistemological enquiries into the realities of digital musical instruments, emphasising that technology can never be neutral

Choosers: A Visual Programming Language for Nondeterministic Music Composition by Non-Programmers

This thesis focuses on the design of Choosers, a prototype algorithmic programming system centred around a new abstraction (of the same name) designed to allow non-programmers access to nondeterministic music composition methods. Algorithmic composition typically involves structural elements such as indeterminism, parallelism, choice, multi-choice, nesting, weighting, and looping. There are powerful existing tools for manipulating these and other elements of music. However, while these systems give substantial compositional power to musicians who are also skilled programmers, many musicians who lack programming skills find these tools inaccessible and difficult to understand and use. This thesis presents the design and evaluation of a prototype visual programming language designed to allow structural elements of the kind involved in nondeterministic music composition to be readily visualised and manipulated, while making little or no demand on programming ability. Initially, a Cognitive Dimensions of Notations review of a representative selection of user interfaces for algorithmic composition software was conducted. The review led to a set of findings used to identify candidate design principles which were then tested via a series of design exercises. The findings from these design exercises led to the development of a new abstraction, the Chooser, via a series of iterative design cycles. Once a candidate design had been finalised it was evaluated with participants via two sets of programming walkthroughs, with the findings from each step used to refine the formalism. The final study used Choosers as a design probe through a series of interviews with domain experts in which manipulable compositions were introduced to prompt discussions on potential future implications for music computing education, music production, and music composition

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments