Computational composition strategies in audiovisual laptop performance

We live in a cultural environment in which computer based musical performances have become ubiquitous. Particularly the use of laptops as instruments is a thriving practice in many genres and subcultures. The opportunity to command the most intricate level of control on the smallest of time scales in music composition and computer graphics introduces a number of complexities and dilemmas for the performer working with algorithms. Writing computer code to create audiovisuals offers abundant opportunities for discovering new ways of expression in live performance while simultaneously introducing challenges and presenting the user with difficult choices. There are a host of computational strategies that can be employed in live situations to assist the performer, including artificially intelligent performance agents who operate according to predefined algorithmic rules. This thesis describes four software systems for real time multimodal improvisation and composition in which a number of computational strategies for audiovisual laptop performances is explored and which were used in creation of a portfolio of accompanying audiovisual compositions

The Visualization and Representation of Electroacoustic Music

In Chapters 1 and 2 there are definitions and a review of electroacoustic music, and then visualization generally and as applied to music. Chapter 3 is a review of specific and relevant literature as regards to the visualization of electroacoustic music. Chapter 4 introduces the concepts of imagining as opposed to discovering new sound, and what is important to this research about these terms; in addition what is meant and indicated by them. Chapter 5 deals with the responses that composers currently working have made to the enquiry concerning visualization. In this chapter these responses are dealt with as case studies. In a similar way, Chapter 6 looks at some examples of historical work in electroacoustic music, again as case studies. In Chapter 7 a taxonomical structure for the use of visualization in electroacoustic composition is established and derived from the case study results. Chapter 8 looks at relevant examples of software and how they offer visualization case studies. Chapter 9 looks at the place of the archive in various stages of the compositional process. Chapter 10 investigates the problems of visualizing musical timbre as possible evidence for future strategies. Chapter 11 offers some conclusions and implications as to the main research questions, as well as more specific outlines of potential strategies for the visualization of electroacoustic music

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

The Design of a Smart City Sonification System Using a Conceptual Blending and Musical Framework, Web Audio and Deep Learning Techniques

Presented at the 26th International Conference on Auditory Display (ICAD 2021) 25-28 June 2021, Virtual conference.This paper describes an auditory display system for smart city data for Dublin City, Ireland. It introduces and describes the different layers of the system and outlines how they operate individually and interact with one another. The system uses a deep learning model called a variational autoencoder to generate musical content to represent data points. Further data-to-sound mappings are introduced via parameter mapping sonification techniques during sound synthesis and post-processing. Conceptual blending and music theory provide frameworks, which govern the design of the system. The paper ends with a discussion of the design process that contextualizes the contribution, highlighting the interdisciplinary nature of the project, which spans data analytics, music composition and human-computer interaction

SimScene : a web-based acoustic scenes simulator

International audienceWe introduce in this paper a soundscape simulator called SimScene, designed to be used as an experimental tool to characterize the mental representation of sound environments. The soundscape simulator allows a subject to generate a full sonic environment by sequencing and mixing sound elements, and manipulating their sound level and time positioning. To make the simulation process effective, SimScene has not be designed to manipulate individual parameters of individ-ual sounds, but to specify high-level parameters for whole classes of sounds, organized into a hierarchical semantically structured dataset. To avoid any linguistic bias, a listening oriented interface allows subjects to explore the dataset with-out any text written help. The entire software is developed in Javascript using the standard Web Audio technology, and is thus fully supported by most modern web browsers. This fact should allow experimenters to adopt a crowdsourcing approach to experimentation in order to assess hypotheses on large populations, and facilitate the development of ex-perimental protocols to investigate the influence of socio-cultural background on soundscape perception

STOCHASTIC INVERSION INTEGRATING SEISMIC DATA, LITHO-FACIES PHYSICAL PROPERTIES, AND MULTIPLE-POINT GEOSTATISTICS FOR RESERVOIR CHARACTERIZATION

We proposed a novel seismic inversion approach that integrates the physical properties of litho-facies, and geophysical data, within the multiple-point geostatistical frameworks to reduce the uncertainty in predictions of litho-facies spatial arrangement away from wells or control points. The litho-facies groups (rock-type) in the well locations are defined and conditioned to the distribution of elastic properties, including P-wave velocity (Vp) and facies density (ρ) in the well locations. A conceptual geological model (training image) is utilized within a wavelet-based multiple-point geostatistical simulation (WAVESIM) algorithm to generate litho-facies realizations. In our inversion algorithm, the forward model is created by implementing the bivariate Kernel density estimation technique of the litho-facies properties (Vp and ρ) that are distributed in the well locations. The inversion approach is an iterative process, where a particular number of elastic properties (Vp and ρ) for each WAVESIM realization are drawn, and then the forward model was utilized to create synthetic seismograms. For each generated set of the WAVESIM realizations, a series of synthetic seismograms are produced, and one realization is selected that provides the best-match synthetic seismogram compared to the input seismic data using crosscorrelation function. Our inversion technique was successfully applied to synthetic and field datasets. The results demonstrate the efficiency of our inversion approach to characterize highly heterogeneous reservoirs

Dataremix: Aesthetic Experiences of Big Data and Data Abstraction

This PhD by published work expands on the contribution to knowledge in two recent large-scale transdisciplinary artistic research projects: ATLAS in silico and INSTRUMENT | One Antarctic Night and their exhibited and published outputs. The thesis reflects upon this practice-based artistic research that interrogates data abstraction: the digitization, datafication and abstraction of culture and nature, as vast and abstract digital data. The research is situated in digital arts practices that engage a combination of big (scientific) data as artistic material, embodied interaction in virtual environments, and poetic recombination. A transdisciplinary and collaborative artistic practice, x-resonance, provides a framework for the hybrid processes, outcomes, and contributions to knowledge from the research. These are purposefully and productively situated at the objective | subjective interface, have potential to convey multiple meanings simultaneously to a variety of audiences and resist disciplinary definition. In the course of the research, a novel methodology emerges, dataremix, which is employed and iteratively evolved through artistic practice to address the research questions: 1) How can a visceral and poetic experience of data abstraction be created? and 2) How would one go about generating an artistically-informed (scientific) discovery? Several interconnected contributions to knowledge arise through the first research question: creation of representational elements for artistic visualization of big (scientific) data that includes four new forms (genomic calligraphy, algorithmic objects as natural specimens, scalable auditory data signatures, and signal objects); an aesthetic of slowness that contributes an extension to the operative forces in Jevbratt’s inverted sublime of looking down and in to also include looking fast and slow; an extension of Corby’s objective and subjective image consisting of “informational and aesthetic components” to novel virtual environments created from big 3 (scientific) data that extend Davies’ poetic virtual spatiality to poetic objective | subjective generative virtual spaces; and an extension of Seaman’s embodied interactive recombinant poetics through embodied interaction in virtual environments as a recapitulation of scientific (objective) and algorithmic processes through aesthetic (subjective) physical gestures. These contributions holistically combine in the artworks ATLAS in silico and INSTRUMENT | One Antarctic Night to create visceral poetic experiences of big data abstraction. Contributions to knowledge from the first research question develop artworks that are visceral and poetic experiences of data abstraction, and which manifest the objective | subjective through art. Contributions to knowledge from the second research question occur through the process of the artworks functioning as experimental systems in which experiments using analytical tools from the scientific domain are enacted within the process of creation of the artwork. The results are “returned” into the artwork. These contributions are: elucidating differences in DNA helix bending and curvature along regions of gene sequences specified as either introns or exons, revealing nuanced differences in BLAST results in relation to genomics sequence metadata, and cross-correlation of astronomical data to identify putative variable signals from astronomical objects for further scientific evaluation

Developing a flexible and expressive realtime polyphonic wave terrain synthesis instrument based on a visual and multidimensional methodology

The Jitter extended library for Max/MSP is distributed with a gamut of tools for the generation, processing, storage, and visual display of multidimensional data structures. With additional support for a wide range of media types, and the interaction between these mediums, the environment presents a perfect working ground for Wave Terrain Synthesis. This research details the practical development of a realtime Wave Terrain Synthesis instrument within the Max/MSP programming environment utilizing the Jitter extended library. Various graphical processing routines are explored in relation to their potential use for Wave Terrain Synthesis