THINGNESS: COMPOSITIONAL STRATEGIES FOR EMERGING VIRTUAL SONIC OBJECTS

This selection of ten artworks represents the conclusion of my aesthetic and conceptual interests related to composition for the last three years. This research has been focused on the sound composition of immaterial vivid objects, as abstract articulated xeno entities which emerges from the listening. After introducing key concepts and influences from the repertoire and literature, this thesis explores an essential concept called ‘thingness’, which constitutes the main core in the full discourse of my artistic research. My definition of objects starts from this principle, which is unfolded in four main categories: thingness in relation with sound, the listener, time, and space. Thingness has been applied in two types of works: sound installations and mediabased pieces, and likewise extended to collaborations, challenging and strengthening my views within pieces of a wider aesthetic range

Sculpting Unrealities: Using Machine Learning to Control Audiovisual Compositions in Virtual Reality

This thesis explores the use of interactive machine learning (IML) techniques to control audiovisual compositions within the emerging medium of virtual reality (VR). Accompanying the text is a portfolio of original compositions and open-source software. These research outputs represent the practical elements of the project that help to shed light on the core research question: how can IML techniques be used to control audiovisual compositions in VR? In order to find some answers to this question, it was broken down into its constituent elements. To situate the research, an exploration of the contemporary field of audiovisual art locates the practice between the areas of visual music and generative AV. This exploration of the field results in a new method of categorising the constituent practices. The practice of audiovisual composition is then explored, focusing on the concept of equality. It is found that, throughout the literature, audiovisual artists aim to treat audio and visual material equally. This is interpreted as a desire for balance between the audio and visual material. This concept is then examined in the context of VR. A feeling of presence is found to be central to this new medium and is identified as an important consideration for the audiovisual composer in addition to the senses of sight and sound. Several new terms are formulated which provide the means by which the compositions within the portfolio are analysed. A control system, based on IML techniques, is developed called the Neural AV Mapper. This is used to develop a compositional methodology through the creation of several studies. The outcomes from these studies are incorporated into two live performance pieces, Ventriloquy I and Ventriloquy II. These pieces showcase the use of IML techniques to control audiovisual compositions in a live performance context. The lessons learned from these pieces are incorporated into the development of the ImmersAV toolkit. This open-source software toolkit was built specifically to allow for the exploration of the IML control paradigm within VR. The toolkit provides the means by which the immersive audiovisual compositions, Obj_#3 and Ag Fás Ar Ais Arís are created. Obj_#3 takes the form of an immersive audiovisual sculpture that can be manipulated in real-time by the user. The title of the thesis references the physical act of sculpting audiovisual material. It also refers to the ability of VR to create alternate realities that are not bound to the physics of real-life. This exploration of unrealities emerges as an important aspect of the medium. The final piece in the portfolio, Ag Fás Ar Ais Arís takes the knowledge gained from the earlier work and pushes the boundaries to maximise the potential of the medium and the material

Aesthetic Animism: Digital Poetry as Ontological Probe

This thesis is about the poetic edge of language and technology. It inter-relates both computational creation and poetic reception by analysing typographic animation softwares and meditating (speculatively) on a future malleable language that possesses the quality of being (and is implicitly perceived as) alive. As such it is a composite document: a philosophical and practice-based exploration of how computers are transforming literature, an ontological meditation on life and language, and a contribution to software studies. Digital poetry introduces animation, dimensionality and metadata into literary discourse. This necessitates new terminology; an acronym for Textual Audio-Visual Interactivity is proposed: Tavit. Tavits (malleable digital text) are tactile and responsive in ways that emulate living entities. They can possess dimensionality, memory, flocking, kinematics, surface reflectivity, collision detection, and responsiveness to touch, etc…. Life-like tactile tavits involve information that is not only semantic or syntactic, but also audible, imagistic and interactive. Reading mediated language-art requires an expanded set of critical, practical and discourse tools, and an awareness of the historical continuum that anticipates this expansion. The ontological and temporal design implications of tavits are supported with case-studies of two commercial typographic-animation softwares and one custom software (Mr Softie created at OBX Labs, Concordia) used during a research-creation process

Resonance as a design strategy for AI and social robots

Resonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human–robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of “sympathetic resonance” as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human–robot interactions

What can a sonic assemblage do? A biopsychosocial approach to post-acousmatic composition

Thinking and sounding are two terms which complicate one another, hence this thesis follows two trajectories each of which make an original contribution to knowledge. Part 1 (thinking sound) proposes to reground composition away from historically authoritative humanist models, instead suggesting a biopsychosocial approach for a post-acousmatic music. I elaborate a set of models and key concepts, chiefly an eliminativist account of the listener-sound relation; neurocognitively discrete musical domains and dimensions of the Kmatrix; model-based reasoning through a Reception-Interpretation-Action helix; and, mentalizing listening stances based upon dual-process cognition models. This is combined with an art-activist stance where composition is concerned with the effects that a sonic artobject exerts in its vicinity. I propose composition as experimentally concerned with generating new epistemic things through a process of assemblage and heterogeneous engineering. Part 2 (sounding thinking) discusses fixed and live compositions which initiated and respond to my proposed approach. In my practice, I focus on the disruption of specific aesthetic regimens to bring listening into attentional focus, engaging the specificity of the mnemonic traces that sound leaves. The pieces are largely concerned with sonic cultures related to Islam and the MENASA region

Expanding Fields of Architectural Discourse and Practice: Curated Works from the P.E.A.R. Journal.

Expanding Fields of Architectural Discourse and Practice presents a selection of essays, architectural experiments and works that explore the diversity within the fields of contemporary architectural practice and discourse. Specific in this selection is the question of how and why architecture can and should manifest in a critical and reflective capacity, as well as to examine how the discipline currently resonates with contemporary art practice. It does so by reflecting on the first 10 years of the architectural journal, P.E.A.R. (2009 to 2019). The volume argues that the initial aims of the journal – to explore and celebrate the myriad forms through which architecture can exist – are now more relevant than ever to contemporary architectural discourse and practice