Emergent Perception and Video Games that Listen: Applying Sonic Virtuality for Creative and Intelligent NPC Behaviours

‘Non-player characters (NPCs)’ can present well-crafted behaviours and evoke engaging and immersive player experiences but such behaviour in current NPCs is illusory and only achievable within a controlled and linear/fixed video game context. NPCs struggle greatly to portray flexible or creative behaviours within an adaptive or procedurally generated environment and this is even more apparent in their relationship with sound. This paper posits that recent theoretical developments in cognitive psychology offer significant opportunity to advance NPC-AI and proposes that an intelligence framework, based upon Sonic Virtuality and integrated within an NPC, would offer distinct advantages over current systems. To illustrate this vision, a roadmap for future work is laid out using Sonic Virtuality as the foundation for a ‘synthetic listener’; an NPC capable of responding to procedurally generated and external (player-domain) audio. As a philosophical exploration, underlying principles are considered for other perception modalities, presenting an avenue of games-AI research that, ultimately, could dramatically improve NPC- ‘humanness’ and evoke a player-immersion and presence equivalent to linear/fixed AI but in much bigger, more complex virtual worlds

On the Fluidity of Honey and Fugitivity of Sound in Trauma, Ecstasy, and Black Radical Tradition

From Merleau-Ponty’s phenomenological perspective to Derridean postructuralist view, to an intersectional force traversing somatic, social, political, and cultural, sound, in its non-linear epistemology, breaks barriers between forms and escapes any structured definitions. Like the insidious stickiness of honey, sound’s viscosity invaginates, spreads onto the interior, and, by triggering memories and the somatic, threatens the very totality of our identities. At that rupturing moment, we are not the ones subjecting sound to be known as an object; instead, in its fugitive protest and agency, sound flips the roles of the knower and the known and establishes new possibilities of relating to it, of understanding ourselves, and of listening to the world around us. Using the theoretical framework of Fred Moten’s formative volume, In the Break: The Aesthetics of the Black Radical Tradition, the current thesis explores the sound works of two contemporary artists, Christine Sun Kim and Camille Norment, which unsettle the historicity and the ideology of normativity and oppression in sound. By placing these works in close dialogue with Moten’s complex critical analysis, I look for the interinanimation of the drives behind the Black radical tradition in music and literature mid-twentieth century and the artistic exploration of sound in the last seven years. Furthermore, I follow In the Break’s provocative engagements between Western philosophy (Marx, Freud, and Derrida) and Black radical thought (Fanon, Spillers, Menakem, and Delany) to uncover the operative functioning of both in destigmatizing the ways we understand and relate to sound

What can a sonic assemblage do? A biopsychosocial approach to post-acousmatic composition

Thinking and sounding are two terms which complicate one another, hence this thesis follows two trajectories each of which make an original contribution to knowledge. Part 1 (thinking sound) proposes to reground composition away from historically authoritative humanist models, instead suggesting a biopsychosocial approach for a post-acousmatic music. I elaborate a set of models and key concepts, chiefly an eliminativist account of the listener-sound relation; neurocognitively discrete musical domains and dimensions of the Kmatrix; model-based reasoning through a Reception-Interpretation-Action helix; and, mentalizing listening stances based upon dual-process cognition models. This is combined with an art-activist stance where composition is concerned with the effects that a sonic artobject exerts in its vicinity. I propose composition as experimentally concerned with generating new epistemic things through a process of assemblage and heterogeneous engineering. Part 2 (sounding thinking) discusses fixed and live compositions which initiated and respond to my proposed approach. In my practice, I focus on the disruption of specific aesthetic regimens to bring listening into attentional focus, engaging the specificity of the mnemonic traces that sound leaves. The pieces are largely concerned with sonic cultures related to Islam and the MENASA region

A General Theory Of Composition: Cross-modal Perception, Sound, Synergy and Meaning in contemporary composition practice.

Music extends beyond audibility. We perceive a performance – the work-on-stage – not only sonically but also visually and physically. Bridging science, phenomenology, eastern philosophy and music, A General Theory of Composition explores cross-modal perception and new synergies between musical and non-musical modes of creative expression through the media of sound, composition as a whole, and interdisciplinary research by posing the key-questions: What is SOUND? Where does SOUND begin, and where does it end? A General Theory of Composition, as a phenomenological investigation, reconfirms that the nature of both artistic practice and our sensory perception is holistic and reciprocally cross-modal. ‘Multi-sensory fluidity’ (Coessens), ‘sonic sensibility’ (Voegelin), ‘sensory substitution’ (McGann), naturally embedded in our modes of perception, suggest that our perception is always ‘embodied’ (Johnson) within its complex interdisciplinary Gestalt – composition as a whole. From this perspective the multisensory human body can be seen to engage meaningfully with the world via cross-modal listening, that promotes multi-sensory, psychosomatic reciprocal exchange between audible-visual-physical modes of perception. From the observation of my own artistic praxis, the holistic perspective of my theory brings forward the view on musical and/or any practice as a ‘lived experience’ (Stein) through theories of The Empathy Theory (Stein), Spheres of Human Essence (Walther), and The Speech Act Theory (Austin). It shares insights that: 1) sound/music, as language, is a phenomenon rooted in the philosophical domain; 2) there is no pure medium – all media are multi-modal, inter-subjective and reciprocally interconnected; 3) through the mode of ‘active listening’, as a means of communication, sound naturally crosses into other domains; 4) our perceptual modes intertwine, synthesise, and co-exist in a synergetic relationship within the complex chamber of our multi-sensory, psychosomatic, physical bodies. The exegesis is accompanied by two creative works, LIBERATO and SKETCHES, presented in three contextual subdivisions that collectively illustrate the workings of my theory and creative practice: the work-on-stage; cross-modal perception; expanded musical notation