Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts

This thesis’s primary goal is to investigate performance ecologies, that is the compound of humans, artifacts and environmental elements that contribute to the result of a per- formance. In particular, this thesis focuses on designing new interactive technologies for sound and music. The goal of this thesis leads to the following Research Questions (RQs): • RQ1 How can the design of interactive sonic artifacts support a joint expression across different actors (composers, choreographers, and performers, musicians, and dancers) in a given performance ecology? • RQ2 How does each different actor influence the design of different artifacts, and what impact does this have on the overall artwork? • RQ3 How do the different actors in the same ecology interact, and appropriate an interactive artifact? To reply to these questions, a new framework named ARCAA has been created. In this framework, all the Actors of a given ecology are connected to all the Artifacts throughout three layers: Role, Context and Activity. This framework is then applied to one systematic literature review, two case studies on music performance and one case study in dance performance. The studies help to better understand the shaded roles of composers, per- formers, instrumentalists, dancers, and choreographers, which is relevant to better design interactive technologies for performances. Finally, this thesis proposes a new reflection on the blurred distinction between composing and designing a new instrument in a context that involves a multitude of actors. Overall, this work introduces the following contributions to the field of interaction design applied to music technology: 1) ARCAA, a framework to analyse the set of inter- connected relationship in interactive (music) performances, validated through 2 music studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de- signing music interactive system for performance (music or dance), accounting for the needs of the various actors and for the overlapping on music composition and design of in- teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings; 4) Proposal of a methodological approach combining autobiographical and idiographical design approaches in interactive performances.O objetivo principal desta tese é investigar as ecologias performativas, conjunto formado pelos participantes humanos, artefatos e elementos ambientais que contribuem para o resultado de uma performance. Em particular, esta tese foca-se na conceção de novas tecnologias interativas para som e música. O objetivo desta tese originou as seguintes questões de investigação (Research Questions RQs): • RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con- junta entre diferentes atores (compositores, coreógrafos e performers, músicos e dançarinos) numa determinada ecologia performativa? • RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso tem no trabalho artístico global? • RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam de um artefato interativo? Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA. Nesta framework, todos os atores (Actores) de uma dada ecologia estão conectados a todos os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre performance musical e a um estudo de caso em performance de dança. Estes estudos aju- daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes, instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec- nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre a distinção entre compor e projetar um novo instrumento num contexto que envolve uma multiplicidade de atores. Este trabalho apresenta as seguintes contribuições principais para o campo do design de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o conjunto de relações interconectadas em performances interativas, validado através de dois estudos de caso relacionados com a música, um estudo de caso relacionado com a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as necessidades dos vários atores e a sobreposição entre a composição musical e o design de tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram as ecologias performativas no NIME, com base numa análise sistemática da literatura dos artigos apresentados e publicados nestas conferência; 4) Proposta de uma aborda- gem metodológica combinando abordagens de design autobiográfico e idiográfico em performances interativas

Proceedings of the 19th Sound and Music Computing Conference

Proceedings of the 19th Sound and Music Computing Conference - June 5-12, 2022 - Saint-Étienne (France). https://smc22.grame.f

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

Tracing the Compositional Process. Sound art that rewrites its own past: formation, praxis and a computer framework

The domain of this thesis is electroacoustic computer-based music and sound art. It investigates a facet of composition which is often neglected or ill-defined: the process of composing itself and its embedding in time. Previous research mostly focused on instrumental composition or, when electronic music was included, the computer was treated as a tool which would eventually be subtracted from the equation. The aim was either to explain a resultant piece of music by reconstructing the intention of the composer, or to explain human creativity by building a model of the mind. Our aim instead is to understand composition as an irreducible unfolding of material traces which takes place in its own temporality. This understanding is formalised as a software framework that traces creation time as a version graph of transactions. The instantiation and manipulation of any musical structure implemented within this framework is thereby automatically stored in a database. Not only can it be queried ex post by an external researcher—providing a new quality for the empirical analysis of the activity of composing—but it is an integral part of the composition environment. Therefore it can recursively become a source for the ongoing composition and introduce new ways of aesthetic expression. The framework aims to unify creation and performance time, fixed and generative composition, human and algorithmic “writing”, a writing that includes indeterminate elements which condense as concurrent vertices in the version graph. The second major contribution is a critical epistemological discourse on the question of ob- servability and the function of observation. Our goal is to explore a new direction of artistic research which is characterised by a mixed methodology of theoretical writing, technological development and artistic practice. The form of the thesis is an exercise in becoming process-like itself, wherein the epistemic thing is generated by translating the gaps between these three levels. This is my idea of the new aesthetics: That through the operation of a re-entry one may establish a sort of process “form”, yielding works which go beyond a categorical either “sound-in-itself” or “conceptualism”. Exemplary processes are revealed by deconstructing a series of existing pieces, as well as through the successful application of the new framework in the creation of new pieces

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers

Virtual Reality

At present, the virtual reality has impact on information organization and management and even changes design principle of information systems, which will make it adapt to application requirements. The book aims to provide a broader perspective of virtual reality on development and application. First part of the book is named as "virtual reality visualization and vision" and includes new developments in virtual reality visualization of 3D scenarios, virtual reality and vision, high fidelity immersive virtual reality included tracking, rendering and display subsystems. The second part named as "virtual reality in robot technology" brings forth applications of virtual reality in remote rehabilitation robot-based rehabilitation evaluation method and multi-legged robot adaptive walking in unstructured terrains. The third part, named as "industrial and construction applications" is about the product design, space industry, building information modeling, construction and maintenance by virtual reality, and so on. And the last part, which is named as "culture and life of human" describes applications of culture life and multimedia-technology

Biohacking and code convergence : a transductive ethnography

Cette thèse se déploie dans un espace de discours et de pratiques revendicatrices, à l’inter- section des cultures amateures informatiques et biotechniques, euro-américaines contempo- raines. La problématique se dessinant dans ce croisement culturel examine des métaphores et analogies au coeur d’un traffic intense, au milieu de voies de commmunications imposantes, reliant les technologies informatiques et biotechniques comme lieux d’expression médiatique. L’examen retrace les lignes de force, les médiations expressives en ces lieux à travers leurs manifestations en tant que codes —à la fois informatiques et génétiques— et reconnaît les caractères analogiques d’expressivité des codes en tant que processus de convergence. Émergeant lentement, à partir des années 40 et 50, les visions convergentes des codes ont facilité l’entrée des ordinateurs personnels dans les marchés, ainsi que dans les garages de hackers, alors que des bricoleurs de l’informatique s’en réclamaient comme espace de liberté d’information —et surtout d’innovation. Plus de cinquante ans plus tard, l’analogie entre codes informatiques et génétiques sert de moteur aux revendications de liberté, informant cette fois les nouvelles applications de la biotechnologie de marché, ainsi que l’activité des biohackers, ces bricoleurs de garage en biologie synthétique. Les pratiques du biohacking sont ainsi comprises comme des individuations : des tentatives continues de résoudre des frictions, des tensions travaillant les revendications des cultures amateures informatiques et biotechniques. Une des manières de moduler ces tensions s’incarne dans un processus connu sous le nom de forking, entrevu ici comme l’expérience d’une bifurcation. Autrement dit, le forking est ici définit comme passage vers un seuil critique, déclinant la technologie et la biologie sur plusieurs modes. Le forking informe —c’est-à-dire permet et contraint— différentes vi- sions collectives de l’ouverture informationnelle. Le forking intervient aussi sur les plans des iii semio-matérialités et pouvoirs d’action investis dans les pratiques biotechniques et informa- tiques. Pris comme processus de co-constitution et de différentiation de l’action collective, les mouvements de bifurcation invitent les trois questions suivantes : 1) Comment le forking catalyse-t-il la solution des tensions participant aux revendications des pratiques du bioha- cking ? 2) Dans ce processus de solution, de quelles manières les revendications changent de phase, bifurquent et se transforment, parfois au point d’altérer radicalement ces pratiques ? 3) Quels nouveaux problèmes émergent de ces solutions ? L’effort de recherche a trouvé ces questions, ainsi que les plans correspondants d’action sémio-matérielle et collective, incarnées dans trois expériences ethnographiques réparties sur trois ans (2012-2015) : la première dans un laboratoire de biotechnologie communautaire new- yorkais, la seconde dans l’émergence d’un groupe de biotechnologie amateure à Montréal, et la troisième à Cork, en Irlande, au sein du premier accélérateur d’entreprises en biologie synthétique au monde. La logique de l’enquête n’est ni strictement inductive ou déductive, mais transductive. Elle emprunte à la philosophie de la communication et de l’information de Gilbert Simondon et découvre l’épistémologie en tant qu’acte de création opérant en milieux relationnels. L’heuristique transductive offre des rencontres inusitées entre les métaphores et les analogies des codes. Ces rencontres étonnantes ont aménagé l’expérience de la conver- gence des codes sous forme de jeux d’écritures. Elles se sont retrouvées dans la recherche ethnographique en tant que processus transductifs.This dissertation examines creative practices and discourses intersecting computer and biotech cultures. It queries influential metaphors and analogies on both sides of the inter- section, and their positioning of biotech and information technologies as expression media. It follows mediations across their incarnations as codes, both computational and biological, and situates their analogical expressivity and programmability as a process of code conver- gence. Converging visions of technological freedom facilitated the entrance of computers in 1960’s Western hobbyist hacker circles, as well as in consumer markets. Almost fifty years later, the analogy drives claims to freedom of information —and freedom of innovation— from biohacker hobbyist groups to new biotech consumer markets. Such biohacking practices are understood as individuations: as ongoing attempts to resolve frictions, tensions working through claims to freedom and openness animating software and biotech cultures. Tensions get modulated in many ways. One of them, otherwise known as “forking,” refers here to a critical bifurcation allowing for differing iterations of biotechnical and computa- tional configurations. Forking informs —that is, simultaneously affords and constrains— differing collective visions of openness. Forking also operates on the materiality and agency invested in biotechnical and computational practices. Taken as a significant process of co- constitution and differentiation in collective action, bifurcation invites the following three questions: 1) How does forking solve tensions working through claims to biotech freedom? 2) In this solving process, how can claims bifurcate and transform to the point of radically altering biotech practices? 3) what new problems do these solutions call into existence? This research found these questions, and both scales of material action and agency, in- carnated in three extensive ethnographical journeys spanning three years (2012-2015): the first in a Brooklyn-based biotech community laboratory, the second in the early days of a biotech community group in Montreal, and the third in the world’s first synthetic biology startup accelerator in Cork, Ireland. The inquiry’s guiding empirical logic is neither solely deductive or inductive, but transductive. It borrows from Gilbert Simondon’s philosophy of communication and information to experience epistemology as an act of analogical creation involving the radical, irreversible transformation of knower and known. Transductive heuris- tics offer unconvential encounters with practices, metaphors and analogies of code. In the end, transductive methods acknowledge code convergence as a metastable writing games, and ethnographical research itself as a transductive process

History of Computer Art, Second Edition

The development of the use of computers and software in art from the Fifties to the present is explained. As general aspects of the history of computer art an interface model and three dominant modes to use computational processes (generative, modular, hypertextual) are presented. The "History of Computer Art" features examples of early developments in media like cybernetic sculptures, computer graphics and animation (including music videos and demos), video and computer games, reactive installations, virtual reality, evolutionary art and net art. The functions of relevant art works are explained more detailed than usual in such histories. The second edition for the Book on Demand (Lulu Press, 2020) includes an update of chapter II.1.1 (first edition 2014)