INTERACTIVE PHYSICAL DESIGN AND HAPTIC PLAYING OF VIRTUAL MUSICAL INSTRUMENTS

International audienceIn Computer Music, a practical approach of many Digital Musical Instruments is to separate the gestural input stage from the sound synthesis stage. While these instruments offer many creative possibilities, they present a strong rupture with traditional acoustic instruments, as the physical coupling between human and sound is broken. This coupling plays a crucial role for the expressive musical playing of acoustic instruments; we believe restoring it in a digital context is of equal importance for revealing the full expressive potential of digital instruments. This paper first presents haptic and physical modelling technologies for representing the mechano-acoustical instrumental situation in the context of DMIs. From these technologies, a prototype environment has been implemented for both designing virtual musical instruments and interacting with them via a force feedback device, able to preserve the energetic coherency of the musician-sound chain

Understanding and Tuning Mass-Interaction Networks Through Their Modal Representation

International audienceSound is all about vibration, and the GENESIS environment provides an efficient way for modeling and simulating complex vibrating structures, enabling to produce rich sounds. In this paper, we propose an overview of tools recently developed and available within the GENESIS environment, allowing a better understanding on how mass-interaction networks behave and introducing some enhanced tuning of their vibrating properties. All these tools try to address an inherent need of any creative process either in the physical world or in GENESIS, which is to create bidirectional connections between properties of a phenomenon, in our case, audible sounds, and properties of what produced it, here, mass-interaction networks. For this purpose, we will introduce the topological and modal representations of such mass-interaction networks and appreciate how relevant it can be to switch between these different representations to really apprehend its inner properties and those of the sounds it produces

Physical Modelling Concepts for a Collection of Multisensory Virtual Musical Instruments

International audienceThis paper discusses how haptic devices and physical modelling can be employed to design and simulate multisensory virtual musical instruments, providing the musician with joint audio, visual and haptic feedback. After briefly reviewing some of the main use-cases of haptics in Computer Music, we present GENESIS-RT, a software and hardware platform dedicated to the design and real-time haptic playing of virtual musical instruments using mass-interaction physical modelling. We discuss our approach and report on advancements in modelling various instrument categories, including physical models of percussion, plucked and bowed instruments. Finally, we comment on the constraints, challenges and new possibilities opened by modelling haptic virtual instruments with our platform, and discuss common points and differences in regards to classical Digital Musical Instruments

A Virtual Reality Platform for Musical Creation

International audienceVirtual reality aims at interacting with a computer in a similar form to interacting with an object of the real world. This research presents a VR platform allowing the user (1) to interactively create physically-based musical instruments and sounding objects, (2) play them in real time by using multisensory interaction by ways of haptics, 3D visualisation during playing, and real time physically-based sound synthesis. So doing, our system presents the two main properties expected in VR systems: the possibility of designing any type of objects and manipulating them in a multisensory real time fashion. By presenting our environment, we discuss the scientific underlying questions: (1) concerning the real time simulation, the way to manage simultaneous audio-haptic-visual cooperation during the real time multisensory simulations and (2) the Computer Aided Design functionalities for the creation of new physically-based musical instruments and sounding objects

Tangibility, Presence, Materiality, Reality in Artistic Creation with Digital Technology

International audienceThe democratization of Computer Arts and Computer Music has, due to dematerialization (virtualization) consequence of digital technologies, considerably widened the boundaries of creativity. As we are now entering a second phase that has been labeled “post-digital”, we are called to reconcile this openness with notions such as embodiment, presence, enaction and tangibility. These notions are in our view inherently linked to creativity. Here we outline some approaches to this problem under development within the “European Art-Science-Technology Network” (EASTN). Several areas of artistic creation are represented (Music, Animation, Multi-sensory Arts, Architecture, Fine Arts, Graphic communication, etc.). A main objective of this network is to establish common grounds through collaborative reflection and work on the above notions, using the concept of tangibility as a focal point. In this paper we describe several different approaches to the tangibility, in relation to concepts such as reality, materiality, objectivity, presence, concreteness, etc. and their antonyms. Our objective is to open a debate on tangibility, in the belief that it has a strong unifying potential but is also at the same time presents challenging and difficult to define. Here we present some initial thoughts on this topic in a first effort to bring together the approaches that arise from the different practices and projects developed within the partner institutions involved in the EASTN network

Technologies logicielles de la modélisation, de la simulation et de l'interaction multisensorielles instrumentales pour la création

The presented works have firstly questioned the possible inter-relations between physics-based modelling and simulation on the one hand, and creation and creativity on the other. More specifically, the possibility and fundamental interests of rooting creative activities onto physics-based modelling have been considered.This theme was approached from a Computer Sciences perspective, through the invention and the design of a new family of software environments aimed at creation with masses-interactions physics-based networks. In particular: GENESIS for musical creation, and MIMESIS for movement and computer graphics animation.These software environments empower the artist with innovative and complete creation processes, centred on physical modelling activities.Each and all of these environments are characterized by a specific “software environment paradigm”: dedicated features, software architecture specificities, adequate and necessary human-computer interaction means (interfaces and languages), and particular usage protocols (in our case: modelling and creative processes).These elements have been situated in the scientific and technological history of computer creation tools, and in the state of the art of the various concerned scientific domains. This work has in particular led to analysing the properties of masses-interactions physical networks (modularity, generic potential, calculability, usability, …), in relation with other categories of algorithms, models and formalisms used nowadays to address human sensori-motor skills. It also accompanied the demonstration that the building of physical models by an artist can be deeply relevant. This relevance lies in the qualities of the phenomena that are finally generated, but also, which is more original, and probably more essential, in the modelling/creation processes themselves.Finally, the works have been extended to various other axes: contributions to synchronous real-time simulation architecture and software for multisensory interaction, including force-feedback, sound, and image, by means of physics-based simulation; experimental modelling works; contributions to the evolutions of the formal basis of the physics networks paradigm; contributions to graphical representation of punctual movements, etc.These past works and the achieved results altogether lead to introduce new research perspectives, which we name software technologies of multisensory instrumental interaction, with, as a far end view, the foundation of a new “science of the multisensory computer” and the possibility of a “multisensory art”.Les travaux présentés ont été pour l’essentiel traversés par un questionnement sur les relations liant modélisation et simulation physiques et création ou, plus spécifiquement, sur la possibilité et l’intérêt fondamental d’appuyer une activité de création sur la modélisation. Ce thème a été abordé sous l’angle de l’informatique appliquée (en anglais, les « computer sciences »), au moyen de l’invention et de la réalisation d’une nouvelle famille d’environnements logiciels pour la création avec les réseaux physiques masses-interactions, dont en particulier les modeleurs-simulateurs GENESIS pour la création musicale, puis MIMESIS pour la création du mouvement et l’image animée. Ces environnements permettent à l’artiste d’organiser des processus de création complets et novateurs, fondés sur l’activité de modélisation physique.Ils sont chacun, et dans leur ensemble, caractérisés par un paradigme d’environnement logiciel nouveau : des fonctionnalités, des caractéristiques en matière d’architecture logicielle, des ergonomies nécessaires, adéquates et spécifiques, ainsi que des protocoles d’utilisation – ici des processus de modélisation et de création – qui leur sont propres. Ces propriétés ont été situées dans l’histoire scientifique et technologique des outils de création et dans les états de l’art des communautés scientifiques concernées. Ce travail a notamment donné lieu à une étude des propriétés des réseaux physiques masses-interactions (généricité, modularité, calculabilité, utilisabilité, généralité...) en vis-à-vis des catégories d’algorithmes, modèles et formalismes utilisés pour adresser la sensori-motricité humaine. Il s’est de plus accompagné d’une analyse du fait que l’activité de construction de modèles physiques peut présenter une pertinence profonde pour l’artiste – pertinence qui se joue non seulement dans les qualités des phénomènes générés, mais encore, de façon plus originale et peut-être plus essentielle, dans le processus de modélisation/création lui-même.Enfin, les travaux se sont étendus à plusieurs autres axes d’études : contributions aux logiciels de simulation temps réel pour l’interaction multisensorielle incluant le retour d’effort, le son et l’image, au moyen de la simulation physique ; travaux expérimentaux en modélisation ; contributions à l’amélioration du cadre formel des réseaux physiques masses-interactions ; contributions aux procédés de mise en forme visuelle de mouvements ponctuels ; travaux de positionnement théorique ; etc.L’expérience et les résultats acquis conduisent à introduire de nouvelles perspectives de recherche, sous le fil conducteur des technologies logicielles de l’interaction multisensorielle instrumentale, avec en ligne de mire la fondation d’une « science de l’ordinateur multisensoriel » et la possibilité d’un « art multisensoriel »