Executing Hierarchical Interactive Scores in ReactiveML

National audienceInteractive scores proposes a model to write and execute interactive multimedia scores. The software i-score implements the above model using Hierarchical Time Stream Petri Nets (HTSPN). However, this model is very static and modelling new features would require a complete redesign of the network or sometimes they cannot be expressed. Another problem of i-score is that it does not provide a good visual feedback of the execution of the scenario. In this work, we define and implement an interpreter of interactive scores using the synchronous programming language ReactiveML. Our work takes advantage of the expressiveness of the reactive model and the power of functional programming to develop an interpreter more dynamic and simple. Contrary to the Petri Net model, our approach allows to model precisely the hierarchical behaviour, and permits the easy prototyping of new features. We also propose a visualization system using the environment INScore that provides a real-time visualization of the execution of the score

The arpeggigon: declarative programming of a full-fledged musical application

There are many systems and languages for music that essentially are declarative, often following the synchronous dataflow paradigm. As these tools, however, are mainly aimed at artists, their application focus tends to be narrow and their usefulness as general purpose tools for developing musical applications limited, at least if one desires to stay declarative. This paper demonstrates that Functional Reactive Programming (FRP) in combination with Reactive Values and Relations (RVR) is one way of addressing this gap. The former, in the synchronous dataflow tradition, aligns with the temporal and declarative nature of music, while the latter allows declarative interfacing with external components as needed for full-fledged musical applications. The paper is a case study around the development of an interactive cellular automaton for composing groove-based music. It illustrates the interplay between FRP and RVR as well as programming techniques and examples generally useful for musical, and other time-aware, interactive applications

Reactivity of Cooperative Systems: Application to ReactiveML -- extended version

Cooperative scheduling enables efficient sequential implementations of concurrency. It is widely used to provide lightweight threads facilities as libraries or programming constructs in many programming languages. However, it is up to programmers to actually cooperate to ensure the reactivity of their programs. We present a static analysis that checks the reactivity of programs by abstracting them into so-called behaviors using a type-and-effect system. Our objective is to find a good compromise between the complexity of the analysis and its precision for typical reactive programs. The simplicity of the analysis is mandatory for the programmer to be able to understand error messages and how to fix reactivity problems. Our work is applied and implemented in the functional synchronous language ReactiveML. It handles recursion, higher-order processes and first-class signals. We prove the soundness of our analysis with respect to the big-step semantics of the language: a well-typed program with reactive effects is reactive. The analysis is easy to implement and generic enough to be applicable to other models of concurrency such as coroutines. This research report is the extended version of the article published at the 21st International Static Analysis Symposium.L'ordonnancement coopératif permet l'implémentation séquentielle efficace de la concurrence. Il est largement utilisé pour fournir des threads légers sous forme de bibliothèques ou de constructions de programmation dans de nombreux langages de programmation. Toutefois, il appartient aux programmeurs d'appeler des primitives de coopération pour assurer la réactivité de leurs programmes. Nous présentons une analyse statique qui vérifie la réactivité des programmes en les abstrayant en comportements à l'aide d'un système de types à effets. Notre objectif est de trouver un bon compromis entre la complexité de l'analyse et sa précision pour les programmes réactifs typiques. La simplicité de l'analyse est obligatoire pour que le programmeur soit en mesure de comprendre les messages d'erreur et comment résoudre les problèmes de réactivité. Notre travail est appliqué et mis en oeuvre dans le langage synchrone fonctionnel ReactiveML. Il gère la récursivité, les processus d'ordre supérieur et les signaux de première classe. Nous prouvons la correction de notre analyse par rapport à la sémantique grands pas du langage~: un programme bien typé avec des effets réactifs est réactif. L'analyse est facile à mettre en oeuvre et assez générique pour être applicable à d'autres modèles de concurrence, tels que les coroutines. Ce rapport de recherche est la version étendue de l'article publié dans les actes de la 21ème édition de l'International Static Analysis Symposium

Session-based concurrency: between operational and declarative views

Communication-based software is ubiquitous nowadays. From e-banking to e-shopping, online activities often involve message exchanges between software components. These interactions are often governed by protocols that explicitly describe the sequences of communication actions that should be executed by each component. Crucially, these protocols are not isolated from a program’s context: external conditions such as timing constraints or exceptional events that occur during execution can affect message exchanges. As an additional difficulty, individual components are typically developed in different programming languages. In this setting, certifying that a program conforms to its intended protocols is challenging. A widely studied program verification technique uses behavioral type systems, which exploit abstract representations of these protocols to check that the program executes communication actions as intended. Unfortunately, the abstractions offered by behavioral type systems may neglect the influence that external conditions have on the program. This thesis addresses this issue by considering programming languages with declarative features, in which the governing conditions of the program can be adequately described. Our work develops correct translations between programming languages to show that languages with declarative features can indeed articulate a unified view of communication-based programs. Specifically, these translations demonstrate that the operational features of communication-based programs can be correctly represented by languages with declarative features. An additional contribution is a hybrid language that combines the best of both worlds, enabling the analysis of operational and declarative features in communication-based programs

Funky grooves: declarative programming of full-fledged musical applications

There are many systems and languages for music that essentially are declarative, often following the synchronous dataflow paradigm. As these tools, however, are mainly aimed at artists, their application focus tends to be narrow and their usefulness as general purpose tools for developing musical applications limited, at least if one desires to stay declarative. This paper demonstrates that Functional Reactive Programming (FRP) in combination with Reactive Values and Relations (RVR) is one way of addressing this gap. The former, in the synchronous dataflow tradition, aligns with the temporal and declarative nature of music, while the latter allows declarative interfacing with external components as needed for full-fledged musical applications. The paper is a case study around the development of an interactive cellular automaton for composing groove-based music

Multimedia scheduling for interactive multimedia systems

International audienceScheduling for real-time interactive multimedia systems (IMS) raises specific challenges that require particular attention. Examples are triggering and coordination of heterogeneous tasks, especially for IMS that use a physical time, but also a musical time that depends on a particular performance, and how tasks that deal with audio processing interact with control tasks. Moreover , IMS have to ensure a timed scenario, for instance specified in an augmented musical score, and current IMS do not deal with their reliability and predictability. We present how to formally interleave audio processing with control by using buffer types that represent audio buffers and the way of interrupting computations that occur on them, and how to check the property of time-safety of IMS timed scenarios, in particular augmented scores for the IMS Antescofo for automatic accompaniment developed at Ircam. Our approach is based on the extension of an intermediate representation similar to the E code of the real-time embedded programming language Giotto, and on static analysis procedures run on the graph of the intermediate representation

A Dynamic Timed-Language for Computer-Human Musical Interaction

In this paper, we present the programing of time and interaction in Antescofo, a real-time system for performance coordination between musicians and computer processes during live music performance. To this end, Antescofo relies on artificial machine listening and a domain specific real-time programing language. It extends each paradigm through strong coupling of the two and strong emphasis on temporal semantics and behavior of the system. The challenge in bringing human actions in the loop of computing is strongly related to temporal semantics of the language, and timeliness of live execution despite heterogeneous nature of time in the two mediums. Interaction scenarii are expressed at a symbolic level through the management of musical time (i.e. events like notes or beats in relative tempi) and of the 'physical' time (with relationships like succession, delay, duration, speed). Antescofo unique features are presented through a series of paradigmatic program samples which illustrate how to manage execution of different audio processes through time and their interactions with an external environment. The Antescofo approach has been validated through numerous uses of the system in live electronic performances in contemporary music repertoire by various international music ensembles.Ce papier présente le système temps-réel Antescofo et son langage dédié. Celui-ci permet de décrire des scénarios temporels où des processus de musique électronique sont calculés et ordonnancés en fonction du jeu d'un musicien. Pour ce faire, Antescofo couple un système de suivi de partition avec un module réactif. L'originalité du système réside dans la sémantique temporelle du langage adaptée aux caractéristiques critiques de l'interaction musicale. Le temps et les événements peuvent s'exprimés de façon symbolique dans une échelle absolue (en seconde) ou des échelles relatives à des tempos. Nous aborderons les caractéristiques du langage et de la partie réactive d'Antescofo illustrées à travers une série d'exemples paradigmatiques

OSSIA: Towards a unified interface for scoring time and interaction

International audienceThe theory of interactive scores addresses the writing and execution of temporal constraints between musical objects, with the ability to describe the use of interactivity in the scores. In this paper, a notation for the use of conditional branching in interactive scores will be introduced. It is based on a high level formalism for the authoring of interactive scores developed during the course of the OS-SIA research project. This formalism is meant to be at the same time easily manipulated by composers, and translat-able to multiple formal methods used in interactive scores like Petri nets and timed automaton. An application programming interface that allows the interactive scores to be embedded in other software and the authoring software, I-SCORE, will be presented

A visual framework for dynamic mixed music notation

International audienceWe present a visual notation framework for real-time, score-based computer music where human musicians play together with electronic processes, mediated by the Antescofo reactive software. This framework approaches the composition and performance of mixed music by displaying several perspectives on the score's contents. Our particular focus is on dynamic computer actions, whose parameters are calculated at run-time. For their visualization, we introduce four models: an extended action view, a staff-based simulation trace, a tree-based hierarchical display of the score code, and an out-of-time inspector panel. Each model is illustrated in code samples and case studies from actual scores. We argue the benefits of a multifaceted visual language for mixed music, and for the relevance of our proposed models towards reaching this goal