Design and analysis of target-sensitive real-time systems

A significant number of real-time control applications include computational activities where the results have to be delivered at precise instants, rather than within a deadline. The performance of such systems significantly degrades if outputs are generated before or after the desired target time. This work presents a general methodology that can be used to design and analyze target-sensitive applications in which the timing parameters of the computational activities are tightly coupled with the physical characteristics of the system to be controlled. For the sake of clarity, the proposed methodology is illustrated through a sample case study used to show how to derive and verify real-time constraints from the mission requirements. Software implementation issues necessary to map the computational activities into tasks running on a real-time kernel are also discussed to identify the kernel mechanisms necessary to enforce timing constraints and analyze the feasibility of the application. A set of experiments are finally presented with the purpose of validating the proposed methodology

Low-Latency Music Software Using Off-The-Shelf Operating Systems

Operating systems are often the limiting factor in creating low-latency interactive computer music systems. Real-time music applications require operating system support for memory management, process scheduling, media I/O, and general development, including debugging. We present performance measurements for some current operating systems, including NT4, Windows95, and Irix 6.4. While Irix was found to give rather good real-time performance, NT4 and Windows95 suffer from both process scheduling delays and high audio output latency. The addition of WDM Streaming to NT and Windows offers some promise of lower latency, but WDM Streaming may actually make performance worse by circumventing priority-based scheduling

A distributed approach to surround sound production

The requirement for multi-channel surround sound in audio production applications is growing rapidly. Audio processing in these applications can be costly, particularly in multi-channel systems. A distributed approach is proposed for the development of a realtime spatialization system for surround sound music production, using Ambisonic surround sound methods. The latency in the system is analyzed, with a focus on the audio processing and network delays, in order to ascertain the feasibility of an enhanced, distributed real-time spatialization system

A distributed approach to surround sound production

The requirement for multi-channel surround sound in audio production applications is growing rapidly. Audio processing in these applications can be costly, particularly in multi-channel systems. A distributed approach is proposed for the development of a realtime spatialization system for surround sound music production, using Ambisonic surround sound methods. The latency in the system is analyzed, with a focus on the audio processing and network delays, in order to ascertain the feasibility of an enhanced, distributed real-time spatialization system

Improvisation, Computers, and Interaction : Rethinking Human-Computer Interaction Through Music

Interaction is an integral part of all music. Interaction is part of listening, of playing, of composing and even of thinking about music. In this thesis the multiplicity of modes in which one may engage interactively in, through and with music is the starting point for rethinking Human-Computer Interaction in general and Interactive Music in particular. I propose that in Human-Computer interaction the methodology of control, interaction-as-control, in certain cases should be given up in favor for a more dynamic and reciprocal mode of interaction, interaction-as-difference: Interaction as an activity concerned with inducing differences that make a difference. Interaction-as-difference suggests a kind of parallelity rather than click-and-response. In essence, the movement from control to difference was a result of rediscovering the power of improvisation as a method for organizing and constructing musical content and is not to be understood as an opposition: It is rather a broadening of the more common paradigm of direct manipulation in Human-Computer Interaction. Improvisation is at the heart of all the sub-projects included in this thesis, also, in fact, in those that are not immediately related to music but more geared towards computation. Trusting the self-organizing aspect of musical improvisation, and allowing it to diffuse into other areas of my practice, constitutes the pivotal change that has radically influenced my artistic practice. Furthermore, is the work-in-movement (re-)introduced as a work kind that encompasses radically open works. The work-in-movement, presented and exemplified by a piece for guitar and computer, requires different modes of representation as the traditional musical score is too restrictive and is not able to communicate that which is the most central aspect: the collaboration, negotiation and interaction. The Integra framework and the relational database model with its corresponding XML representation is proposed as a means to produce annotated scores that carry past performances and version with it. The common nominator, the prerequisite, for interaction-as-difference and a improvisatory and self-organizing attitude towards musical practice it the notion of giving up of the Self. Only if the Self is able and willing to accept the loss the priority of interpretation (as for the composer) or the faithfulness to ideology or idiomatics (performer). Only is one is willing to forget is interaction-as-difference made possible. Among the artistic works that have been produced as part of this inquiry are some experimental tools in the form of computer software to support the proposed concepts of interactivity. These, along with the more traditional musical work make up both the object and the method in this PhD project. These sub-projects contained within the frame of the thesis, some (most) of which are still works-in-progress, are used to make inquiries into the larger question of the significance of interaction in the context of artistic practice involving computers

Low Latency Audio Processing

PhDLatency in the live audio processing chain has become a concern for audio engineers and system designers because significant delays can be perceived and may affect synchronisation of signals, limit interactivity, degrade sound quality and cause acoustic feedback. In recent years, latency problems have become more severe since audio processing has become digitised, high-resolution ADCs and DACs are used, complex processing is performed, and data communication networks are used for audio signal transmission in conjunction with other traffic types. In many live audio applications, latency thresholds are bounded by human perceptions. The applications such as music ensembles and live monitoring require low delay and predictable latency. Current digital audio systems either have difficulties to achieve or have to trade-off latency with other important audio processing functionalities. This thesis investigated the fundamental causes of the latency in a modern digital audio processing system: group delay, buffering delay, and physical propagation delay and their associated system components. By studying the time-critical path of a general audio system, we focus on three main functional blocks that have the significant impact on overall latency; the high-resolution digital filters in sigma-delta based ADC/DAC, the operating system to process low latency audio streams, and the audio networking to transmit audio with flexibility and convergence. In this work, we formed new theory and methods to reduce latency and accurately predict latency for group delay. We proposed new scheduling algorithms for the operating system that is suitable for low latency audio processing. We designed a new system architecture and new protocols to produce deterministic networking components that can contribute the overall timing assurance and predictability of live audio processing. The results are validated by simulations and experimental tests. Also, this bottom-up approach is aligned with the methodology that could solve the timing problem of general cyber-physical systems that require the integration of communication, software and human interactions