Screen-based musical instruments as semiotic machines

The ixi software project started in 2000 with the intention to explore new interactive patterns and virtual interfaces in computer music software. The aim of this paper is not to describe these programs, as they have been described elsewhere, but rather explicate the theoretical background that underlies the design of these screen-based instruments. After an analysis of the similarities and differences in the design of acoustic and screen-based instruments, the paper describes how the creation of an interface is essentially the creation of a semiotic system that affects and influences the musician and the composer. Finally the terminology of this semiotics is explained as an interaction model

Free associative composition: Practice led research into composition techniques that help enable free association.

The original compositions presented in this portfolio are the product of practice led research into developing and implementing composition techniques that enable free association. This com-mentary outlines the different approaches I have taken and the reasoning behind them

Introduction to Gestural Similarity in Music. An Application of Category Theory to the Orchestra

Mathematics, and more generally computational sciences, intervene in several aspects of music. Mathematics describes the acoustics of the sounds giving formal tools to physics, and the matter of music itself in terms of compositional structures and strategies. Mathematics can also be applied to the entire making of music, from the score to the performance, connecting compositional structures to acoustical reality of sounds. Moreover, the precise concept of gesture has a decisive role in understanding musical performance. In this paper, we apply some concepts of category theory to compare gestures of orchestral musicians, and to investigate the relationship between orchestra and conductor, as well as between listeners and conductor/orchestra. To this aim, we will introduce the concept of gestural similarity. The mathematical tools used can be applied to gesture classification, and to interdisciplinary comparisons between music and visual arts.Comment: The final version of this paper has been published by the Journal of Mathematics and Musi

Visual arts and 2D – 3D motion typo.graphic design

This project explores issues of typographic design within multimedia contexts, which include interactive, sonic and animated components

A Review of the "Digital Turn" in the New Literacy Studies

Digital communication has transformed literacy practices and assumed great importance in the functioning of workplace, recreational, and community contexts. This article reviews a decade of empirical work of the New Literacy Studies, identifying the shift toward research of digital literacy applications. The article engages with the central theoretical, methodological, and pragmatic challenges in the tradition of New Literacy Studies, while highlighting the distinctive trends in the digital strand. It identifies common patterns across new literacy practices through cross-comparisons of ethnographic research in digital media environments. It examines ways in which this research is taking into account power and pedagogy in normative contexts of literacy learning using the new media. Recommendations are given to strengthen the links between New Literacy Studies research and literacy curriculum, assessment, and accountability in the 21st century

Intersemiotic translation and transformational creativity

In this article we approach a case of intersemiotic translation as a paradigmatic example of Boden’s ‘transformational creativity’ category. To develop our argument, we consider Boden’s fundamental notion of ‘conceptual space’ as a regular pattern of semiotic action, or ‘habit’ (sensu Peirce). We exemplify with Gertrude Stein’s intersemiotic translation of Cézanne and Picasso’s proto-cubist and cubist paintings. The results of Stein’s IT transform the conceptual space of modern literature, constraining it towards new patterns of semiosis. Our association of Boden’s framework to describe a cognitive creative phenomenon with a philosophically robust theory of meaning results in a cognitive semiotic account of IT

The Art of Engaging: Implications for Computer Music Systems

The art of engaging with computer music systems is multifaceted. This paper will provide an overview of the issues of interface between musician and computer, cognitive aspects of engagement as involvement, and metaphysical understandings of engagement as proximity. Finally, this paper will examine implications for the design of computer music systems when these issues are taken into account

Deconstructing Speech: new tools for speech manipulation

My research at the London College of Communication is concerned with archives of recorded speech, what new tools need to be devised for its manipulation and how to go about this process of invention. Research into available forms of analysis of speech is discussed below with regard to two specific areas, feature vectors from linear predictive coding (LPC) analysis and hidden Markov-model-based automatic speech recognition (ASR) systems. These are discussed in order to demonstrate that whilst aspects of each may be useful in devising a system of speech-archive manipulation for artistic use. Their drawbacks and deficiencies for use in art – consequent of the reasons for their invention – necessitate the creation of tools with artistic, rather than engineering agendas in mind. It is through the initial process of devising conceptual tools for understanding speech as sound objects that I have been confronted with issues of semiotics and semantics of the voice and of the relationship between sound and meaning in speech, and of the role of analysis in mediating existing methods of communication. This is discussed with reference to Jean-Jacques Nattiez’s Music and Discourse: Towards a Semiology of Music (Nattiez 1987). The ‘trace’ – a neutral level of semiotic analysis proposed by Nattiez, far from being hypothetical as suggested by Hatten (1992: 88–98) and others, is present by analogy to many forms of mediation in modern spoken communication and the reproduction of music, and it is precisely this neutrality with regards to meaning that tools for manipulation of speech must possess, since the relationships between the sound of speech and its meaning are ‘intense’ (after Deleuze 1968