Screen-based musical instruments as semiotic machines

The ixi software project started in 2000 with the intention to explore new interactive patterns and virtual interfaces in computer music software. The aim of this paper is not to describe these programs, as they have been described elsewhere, but rather explicate the theoretical background that underlies the design of these screen-based instruments. After an analysis of the similarities and differences in the design of acoustic and screen-based instruments, the paper describes how the creation of an interface is essentially the creation of a semiotic system that affects and influences the musician and the composer. Finally the terminology of this semiotics is explained as an interaction model

Sensing and mapping for interactive performance

This paper describes a trans-domain mapping (TDM) framework for translating meaningful activities from one creative domain onto another. The multi-disciplinary framework is designed to facilitate an intuitive and non-intrusive interactive multimedia performance interface that offers the users or performers real-time control of multimedia events using their physical movements. It is intended to be a highly dynamic real-time performance tool, sensing and tracking activities and changes, in order to provide interactive multimedia performances. From a straightforward definition of the TDM framework, this paper reports several implementations and multi-disciplinary collaborative projects using the proposed framework, including a motion and colour-sensitive system, a sensor-based system for triggering musical events, and a distributed multimedia server for audio mapping of a real-time face tracker, and discusses different aspects of mapping strategies in their context. Plausible future directions, developments and exploration with the proposed framework, including stage augmenta tion, virtual and augmented reality, which involve sensing and mapping of physical and non-physical changes onto multimedia control events, are discussed

Digital 3D modelling - Introduction

The transformation of the sculptor from a performer with, e.g. clay, to someone whotrains computers to process data for new 3D images, or simply asks the computer to produce a 3D model, is naturally altering the perception of what it means to make visual art and who makes such art.In a teaching context, an important question concerns what should be taught in the visual arts and how the increasing digital entanglement in all practices should be handled in the various disciplines of the teaching subject

Serious Games in Cultural Heritage

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Processpatching: defining new methods in aRt&D

In the context of a rapidly changing domain of contemporary electronic art practice- where the speed of technological innovation and the topicality of art 'process as research' methods are both under constant revision- the process of collaboration between art, computer science and engineering is an important addition to existing 'R&D'. Scholarly as well as practical exploration of artistic methods, viewed in relation to the field of new technology, can be seen to enable and foster innovation in both the conceptualisation and practice of the electronic arts. At the same time, citing new media art in the context of technological innovation brings a mix of scientific and engineering issues to the fore and thereby demands an extended functionality that may lead to R&D, as technology attempts to take account of aesthetic and social considerations in its re-development. This new field of new media or electronic art R&D is different from research and development aimed at practical applications of new technologies as we see them in everyday life. A next step for Research and Development in Art (aRt&D) is a formalisation of the associated work methods, as an essential ingredient for interdisciplinary collaboration. This study investigates how electronic art patches together processes and methods from the arts, engineering and computer science environments. It provides a framework describing the electronic art methods to improve collaboration by informing others about one's artistic research and development approach. This investigation is positioned in the electronic art laboratory where new alliances with other disciplines are established. It provides information about the practical and theoretical aspects of the research and development processes of artists. The investigation addresses fundamental questions about the 'research and development methods' (discussed and defined at length in these pages), of artists who are involved in interdisciplinary collaborations amongst and between the fields of Art, Computer Science, and Engineering. The breadth of the fields studied necessarily forced a tight focus on specific issues in the literature, addressed herein through a series of focused case studies which demonstrate the points of synergy and divergence between the fields of artistic research and development, in a wider art&D' context. The artistic methods proposed in this research include references from a broad set of fields (e. g. Technology, Media Arts, Theatre and Performance, Systems Theories, the Humanities, and Design Practice) relevant to and intrinsically intertwined with this project and its placement in an interdisciplinary knowledge domain. The aRt&D Matrix provides a complete overview of the observed research and development methods in electronic arts, including references to related disciplines and methods from other fields. The new Matrix developed and offered in this thesis also provides an instrument for analysing the interdisciplinary collaboration process that exclusively reflects the information we need for the overview of the team constellation. The tool is used to inform the collaborators about the backgrounds of the other participants and thus about the expected methods and approaches. It provides a map of the bodies of knowledge and expertise represented in any given cross-disciplinary team, and thus aims to lay the groundwork for a future aRt&D framework of use to future scholars and practitioners alike

Generating Rembrandt: Artificial Intelligence, Copyright, and Accountability in the 3A Era--The Human-like Authors are Already Here- A New Model

Artificial intelligence (AI) systems are creative, unpredictable, independent, autonomous, rational, evolving, capable of data collection, communicative, efficient, accurate, and have free choice among alternatives. Similar to humans, AI systems can autonomously create and generate creative works. The use of AI systems in the production of works, either for personal or manufacturing purposes, has become common in the 3A era of automated, autonomous, and advanced technology. Despite this progress, there is a deep and common concern in modern society that AI technology will become uncontrollable. There is therefore a call for social and legal tools for controlling AI systems’ functions and outcomes. This Article addresses the questions of the copyrightability of artworks generated by AI systems: ownership and accountability. The Article debates who should enjoy the benefits of copyright protection and who should be responsible for the infringement of rights and damages caused by AI systems that independently produce creative works. Subsequently, this Article presents the AI Multi- Player paradigm, arguing against the imposition of these rights and responsibilities on the AI systems themselves or on the different stakeholders, mainly the programmers who develop such systems. Most importantly, this Article proposes the adoption of a new model of accountability for works generated by AI systems: the AI Work Made for Hire (WMFH) model, which views the AI system as a creative employee or independent contractor of the user. Under this proposed model, ownership, control, and responsibility would be imposed on the humans or legal entities that use AI systems and enjoy its benefits. This model accurately reflects the human-like features of AI systems; it is justified by the theories behind copyright protection; and it serves as a practical solution to assuage the fears behind AI systems. In addition, this model unveils the powers behind the operation of AI systems; hence, it efficiently imposes accountability on clearly identifiable persons or legal entities. Since AI systems are copyrightable algorithms, this Article reflects on the accountability for AI systems in other legal regimes, such as tort or criminal law and in various industries using these systems