Robotic arts: Current practices, potentials, and implications

Given that the origin of the “robot” comes from efforts to create a worker to help people, there has been relatively little research on making a robot for non-work purposes. However, some researchers have explored robotic arts since Leonardo da Vinci. Many questions can be posed regarding the potentials of robotic arts: (1) Is there anything we can call machine-creativity? (2) Can robots improvise artworks on the fly? and (3) Can art robots pass the Turing test? To ponder these questions and see the current status quo of robotic arts, the present paper surveys the contributions of robotics in diverse forms of arts, including drawing, theater, music, and dance. The present paper describes selective projects in each genre, core procedure, possibilities and limitations within the aesthetic computing framework. Then, the paper discusses implications of these robotic arts in terms of both robot research and art research, followed by conclusions including answers to the questions posed at the outset

Interactive intelligence: behaviour-based AI, musical HCI and the Turing Test

The field of behaviour-based artificial intelligence (AI), with its roots in the robotics research of Rodney Brooks, is not predominantly tied to linguistic interaction in the sense of the classic Turing test (or, "imitation game"). Yet, it is worth noting, both are centred on a behavioural model of intelligence. Similarly, there is no intrinsic connection between musical AI and the language-based Turing test, though there have been many attempts to forge connections between them. Nonetheless, there are aspects of musical AI and the Turing test that can be considered in the context of non-language-based interactive environments–-in particular, when dealing with real-time musical AI, especially interactive improvisation software. This paper draws out the threads of intentional agency and human indistinguishability from Turing’s original 1950 characterisation of AI. On the basis of this distinction, it considers different approaches to musical AI. In doing so, it highlights possibilities for non-hierarchical interplay between human and computer agents

Little Drummer Bot: Building, Testing, and Interfacing With a New Expressive Mechatronic Drum System

(Abstract to follow

AVISARME: Audio Visual Synchronization Algorithm for a Robotic Musician Ensemble

This thesis presents a beat detection algorithm which combines both audio and visual inputs to synchronize a robotic musician to its human counterpart. Although there has been considerable work done to create sophisticated methods for audio beat detection, the visual aspect of musicianship has been largely ignored. With advancements in image processing techniques, as well as both computer and imaging technologies, it has recently become feasible to integrate visual inputs into beat detection algorithms. Additionally, the proposed method for audio tempo detection also attempts to solve many issues that are present in current algorithms. Current audio-only algorithms have imperfections, whether they are inaccurate, too computationally expensive, or suffer from terrible resolution. Through further experimental testing on both a popular music database and simulated music signals, the proposed algorithm performed statistically better in both accuracy and robustness than the baseline approaches. Furthermore, the proposed approach is extremely efficient, taking only 45ms to compute on a 2.5s signal, and maintains an extremely high temporal resolution of 0.125 BPM. The visual integration also relies on Full Scene Tracking, allowing it to be utilized for live beat detection for practically all musicians and instruments. Numerous optimization techniques have been implemented, such as pyramidal optimization (PO) and clustering techniques which are presented in this thesis. A Temporal Difference Learning approach to sensor fusion and beat synchronization is also proposed and tested thoroughly. This TD learning algorithm implements a novel policy switching criterion which provides a stable, yet quickly reacting estimation of tempo. The proposed algorithm has been implemented and tested on a robotic drummer to verify the validity of the approach. The results from testing are documented in great detail and compared with previously proposed approaches

Embodied responses to musical experience detected by human bio-feedback brain features in a geminoid augmented architecture

This paper presents the conceptual framework for a study of musical experience and the associated architecture centred on Human-Humanoid Interaction (HHI). On the grounds of the theoretical and experimental literature on the biological foundation of music, the grammar of music perception and the perception and feeling of emotions in music hearing, we argue that music cognition is specific and that it is realized by a cognitive capacity for music that consists of conceptual and affective constituents. We discuss the relationship between such constituents that enables understanding, that is extracting meaning from music at the different levels of the organization of sounds that are felt as bearers of affects and emotions. To account for the way such cognitive mechanisms are realized in music hearing and extended to movements and gestures we bring in the construct of tensions and of music experience as a cognitive frame. Finally, we describe the principled approach to the design and the architecture of a BCI-controlled robotic system that can be employed to map and specify the constituents of the cognitive capacity for music as well as to simulate their contribution to music meaning understanding in the context of music experience by displaying it through the Geminoid robot movements

2011 December

Press releases for December of 2011

Musicians (Don't) Play Algorithms. Or: What makes a musical performance

Our private perception of listening to an individualized playlist during a jog is very different from the interaction we might experience at a live concert. We do realize that music is not necessarily a performing art, such as dancing or theater, while our demands regarding musical performances are conflicting: We expect perfect sound quality and the thrill of the immediate. We want the artist to overwhelm us with her virtuosity and we want her to struggle, just like a human. We want to engage with the musical expression and rely on visual and physical cues. Considering that the ears of today’s listeners are used to technologically mediated music, in this paper I explore the unique qualities of musical live performances and examine if our conception allows for new mechatronic inventions, in particular robotic musicians, to participate in this art form. Some of Godlovitch’s main thoughts expounded in his work on “musical performance” [11] serve as a reference and starting point for this investigation. His concept of ‘personalism’, which deprives computer-/program-based musical performances from expressive potential and creative accomplishment is an issue that I want to challenge by pointing out new approaches arising from a reflective discourse on technology, embodiment and expression. The enquiry conducted illustrates, how in reasoning about machine performers and algorithmic realization of music, we also examine the perceptual, physical and social aspects of human musicianship, reconceptualizing our understanding of a musical live performance

ELAIA 2018

Over the years, the Program has continued to grow and flourish, and the depth of its research continues to increase. This inaugural journal represents the fruits of that development, containing capstone research projects from the 2018 Honors Program senior class and their faculty mentors. The Table of Contents is diverse, and in that way it is a crystal clear reflection of our program’s community of scholars. I, along with the members of the Honors Council, am gratified by the work of each student and faculty mentor printed within these pages. Congratulations, everyone! - Stephen Lowe, Honors Program Directo