Enhancing stroke generation and expressivity in robotic drummers - A generative physics model approach

The goal of this master's thesis research is to enhance the stroke generation capabilities and musical expressivity in robotic drummers. The approach adopted is to understand the physics of human fingers-drumstick-drumhead interaction and try to replicate the same behavior in a robotic drumming system with the minimum number of degrees of freedom. The model that is developed is agnostic to the exact specifications of the robotic drummer that will attempt to emulate human like drum strokes, and therefore can be used in any robotic drummer that uses actuators with complete control over the motor position angle. Initial approaches based on exploiting the instability of a PID control system to generate multiple bounces and the limitations of this approach are also discussed in depth. In order to assess the success of the model and the implementation in the robotic platform a subjective evaluation was conducted. The evaluation results showed that, the observed data was statistically equivalent to the subjects resorting to a blind guess in order to distinguish between a human playing a multiple bounce stroke and a robot playing a similar kind of stroke.M.S

Expressive Musical Robots: Building, Evaluating, and Interfacing with an Ensemble of Mechatronic Instruments

An increase in the number of parameters of expression on musical robots can result in an increase in their expressivity as musical instruments. This thesis focuses on the design, construction, and implementation of four new robotic instruments, each designed to add more parametric control than is typical for the current state of the art of musical robotics. The principles followed in the building of the four new instruments are scalable and can be applied to musical robotics in general: the techniques exhibited in this thesis for the construction and use of musical robotics can be used by composers, musicians, and installation artists to add expressive depth to their own works with robotic instruments. Accompanying the increase in parametric depth applied to the musical robotics is an increase in difficulty in interfacing with them: robots with a greater number of actuators require more time to program. This document aims to address this problem in two ways: the use of closed-loop control for low-level adjustments of the robots and the use of a parametric encoding-equipped musical robot network to provide composers with intuitive musical commands for the robots. The musical robots introduced, described, and applied in this thesis were conceived of as musical instruments for performance and installation use by artists. This thesis closes with an exhibition of the performance and installation uses of these new robots and with a discussion of future research directions

Realizing Architecture’s Disruptive Potential

Digital technologies—computers and computer-controlled machines have pervaded all aspects of life, delivering sustained and accelerated rates of societal and economic evo - lution. Yet, in architecture, such an embrace of digital technologies and attendant intellectual disposition is not widely accepted. Whilst increasing numbers of progressive firms and research institutions are forging rapidly forward in such a technological upgrade, the larger populace of architects and the architecture produced thereof is decidedly averse to it

Playing fast and loose with music recognition

We report lessons from iteratively developing a music recognition system to enable a wide range of musicians to embed musical codes into their typical performance practice. The musician composes fragments of music that can be played back with varying levels of embellishment, disguise and looseness to trigger digital interactions. We collaborated with twenty-three musicians, spanning professionals to amateurs and working with a variety of instruments. We chart the rapid evolution of the system to meet their needs as they strove to integrate music recognition technology into their performance practice, introducing multiple features to enable them to trade-off reliability with musical expression. Collectively, these support the idea of deliberately introducing ‘looseness’ into interactive systems by addressing the three key challenges of control, feedback and attunement, and highlight the potential role for written notations in other recognition-based systems

Exploration in robotic musical instrument design

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (leaves 169-173).This thesis presents several works involving robotic musical instruments. Robots have long been used in industry for performing repetitive tasks, or jobs requiring superhuman strength. However, more recently robots have found a niche as musical instruments. The works presented here attempt to address the musicality of these instruments, their use in various settings, and the relationship of a robotic instrument to its human player in terms of mapping and translating gesture to sound. The primary project, The Chandelier, addresses both hardware and software issues, and builds directly from experience with two other works, The Marshall Field's Flower Show and Jeux Deux. The Marshall Field's Flower Show is an installation for several novel musical instruments and controllers. Presented here is a controller and mapping system for a Yamaha Disklavier player piano that allows for real-time manipulation of musical variations on famous compositions. The work is presented in the context of the exhibit, but also discussed in terms of its underlying software and technology. Jeux Deux is a concerto for hyperpiano, orchestra, and live computer graphics.(cont.) The software and mapping schema for this piece are presented in this thesis as a novel method for live interaction, in which a human player duets with a computer controlled player piano. Results are presented in the context of live performance. The Chandelier is the culmination of these past works, and presents a full-scale prototype of a new robotic instrument. This instrument explores design methodology, interaction, and the relationship-and disconnect-of a human player controlling a robotic instrument. The design of hardware and software, and some mapping schema are discussed and analyzed in terms of playability, musicality, and use in public installation and individual performance. Finally, a proof-of-concept laser harp is presented as a low-cost alternative musical controller. This controller is easily constructed from off-the-shelf parts. It is analyzed in terms of its sensing abilities and playability.Michael A. Fabio.S.M

Paradoxes of interactivity: perspectives for media theory, human-computer interaction, and artistic investigations

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. "Paradoxes of Interactivity" brings together reflections on "interactivity" from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Paradoxes of Interactivity

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. »Paradoxes of Interactivity« brings together reflections on »interactivity« from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Toward Interactive Music Generation: A Position Paper

Music generation using deep learning has received considerable attention in recent years. Researchers have developed various generative models capable of imitating musical conventions, comprehending the musical corpora, and generating new samples based on the learning outcome. Although the samples generated by these models are persuasive, they often lack musical structure and creativity. For instance, a vanilla end-to-end approach, which deals with all levels of music representation at once, does not offer human-level control and interaction during the learning process, leading to constrained results. Indeed, music creation is a recurrent process that follows some principles by a musician, where various musical features are reused or adapted. On the other hand, a musical piece adheres to a musical style, breaking down into precise concepts of timbre style, performance style, composition style, and the coherency between these aspects. Here, we study and analyze the current advances in music generation using deep learning models through different criteria. We discuss the shortcomings and limitations of these models regarding interactivity and adaptability. Finally, we draw the potential future research direction addressing multi-agent systems and reinforcement learning algorithms to alleviate these shortcomings and limitations

Strung Together: A Practical Exploration of Music-Cultural Hybridity, Interaction, and Collaboration

Strung Together is a collaborative performance project commissioned in September 2017 by Diaspora Arts Connection in San Francisco, US. Drawing from models of cultural integration and collaborative creativity, this research project enabled the development of a practical methodology through which the improvisatory approaches of non-congruent music-cultures might be combined to create a programme of original, eclectic works, within a limited time frame. Considering the subtle boundaries which lie between coexistence, assimilation, and synthesis within intercultural collaborations, Strung Together explored how different initial musical stimuli might alter the balance, whilst maintaining contextually-relative improvisatory freedom/s; and optimising productivity. Here the blending of three improvisation-based music traditions was investigated – Persian Classical, Arabic traditional and Western contemporary – through a process comprising: continual dialogue; collective composition; coalesced methods of improvisation; rearrangement and refinement; rehearsals; and live performance. Acting as musical director/performer, I sourced four professional musicians from the San Francisco Bay area – each expert in different traditions of improvisatory music(s) – to form a quintet; and was ultimately responsible for the project’s curation and delivery. I provided various pre-composed musical stimuli, Fragment(s), each of which incorporated influences from the performers’ respective music traditions and served as initial platforms for the development of the final pieces. We gathered together daily for one week, and during this time collectively developed, arranged, and rehearsed a complete performance programme of new, hybrid music. A live performance took place on the final day at the renowned Buriel Clay Theatre, which was streamed live via social media, reaching a worldwide audience. This presentation will reflect on the creative practice behind Strung Together, demonstrating that by inaugurating a democratic environment, where manifold approaches to music-making are considered and respected at a structural level, music-cultural hybridity is achievable within a limited time frame