Analyzing Visual Mappings of Traditional and Alternative Music Notation

In this paper, we postulate that combining the domains of information visualization and music studies paves the ground for a more structured analysis of the design space of music notation, enabling the creation of alternative music notations that are tailored to different users and their tasks. Hence, we discuss the instantiation of a design and visualization pipeline for music notation that follows a structured approach, based on the fundamental concepts of information and data visualization. This enables practitioners and researchers of digital humanities and information visualization, alike, to conceptualize, create, and analyze novel music notation methods. Based on the analysis of relevant stakeholders and their usage of music notation as a mean of communication, we identify a set of relevant features typically encoded in different annotations and encodings, as used by interpreters, performers, and readers of music. We analyze the visual mappings of musical dimensions for varying notation methods to highlight gaps and frequent usages of encodings, visual channels, and Gestalt laws. This detailed analysis leads us to the conclusion that such an under-researched area in information visualization holds the potential for fundamental research. This paper discusses possible research opportunities, open challenges, and arguments that can be pursued in the process of analyzing, improving, or rethinking existing music notation systems and techniques.Comment: 5 pages including references, 3rd Workshop on Visualization for the Digital Humanities, Vis4DH, IEEE Vis 201

Augmenting Music Sheets with Harmonic Fingerprints

Conventional Music Notation (CMN) is the well-established foundation for the written communication of musical information, such as rhythm, harmony, or timbre. However, CMN suffers from the complexity of its visual encoding and the need for extensive training to acquire proficiency and legibility. While alternative notations using additional visual variables (such as color to improve pitch identification) have been proposed, the music community does not readily accept notation systems that vary widely from the CMN. Therefore, to support student musicians in understanding the harmonic relationship of notes, instead of replacing the CMN, we present a visualization technique that augments a digital music sheet with a harmonic fingerprint glyph. Our design exploits the circle of fifths - a fundamental concept in music theory, as a visual metaphor. By attaching these visual glyphs to each bar of a selected composition we provide additional information about the salient harmonic features available in a musical piece. We conducted a user study to analyze the performance of experts and non-experts in an identification and comparison task of recurring patterns. The evaluation shows that the harmonic fingerprint supports these tasks without the need for close-reading, as when compared to a not-annotated music sheet.Comment: (9+1) pages; 5 figures; User Stud

Computer Music Algorithms. Bio-inspired and Artificial Intelligence Applications

2014 - 2015Music is one of the arts that have most benefited from the invention of computers. Originally, the term Computer Music was used in the scientific community to identify the application of information technology in music composition. It began over time to include the theory and application of new or existing technologies in music, such as sound synthesis, sound design, acoustic, psychoacoustic. Thanks to its interdisciplinary nature, Computer Music can be seen as the encounter of different disciplines. In the last years technology has redefined the way individuals can work, communicate, share experiences, constructively debate, and actively participate to any aspect of the daily life, ranging from business to education, from political and intellectual to social, and also in music activity, such as play music, compose music and so on. In this new context, Computer Music has become an emerging research area for the application of Computational Intelligence techniques, such as machine learning, pattern recognition, bio-inspired algorithms and so on. My research activity is concerned with the Bio-inspired and Artificial Intelligence Applications in the Computer Music. Some of the problems I addressed are summarized in the following. Automatic composition of background music for games, films and other human activities: EvoBackMusic. Systems for real-time composition of background music respond to changes of the environment by generating music that matches the current state of the environment and/or of the user. We propose one such a system that we call EvoBackMusic. It is a multiagent system that exploits a feed-forward neural network and a multi-objective genetic algorithm to produce background music. The neural network is trained to learn the preferences of the user and such preferences are exploited by the genetic algorithm to compose the music. The composition process takes into account a set of controllers that describe several aspects of the environment, like the dynamism of both the user and the 2 context, other physical characteristics, and the emotional state of the user. Previous system mainly focus on the emotional aspect. Publications: • Roberto De Prisco, Delfina Malandrino, Gianluca Zaccagnino, Rocco Zaccagnino: ‘‘An Evolutionary Composer for Real-Time Background Music’’. EvoMUSART 2016: 135-151. Interaction modalities for music performances: MarcoSmiles. In this field we considered new interaction modalities during music performances by using hands without the support of a real musical instrument. Exploiting natural user interfaces (NUI), initially conceived for the game market, it is possible to enhance the traditional modalities of interaction when accessing to technology, build new forms of interactions by transporting users in a virtual dimension, but that fully reflects the reality, and finally, improve the overall perceived experience. The increasing popularity of these innovative interfaces involved their adoption in other fields, including Computer Music. We propose a system, named MarcoSmiles, specifically designed to allow individuals to perform music in an easy, innovative, and personalized way. The idea is to design new interaction modalities during music performances by using hands without the support of a real musical instrument. We exploited Artificial Neural Networks to customize the virtual musical instrument, to provide the information for the mapping of the hands configurations into musical notes and, finally, to train and test these configurations. We performed several tests to study the behavior of the system and its efficacy in terms of learning capabilities. Publications: • Roberto De Prisco, Delfina Malandrino, Gianluca Zaccagnino, Rocco Zaccagnino: ‘‘Natural Users Interfaces to support and enhance Real-Time Music Performance’’. AVI 2016. 3 Bio-inspired approach for automatic music composition Here we describe a new bio-inspired approach for automatic music composition in a specific style: Music Splicing System. Splicing systems were introduced by Tom Head (1987) as a formal model of a recombination process between DNA molecules. The existing literature on splicing systems mainly focuses on the computational power of these systems and on the properties of the generated languages; very few applications based on splicing systems have been introduced. We show a novel application of splicing systems to build an automatic music composer. As a result of a performance study we proved that our composer outperforms other meta-heuristics by producing better music according to a specific measure of quality evaluation, and this proved that the proposed system can be seen also as a new valid bio-inspired strategy for automatic music composition. Publications: ▪ Clelia De Felice, Roberto De Prisco, Delfina Malandrino, Gianluca Zaccagnino, Rocco Zaccagnino, Rosalba Zizza: ‘‘Splicing Music Composition’’. Information Sciences Journal, 385: 196 – 215 (2017). ▪ Clelia De Felice, Roberto De Prisco, Delfina Malandrino, Gianluca Zaccagnino, Rocco Zaccagnino, Rosalba Zizza: ‘‘Chorale Music Splicing System: An Algorithmic Music Composer Inspired by Molecular Splicing’’. EvoMusart 2015: 50 – 61. Music and Visualization Here we describe new approaches for learning of harmonic and melodic rules of classic music, by using visualization techniques: VisualMelody and VisualHarmony. Experienced musicians have the ability to understand the structural elements of music compositions. Such an ability is built over time through the study of music theory, the understanding of rules that guide the composition of music, and through countless hours of practice. The learning process is hard, especially for classical music, where the rigidity of the music structures and styles requires great effort to understand, assimilate, and then master the learned notions. In particular, we focused our attention on a specific type of music compositions, namely, music in chorale style (4-voice music). Composing such type of music 4 is often perceived as a difficult task, because of the rules the composer has to adhere to. In this paper we propose a visualization technique that can help people lacking a strong knowledge of music theory. The technique exploits graphic elements to draw the attention on the possible errors in the composition. We then developed two interactive systems, named VisualMelody and VisualHarmony, that employ the proposed visualization techniques to facilitate the understanding of the structure of music compositions. The aim is to allow people to make 4-voice music composition in a quick and effective way, i.e., avoiding errors, as dictated by classical music theory rules. Publications: ▪ Roberto De Prisco, Delfina Malandrino, Donato Pirozzi, Gianluca Zaccagnino, Rocco Zaccagnino: ‘‘Understanding the structure of music compositions: is visualization an effective approach?’’ Information Visualization Journal, 2016. (DOI): 10.1177/1473871616655468 • Delfina Malandrino, Donato Pirozzi, Gianluca Zaccagnino, Rocco Zaccagnino: ‘‘A Color-Based Visualization Approach to Understand Harmonic Structures of Musical Compositions’’. IV 2015: 56-61. • Delfina Malandrino, Donato Pirozzi, Gianluca Zaccagnino, Rocco Zaccagnino: ‘‘Visual Approaches for Harmonic Analysis of 4-part Music: Implementation and Evaluation’’. Major revision – Journal of Visual Languages and Computing, 2016. [edited by Author]XIV n.s

Art/Sci Nexus, 9 Evenings Revisited

Following the exhibition of Hybrid Bodies at KKW in 2016 Andrew Carnie and I were invited back to act as mentors to a group of young artists and scientists from all over Europe undertaking a week long workshop designed to lead to new art/science collaborations. We were also invited to present the Hybrid Bodies project at a one day public event preceding the workshop

Reworking musical strategies in the digital age

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University, 2/2/2011.This thesis comprises a portfolio of creative musical work and a written commentary. The creative work seeks to rework musical strategies through technology by challenging aspects of how music is traditionally performed, composed and presented. The portfolio of submitted work is divided into five main projects. The first project is E-tudes, a set of four compositions for live electronics and six keyboard players. The second project is a composition called On Violence, for piano, live electronics, sensors and computer display. The third project is Zizek!?, a computer-mediated-performance for three improvisers that serves as an alternative soundtrack to a documentary about Slovenian philosopher Slavoj Zizek. The fourth project is a collection of small experimental pieces for fixed media called FreuPinta. The fifth project consists of a selection of different improvisations that I devised or participated in using a computer environment I developed for live improvisation. Throughout the portfolio recent technological advancements are considered not for their use in implementing pre-existing models of music-making but rather for their potential to challenge preconceived notions about music. The written commentary gives the theoretical tools necessary to understand the underlying reasoning, preoccupations and concerns behind the submitted work as well as providing supplementary information about the musical results and the computer programmes developed as part of this research.Financial support from the Overseas Research Scheme (ORS), Brunel University School of Arts and the PRS for Music Foundation was used for this study