Analyzing Visual Mappings of Traditional and Alternative Music Notation

In this paper, we postulate that combining the domains of information visualization and music studies paves the ground for a more structured analysis of the design space of music notation, enabling the creation of alternative music notations that are tailored to different users and their tasks. Hence, we discuss the instantiation of a design and visualization pipeline for music notation that follows a structured approach, based on the fundamental concepts of information and data visualization. This enables practitioners and researchers of digital humanities and information visualization, alike, to conceptualize, create, and analyze novel music notation methods. Based on the analysis of relevant stakeholders and their usage of music notation as a mean of communication, we identify a set of relevant features typically encoded in different annotations and encodings, as used by interpreters, performers, and readers of music. We analyze the visual mappings of musical dimensions for varying notation methods to highlight gaps and frequent usages of encodings, visual channels, and Gestalt laws. This detailed analysis leads us to the conclusion that such an under-researched area in information visualization holds the potential for fundamental research. This paper discusses possible research opportunities, open challenges, and arguments that can be pursued in the process of analyzing, improving, or rethinking existing music notation systems and techniques.Comment: 5 pages including references, 3rd Workshop on Visualization for the Digital Humanities, Vis4DH, IEEE Vis 201

Interactive music visualization: implementation, realization and evaluation

This thesis describes all process of the development of music visualization, starting with the implementation, followed by realization and then evaluation. The main goal is to have to knowledge of how the audience live performance experience can be enhanced through music visualization. With music visualization is possible to give a better understanding about the music feelings constructing an intensive atmosphere in the live music performance, which enhances the connection between the live music and the audience through visuals. These visuals have to be related to the live music, furthermore has to quickly respond to live music changes and introduce novelty into the visuals. The mapping between music and visuals is the focus of this project, in order to improve the relationship between the live performance and the spectators. The implementation of music visualization is based on the translation of music into graphic visualizations, therefore at the beginning the project was based on the existent works. Later on, it was decided to introduce new ways of conveying music into visuals. Several attempts were made in order to discover the most efficient mapping between music and visualization so people can fully connect with the performance. Throughout this project, those attempts resulted in several music visualizations created for four live music performances, afterwards it was produced an online survey to evaluate those live performances with music visualization. In the end, all conclusions are presented based on the results of the online survey, and also is explained which music elements should be depicted in the visuals, plus how those visuals should respond to the selected music elements.Universidade da Madeir

Emotion Recognition Education in Western Art Music Appreciation

Because Western art music is harmonically more complex than popular music and because it is written with musical notation, it may be challenging for certain people with no music training (non-musicians), those who did not grow up with Western art music, or those who did not choose to listen to this type of music for enjoyment to understand and appreciate it. Furthermore, there is a prevalent belief that Western art music is for the wealthy and elderly. This belief may be preventing symphony orchestra groups from cultivating new audiences. This study aims to determine if a narrative music listening activity would generate emotional response and cognitive engagement in a study group of non-Western art music listeners and prompt them to create musical narratives. Theoretically, narrative form music listening may present episodic memories, which can be built up into stories. To test the effect of narrative music listening activities, an online survey was distributed to non-Western art music listeners in the 20 through 40 age range, and pretest–treatment–posttest activity was devised and administered to three groups, an absolute music listening group, a programmatic music listening group, and a polyphonic texture listening group. In the treatment section, the creative listening activity, participants were prompted to create musical narratives, which take the form of colors, shapes, dialogues, or explicit stories. Participants were then asked to write about the music they heard before and after the narrative music listening activity. Participants’ motivation to attend a Western art music concert was assessed via a motivation scale using Likert scales. The results suggest that this online activity’s multimodality was a promising method for enhancing the appreciation of Western art music

Real-time 3D Graphic Augmentation of Therapeutic Music Sessions for People on the Autism Spectrum

This thesis looks at the requirements analysis, design, development and evaluation of an application, CymaSense, as a means of improving the communicative behaviours of autistic participants through therapeutic music sessions, via the addition of a visual modality. Autism spectrum condition (ASC) is a lifelong neurodevelopmental disorder that can affect people in a number of ways, commonly through difficulties in communication. Interactive audio-visual feedback can be an effective way to enhance music therapy for people on the autism spectrum. A multi-sensory approach encourages musical engagement within clients, increasing levels of communication and social interaction beyond the sessions.Cymatics describes a resultant visualised geometry of vibration through a variety of mediums, typically through salt on a brass plate or via water. The research reported in this thesis focuses on how an interactive audio-visual application, based on Cymatics, might improve communication for people on the autism spectrum.A requirements analysis was conducted through interviews with four therapeutic music practitioners, aimed at identifying working practices with autistic clients. CymaSense was designed for autistic users in exploring effective audio-visual feedback, and to develop meaningful cross-modal mappings of musical practitioner-client communication. CymaSense mappings were tested by 17 high functioning autistic participants, and by 30 neurotypical participants. The application was then trialled as a multimodal intervention for eight participants with autism, over a 12-week series of therapeutic music sessions. The study captured the experiences of the users and identified behavioural changes as a result, including information on how CymaSense could be developed further. This dissertation contributes evidence that multimodal applications can be used within therapeutic music sessions as a tool to increase communicative behaviours for autistic participants