3d augmented mirror: a multimodal interface for string instrument learning and teaching with gesture support

Multimodal interfaces can open up new possibilities for music education, where the traditional model of teaching is based predominantly on verbal feedback. This paper explores the development and use of multimodal interfaces in novel tools to support music practice training. The design of multimodal interfaces for music education presents a challenge in several respects. One is the integration of multimodal technology into the music learning process. The other is the technological development, where we present a solution that aims to support string practice training with visual and auditory feedback. Building on the traditional function of a physical mirror as a teaching aid, we describe the concept and development of an "augmented mirror" using 3D motion capture technology

Music Technology Education and a Plugin-Based Platform as a Tool to Enhance Creativity, Multidisciplinarity, Creative Design, and Collaboration Skills

Music technology is known to have the ability to enhance creativity and creative development among students. A high level of engagement has been shown among students who studied and developed musical projects, and among students who were intellectually involved in the process of meaningful exploration. When students develop a music technology project, they use their software design skills to build and combine different artistic and computational components. Here we present a creative education method for computer science and software engineering students, it uses Muzilator, a plugin-based web platform that enables developers to develop a project as a set of independent web applications (plugins). Students can share their plugins with others or use plugins developed by others. We examined 75 projects of teams of computer science students who participated in a Computer Music course. We studied the characteristics of these projects and Muzilator’s effectiveness as a creative education and collaboration tool. Some of our results show that Muzilator-based projects received higher creativity and multidisciplinarity ratings than did other projects, and that high-risk projects were more creative and artistic than low-risk ones. We also found a gender-dependency: women tended more than men to develop interactive applications, while men tended to choose more theoretic (algorithmic), non-interactive projects. Keywords: educational method, creativity, music education, software design, multidisciplinarity. DOI: 10.7176/JEP/12-11-01 Publication date: April 30th 202

Sonic Skills

It is common for us today to associate the practice of science primarily with the act of seeing—with staring at computer screens, analyzing graphs, and presenting images. We may notice that physicians use stethoscopes to listen for disease, that biologists tune into sound recordings to understand birds, or that engineers have created Geiger tellers warning us for radiation through sound. But in the sciences overall, we think, seeing is believing. This open access book explains why, indeed, listening for knowledge plays an ambiguous, if fascinating, role in the sciences. For what purposes have scientists, engineers and physicians listened to the objects of their interest? How did they listen exactly? And why has listening often been contested as a legitimate form of access to scientific knowledge? This concise monograph combines historical and ethnographic evidence about the practices of listening on shop floors, in laboratories, field stations, hospitals, and conference halls, between the 1920s and today. It shows how scientists have used sonic skills—skills required for making, recording, storing, retrieving, and listening to sound—in ensembles: sets of instruments and techniques for particular situations of knowledge making. Yet rather than pleading for the emancipation of hearing at the expense of seeing, this essay investigates when, how, and under which conditions the ear has contributed to science dynamics, either in tandem with or without the eye

The Photosynthesiser - A methodology for mapping environmental conditions, pivotal to the speed of photosynthesis in plants, through sonification.

Traditionally, the close inspection of data requires visual guidance in the form of displays depicting numeric or graphical representations over time. Sonification offers a way to convey this data through auditory means, relinquishing the need for constant display monitoring. To enable horticulturists to continue to move and work around their environment a proposed sonification mapping system for the key environmental conditions, vital for optimum levels of photosynthesis, has been developed. The outcome of this research was to provide a monitoring system that was both musical and meaningful with regards to data fluctuations and most importantly, could be interpreted by a wide demographic of listeners. A literature review provides an underpinning to both the scientific and artistic merits of sonification whilst a practice-based model was used to develop appropriate musical timbres, offering a natural instrumentation through physical modelling synthesis. Key questions around which musical factors can be used to trigger specific emotions and which of these emotions do we associate with an environment that offers a higher rate or low rate of photosynthesis for plants are explored. Through literary research as well as the deployment and analysis of surveys, a list of musical parameters was identified and a mapping framework designed. To analyse the success of the design, an audio installation was constructed within grounds at the Eden Project. The environmental data of both biomes, tropical and Mediterranean, were sonified into two musical streams and visitors surveyed through quantitative and qualitative methods in an experiment to see if they could correctly associate the music to the biome. The results provided 90% accuracy in the correct identification. It is theorised through this research that the mapping framework designed can be used in the sonification of climate conditions and communicate key traits within each environment

Using Sound to Represent Uncertainty in Spatial Data

There is a limit to the amount of spatial data that can be shown visually in an effective manner, particularly when the data sets are extensive or complex. Using sound to represent some of these data (sonification) is a way of avoiding visual overload. This thesis creates a conceptual model showing how sonification can be used to represent spatial data and evaluates a number of elements within the conceptual model. These are examined in three different case studies to assess the effectiveness of the sonifications. Current methods of using sonification to represent spatial data have been restricted by the technology available and have had very limited user testing. While existing research shows that sonification can be done, it does not show whether it is an effective and useful method of representing spatial data to the end user. A number of prototypes show how spatial data can be sonified, but only a small handful of these have performed any user testing beyond the authors’ immediate colleagues (where n > 4). This thesis creates and evaluates sonification prototypes, which represent uncertainty using three different case studies of spatial data. Each case study is evaluated by a significant user group (between 45 and 71 individuals) who completed a task based evaluation with the sonification tool, as well as reporting qualitatively their views on the effectiveness and usefulness of the sonification method. For all three case studies, using sound to reinforce information shown visually results in more effective performance from the majority of the participants than traditional visual methods. Participants who were familiar with the dataset were much more effective at using the sonification than those who were not and an interactive sonification which requires significant involvement from the user was much more effective than a static sonification, which did not provide significant user engagement. Using sounds with a clear and easily understood scale (such as piano notes) was important to achieve an effective sonification. These findings are used to improve the conceptual model developed earlier in this thesis and highlight areas for future research

16th Sound and Music Computing Conference SMC 2019 (28–31 May 2019, Malaga, Spain)

The 16th Sound and Music Computing Conference (SMC 2019) took place in Malaga, Spain, 28-31 May 2019 and it was organized by the Application of Information and Communication Technologies Research group (ATIC) of the University of Malaga (UMA). The SMC 2019 associated Summer School took place 25-28 May 2019. The First International Day of Women in Inclusive Engineering, Sound and Music Computing Research (WiSMC 2019) took place on 28 May 2019. The SMC 2019 TOPICS OF INTEREST included a wide selection of topics related to acoustics, psychoacoustics, music, technology for music, audio analysis, musicology, sonification, music games, machine learning, serious games, immersive audio, sound synthesis, etc