Embodied Cognition in Performers of Large Acoustic Instruments as a Method of Designing New Large Digital Musical Instruments

We present The Large Instrument Performers Study, an interview-based exploration into how large scale acoustic instrument performers navigate the instrument's size-related aesthetic features during the performance. Through the conceptual frameworks of embodied music cognition and affordance theory, we discuss how the themes that emerged in the interview data reveal the ways size-related aesthetic features of large acoustic instruments influence the instrument performer's choices; how large scale acoustic instruments feature microscopic nuanced performance options; and how despite the preconception of large scale acoustic instruments being scaled up versions of the smaller instrument with the addition of a lower fundamental tone, the instruments o er different sonic and performative features to their smaller counterparts and require precise gestural control that is certainly not scaled up. This is followed by a discussion of how the study findings could influence design features in new large scale digital musical instruments to result in more nuanced control and timbrally rich instruments, and better understanding of how interfaces and instruments influence performers' choices and as a result music repertoire and performance

Zero-shot Singing Technique Conversion

In this paper we propose modifications to the neural network framework, AutoVC for the task of singing technique conversion. This includes utilising a pretrained singing technique encoder which extracts technique information, upon which a decoder is conditioned during training. By swapping out a source singer’s technique information for that of the target’s during conversion, the input spectrogram is reconstructed with the target’s technique. We document the beneficial effects of omitting the latent loss, the importance of sequential training, and our process for fine-tuning the bottleneck. We also conducted a listening study where participants rate the specificity of technique-converted voices as well as their naturalness. From this we are able to conclude how effective the technique conversions are and how different conditions affect them, while assessing the model’s ability to reconstruct its input data

Combining Vision and EMG-Based Hand Tracking for Extended Reality Musical Instruments

Hand tracking is a critical component of natural user interactions in extended reality (XR) environments, including extended reality musical instruments (XRMIs). However, self-occlusion remains a significant challenge for vision-based hand tracking systems, leading to inaccurate results and degraded user experiences. In this paper, we propose a multimodal hand tracking system that combines vision-based hand tracking with surface electromyography (sEMG) data for finger joint angle estimation. We validate the effectiveness of our system through a series of hand pose tasks designed to cover a wide range of gestures, including those prone to self-occlusion. By comparing the performance of our multimodal system to a baseline vision-based tracking method, we demonstrate that our multimodal approach significantly improves tracking accuracy for several finger joints prone to self-occlusion. These findings suggest that our system has the potential to enhance XR experiences by providing more accurate and robust hand tracking, even in the presence of self-occlusion

A framework for music similarity and cover song identification

This paper presents a framework for music information retrieval tasks which relate to music similarity. The framework is based on a pipeline consisting of audio feature extraction, feature aggregation and distance measurements, which generalizes previous work and includes hundreds of similarity models not previously considered in the literature. This general pipeline is subjected to a comprehensive benchmark of analogously defined music similarity models over the task of cover song identification. Experimental results provide scientific evidence for certain preferred combined choices of features, aggregations and distances, while pointing towards novel combinations of such elements with the potential to improve the performance of music similarity models on specific MIR tasks