Towards building a Deep Learning based Automated Indian Classical Music Tutor for the Masses

Music can play an important role in the well-being of the world. Indian classical music is unique in its requirement for rigorous, disciplined, expert-led training that typically goes on for years before the learner can reach a reasonable level of performance. This keeps many, including the first author of this paper, away from mastering the skill. The problem is particularly compounded in rural areas, where the available expertise may be limited and prohibitively expensive, but the interest in learning classical music still prevails, nevertheless. Machine Learning has been complementing, enhancing, and replacing many white-collar jobs and we believe it can help with this problem as well. This paper describes efforts at using Machine Learning techniques, particularly, Long Short-Term Memory for building a system that is a step toward provisioning an Indian Classical Music Tutor for the masses. The system is deployed in the cloud using orchestrated containerization for potential worldwide access, load balancing, and other robust features

DeepSRGM -- Sequence Classification and Ranking in Indian Classical Music with Deep Learning

A vital aspect of Indian Classical Music (ICM) is Raga, which serves as a melodic framework for compositions and improvisations alike. Raga Recognition is an important music information retrieval task in ICM as it can aid numerous downstream applications ranging from music recommendations to organizing huge music collections. In this work, we propose a deep learning based approach to Raga recognition. Our approach employs efficient pre possessing and learns temporal sequences in music data using Long Short Term Memory based Recurrent Neural Networks (LSTM-RNN). We train and test the network on smaller sequences sampled from the original audio while the final inference is performed on the audio as a whole. Our method achieves an accuracy of 88.1% and 97 % during inference on the Comp Music Carnatic dataset and its 10 Raga subset respectively making it the state-of-the-art for the Raga recognition task. Our approach also enables sequence ranking which aids us in retrieving melodic patterns from a given music data base that are closely related to the presented query sequence

Hindustani raga and singer classification using 2D and 3D pose estimation from video recordings

Using pose estimation with video recordings, we apply an action recognition machine learning algorithm to demonstrate the use of the movement information to classify singers and the ragas (melodic modes) they perform. Movement information is derived from a specially recorded video dataset of solo Hindustani (North Indian) raga recordings by three professional singers each performing the same nine ragas, a smaller duo dataset (one singer with tabla accompaniment) as well as recordings of concert performances by the same singers. Data is extracted using pose estimation algorithms, both 2D (OpenPose) and 3D. A two-pathway convolutional neural network structure is proposed for skeleton action recognition to train a model to classify 12-second clips by singer and raga. The model is capable of distinguishing the three singers on the basis of movement information alone. For each singer, it is capable of distinguishing between the nine ragas with a mean accuracy of 38.2% (with the most successful model). The model trained on solo recordings also proved effective at classifying duo and concert recordings. These findings are consistent with the view that while the gesturing of Indian singers is idiosyncratic, it remains tightly linked to patterns of melodic movement: indeed we show that in some cases different ragas are distinguishable on the basis of movement information alone. A series of technical challenges are identified and addressed, with code shared alongside audiovisual data to accompany the paper

Exploring deep learning based methods for information retrieval in Indian classical music

A vital aspect of Indian Classical Music (ICM) is Raga, which serves as a melodic framework for compositions and improvisations alike. Raga Recognition is an important music information retrieval task in ICM as it can aid numerous downstream applications ranging from music recommendations to organizing huge music collections. In this work, we propose a deep learning based approach to Raga recognition. Our approach employs efficient pre-possessing and learns temporal sequences in music data using Long Short Term Memory based Recurrent Neural Networks (LSTM-RNN). We train and test the network on smaller sequences sampled from the original audio while the final inference is performed on the audio as a whole. Our method achieves an accuracy of 88.1% and 97 % during inference on the Comp Music Carnatic dataset and its 10 Raga subset respectively making it the state of-the-art for the Raga recognition task. Our approach also enables sequence ranking which aids us in retrieving melodic patterns from a given music data base that are closely related to the presented query sequence

Repertoire-Specific Vocal Pitch Data Generation for Improved Melodic Analysis of Carnatic Music

Deep Learning methods achieve state-of-the-art in many tasks, including vocal pitch extraction. However, these methods rely on the availability of pitch track annotations without errors, which are scarce and expensive to obtain for Carnatic Music. Here we identify the tradition-related challenges and propose tailored solutions to generate a novel, large, and open dataset, the Saraga-Carnatic-Melody-Synth (SCMS), comprising audio mixtures and time-aligned vocal pitch annotations. Through a cross-cultural evaluation leveraging this novel dataset, we show improvements in the performance of Deep Learning vocal pitch extraction methods on Indian Art Music recordings. Additional experiments show that the trained models outperform the currently used heuristic-based pitch extraction solutions for the computational melodic analysis of Carnatic Music and that this improvement leads to better results in the musicologically relevant task of repeated melodic pattern discovery when evaluated using expert annotations. The code and annotations are made available for reproducibility. The novel dataset and trained models are also integrated into the Python package compIAM1 which allows them to be used out-of-the-box

Computational Analysis of Greek folk music of the Aegean islands

Αν και έχουν αναπτυχθεί νεότερα και πιο ανεπτυγμένα μοντέλα υπολογιστικής μουσικής ανάλυσης με στόχο την αύξηση διαθέσιμης πληροφορίας στον κλάδο της μουσικολογίας, υπάρχει πολύ λίγη έρευνα στην υπολογιστική ανάλυση δημοτικής μουσικής γενικότερα και ελληνικής δημοτικής μουσικής ειδικότερα. Στόχος της παρούσας εργασίας είναι η διερεύνηση ποικίλων τύπων μουσικών χαρακτηριστικών και προτύπων στη δημοτική μουσική των νησιών του Αιγαίου και η παροχή χρήσιμης πληροφορίας σχετικά με τη δομή και το περιεχόμενο του εν λόγω είδους. Επιπρόσθετα, με στόχο τη σύγκριση μουσικών αποσπασμάτων χορών Συρτού και Μπάλου, αλλά και γεωγραφικών περιοχών από τις οποίες προέρχονται, 73 αποσπάσματα συγκεντρώθηκαν συνολικά σε μια βάση δεδομένων και αναλύθηκαν. Η εξαγωγή χαρακτηριστικών και η ανάλυση προτύπων ανέδειξαν μελωδικές και ρυθμικές διαφορές τόσο ανάμεσα στα δύο είδη χορών όσο και στις διάφορες νησιωτικές περιοχές, ενώ υπήρξαν επίσης ποικίλες ομοιότητες σε όλο το σύνολο των δεδομένων.While newer, advanced computational music analysis models have been developed with the intentions of increasing available information in this field, very little research exists on the computational analysis of folk music in general and Greek folk music in specific. The aim of this study was to examine various types of musical features and patterns in the folk music of the Aegean islands and provide useful information about the structure and the content of this music style. In addition, to compare the tunes of Syrtos and Mpalos dances, but also the various island regions from which they originate, a total of 73 tunes were included in the constructed dataset and the analyses. Feature extraction and pattern analysis revealed that there are indeed melodic and temporal differences both between the two dance types and between the island regions, while there were also various important similarities throughout the whole dataset

Listening for the Cosmic Other: Postcolonial Approaches to Music in the Space Age

As government programs such as NASA and SETI seek signs of intelligent life in space and privately-funded programs such as SpaceX finalize plans to colonize Mars in the coming decades, representations of space and extraterrestrial life in American culture have become increasingly relevant. Focusing on Jóhann Jóhannsson’s musical score for Denis Villeneuve’s science-fiction film Arrival (2016), Terry Riley’s Sun Rings (2002) for string quartet, chorus, and recorded space sounds, and former International Space Station Commander Chris Hadfield’s “Songs about Space” Spotify playlist, my research problematizes the ways in which composers, musicians, and even astronauts depict alterity through music and reinforce colonial narratives about outer space. Drawing upon the work of postcolonial theorists and musicologists such as Charles Forsdick (2003), Olivia A. Bloechl (2008), Ania Loomba (2015), and others, this thesis argues that musical depictions of extraterrestrials and space exploration, more generally, reveal the potential for discrimination, misrepresentation, and abuses of power to emerge from space colonization. But, back on Earth, this study also suggests that such fraught theoretical relationships between human colonists and extraterrestrials echo the real-life suffering of colonized indigenous groups and the necessity of decolonization

Computational analysis of world music corpora

PhDThe comparison of world music cultures has been considered in musicological research since the end of the 19th century. Traditional methods from the field of comparative musicology typically involve the process of manual music annotation. While this provides expert knowledge, the manual input is timeconsuming and limits the potential for large-scale research. This thesis considers computational methods for the analysis and comparison of world music cultures. In particular, Music Information Retrieval (MIR) tools are developed for processing sound recordings, and data mining methods are considered to study similarity relationships in world music corpora. MIR tools have been widely used for the study of (mainly) Western music. The first part of this thesis focuses on assessing the suitability of audio descriptors for the study of similarity in world music corpora. An evaluation strategy is designed to capture challenges in the automatic processing of world music recordings and different state-of-the-art descriptors are assessed. Following this evaluation, three approaches to audio feature extraction are considered, each addressing a different research question. First, a study of singing style similarity is presented. Singing is one of the most common forms of musical expression and it has played an important role in the oral transmission of world music. Hand-designed pitch descriptors are used to model aspects of the singing voice and clustering methods reveal singing style similarities in world music. Second, a study on music dissimilarity is performed. While musical exchange is evident in the history of world music it might be possible that some music cultures have resisted external musical influence. Low-level audio features are combined with machine learning methods to find music examples that stand out in a world music corpus, and geographical patterns are examined. The last study models music similarity using descriptors learned automatically with deep neural networks. It focuses on identifying music examples that appear to be similar in their audio content but share no (obvious) geographical or cultural links in their metadata. Unexpected similarities modelled in this way uncover possible hidden links between world music cultures. This research investigates whether automatic computational analysis can uncover meaningful similarities between recordings of world music. Applications derive musicological insights from one of the largest world music corpora studied so far. Computational analysis as proposed in this thesis advances the state-of-the-art in the study of world music and expands the knowledge and understanding of musical exchange in the world.Queen Mary Principal’s research studentship