The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation

With recent breakthroughs in artificial neural networks, deep generative models have become one of the leading techniques for computational creativity. Despite very promising progress on image and short sequence generation, symbolic music generation remains a challenging problem since the structure of compositions are usually complicated. In this study, we attempt to solve the melody generation problem constrained by the given chord progression. This music meta-creation problem can also be incorporated into a plan recognition system with user inputs and predictive structural outputs. In particular, we explore the effect of explicit architectural encoding of musical structure via comparing two sequential generative models: LSTM (a type of RNN) and WaveNet (dilated temporal-CNN). As far as we know, this is the first study of applying WaveNet to symbolic music generation, as well as the first systematic comparison between temporal-CNN and RNN for music generation. We conduct a survey for evaluation in our generations and implemented Variable Markov Oracle in music pattern discovery. Experimental results show that to encode structure more explicitly using a stack of dilated convolution layers improved the performance significantly, and a global encoding of underlying chord progression into the generation procedure gains even more.Comment: 8 pages, 13 figure

Maximum entropy models capture melodic styles

We introduce a Maximum Entropy model able to capture the statistics of melodies in music. The model can be used to generate new melodies that emulate the style of the musical corpus which was used to train it. Instead of using the $n-$body interactions of $(n-1)-$order Markov models, traditionally used in automatic music generation, we use a $k-$nearest neighbour model with pairwise interactions only. In that way, we keep the number of parameters low and avoid over-fitting problems typical of Markov models. We show that long-range musical phrases don't need to be explicitly enforced using high-order Markov interactions, but can instead emerge from multiple, competing, pairwise interactions. We validate our Maximum Entropy model by contrasting how much the generated sequences capture the style of the original corpus without plagiarizing it. To this end we use a data-compression approach to discriminate the levels of borrowing and innovation featured by the artificial sequences. The results show that our modelling scheme outperforms both fixed-order and variable-order Markov models. This shows that, despite being based only on pairwise interactions, this Maximum Entropy scheme opens the possibility to generate musically sensible alterations of the original phrases, providing a way to generate innovation

Comparing Probabilistic Models for Melodic Sequences

Modelling the real world complexity of music is a challenge for machine learning. We address the task of modeling melodic sequences from the same music genre. We perform a comparative analysis of two probabilistic models; a Dirichlet Variable Length Markov Model (Dirichlet-VMM) and a Time Convolutional Restricted Boltzmann Machine (TC-RBM). We show that the TC-RBM learns descriptive music features, such as underlying chords and typical melody transitions and dynamics. We assess the models for future prediction and compare their performance to a VMM, which is the current state of the art in melody generation. We show that both models perform significantly better than the VMM, with the Dirichlet-VMM marginally outperforming the TC-RBM. Finally, we evaluate the short order statistics of the models, using the Kullback-Leibler divergence between test sequences and model samples, and show that our proposed methods match the statistics of the music genre significantly better than the VMM.Comment: in Proceedings of the ECML-PKDD 2011. Lecture Notes in Computer Science, vol. 6913, pp. 289-304. Springer (2011

Markov Chains for Computer Music Generation

Random generation of music goes back at least to the 1700s with the introduction of Musical Dice Games. More recently, Markov chain models have been used as a way of extracting information from a piece of music and generating new music. We explain this approach and give Python code for using it to first draw out a model of the music and then create new music with that model

Modelling hierarchical musical structures with composite probabilistic networks

The thesis is organised as follows:• Chapter 2 provides background information on existing research in the field of computational music harmonisation and generation, as well as some the¬ oretical background on musical structures. Finally, the chapter concludes with an outline of the scope and aims of this research.• Chapter 3 provides a short overview of the field of Machine Learning, ex¬ plaining concepts such as entropy measures and smoothing. The definitions of Markov chains and Hidden Markov models are introduced together with their methods of inference.• Chapter 4 begins with the definition of Hierarchical Hidden Markov models and techniques for linear time inference. It continues by introducing the new concept of Input-Output HHMMs, an extension to the hierarchical models that is derived from Input-Output HMMs.• Chapter 5 is a short chapter that shows the importance of the music rep¬ resentation and model structures for this research, and gives details of the representation.• Chapter 6 outlines the design of the software used for the HHMM modelling, and gives details of the software implementation and use.• Chapter 7 describes how dynamic networks of models were used for the generation of new pieces of music using a "random walk" approach. Several different types of networks are presented, exploring the different possibilities of layering the musical structures and organising the networks.• Chapter 8 tries to evaluate musical examples that were generated with sev¬ eral different types of networks. The evaluation process is both subjective and objective, using the results of a listening experiment as well as cross entropy measures and musical theoretical rules.• Chapter 9 offers a discussion of the methodology of the approach, the con¬ figuration and design of networks and models as well as the learning and generation of the new musical structures.• Chapter 10 concludes the thesis by summarising the research's contribu¬ tions, evaluating whether the project scope has been fulfilled and the major goals of the research have been met

Markov Model Composition of Balinese Reyong Norot Improvisations

Markov models are mathematical structures that model the transition between possible states based on the probability of moving from one state to any other. Thus, given a distribution of starting points, the model produces a chain of states that are visited in sequence. Such models have been used extensively to generate music based on probabilities, as sequences of states can represent sequences of notes and rhythms. While music generation is a common application of Markov models, most existing work attempts to reconstruct the musical style of classical Western composers. In this thesis, we produce a series of Markov chains that model the composition of Balinese gamelan gong kebyar improvisations on the reyong. This music features distinct rules and limitations. Each of the reyong’s four players can play only some of the gamelan\u27s five tones and must use specific patterns learned only by listening and playing. And yet, the music structure also provides room for ample creativity with improvisation. The model’s probability values come from a combination of top-down and bottom-up techniques, making extensive use of Leslie Tilley’s work on the grammar of \textit{reyong norot} and example patterns from her concurrent study of musician Dewa Ketut Alit’s improvisation. The model outputs MIDI files for audio playback of the constructed songs. Though the model’s music lacks some of the improvisational creative quality that humans provide, we find that our model does produce musically interesting reyong elaborations that fit within the confines of Tilley’s grammar

Beating-time gestures imitation learning for humanoid robots

Beating-time gestures are movement patterns of the hand swaying along with music, thereby indicating accented musical pulses. The spatiotemporal configuration of these patterns makes it diÿcult to analyse and model them. In this paper we present an innovative modelling approach that is based upon imitation learning or Programming by Demonstration (PbD). Our approach - based on Dirichlet Process Mixture Models, Hidden Markov Models, Dynamic Time Warping, and non-uniform cubic spline regression - is particularly innovative as it handles spatial and temporal variability by the generation of a generalised trajectory from a set of periodically repeated movements. Although not within the scope of our study, our procedures may be implemented for the sake of controlling movement behaviour of robots and avatar animations in response to music

Interactive Music Generation with Positional Constraints using Anticipation-RNNs

Recurrent Neural Networks (RNNS) are now widely used on sequence generation tasks due to their ability to learn long-range dependencies and to generate sequences of arbitrary length. However, their left-to-right generation procedure only allows a limited control from a potential user which makes them unsuitable for interactive and creative usages such as interactive music generation. This paper introduces a novel architecture called Anticipation-RNN which possesses the assets of the RNN-based generative models while allowing to enforce user-defined positional constraints. We demonstrate its efficiency on the task of generating melodies satisfying positional constraints in the style of the soprano parts of the J.S. Bach chorale harmonizations. Sampling using the Anticipation-RNN is of the same order of complexity than sampling from the traditional RNN model. This fast and interactive generation of musical sequences opens ways to devise real-time systems that could be used for creative purposes.Comment: 9 pages, 7 figure