982 research outputs found
MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
We introduce MIDI-VAE, a neural network model based on Variational
Autoencoders that is capable of handling polyphonic music with multiple
instrument tracks, as well as modeling the dynamics of music by incorporating
note durations and velocities. We show that MIDI-VAE can perform style transfer
on symbolic music by automatically changing pitches, dynamics and instruments
of a music piece from, e.g., a Classical to a Jazz style. We evaluate the
efficacy of the style transfer by training separate style validation
classifiers. Our model can also interpolate between short pieces of music,
produce medleys and create mixtures of entire songs. The interpolations
smoothly change pitches, dynamics and instrumentation to create a harmonic
bridge between two music pieces. To the best of our knowledge, this work
represents the first successful attempt at applying neural style transfer to
complete musical compositions.Comment: Paper accepted at the 19th International Society for Music
Information Retrieval Conference, ISMIR 2018, Paris, Franc
JamBot: Music Theory Aware Chord Based Generation of Polyphonic Music with LSTMs
We propose a novel approach for the generation of polyphonic music based on
LSTMs. We generate music in two steps. First, a chord LSTM predicts a chord
progression based on a chord embedding. A second LSTM then generates polyphonic
music from the predicted chord progression. The generated music sounds pleasing
and harmonic, with only few dissonant notes. It has clear long-term structure
that is similar to what a musician would play during a jam session. We show
that our approach is sensible from a music theory perspective by evaluating the
learned chord embeddings. Surprisingly, our simple model managed to extract the
circle of fifths, an important tool in music theory, from the dataset.Comment: Paper presented at the 29th International Conference on Tools with
Artificial Intelligence, ICTAI 2017, Boston, MA, US
Telling BERT's full story: from Local Attention to Global Aggregation
We take a deep look into the behavior of self-attention heads in the
transformer architecture. In light of recent work discouraging the use of
attention distributions for explaining a model's behavior, we show that
attention distributions can nevertheless provide insights into the local
behavior of attention heads. This way, we propose a distinction between local
patterns revealed by attention and global patterns that refer back to the
input, and analyze BERT from both angles. We use gradient attribution to
analyze how the output of an attention attention head depends on the input
tokens, effectively extending the local attention-based analysis to account for
the mixing of information throughout the transformer layers. We find that there
is a significant discrepancy between attention and attribution distributions,
caused by the mixing of context inside the model. We quantify this discrepancy
and observe that interestingly, there are some patterns that persist across all
layers despite the mixing.Comment: Accepted at EACL 202
- …