2 research outputs found
WuYun: Exploring hierarchical skeleton-guided melody generation using knowledge-enhanced deep learning
Although deep learning has revolutionized music generation, existing methods
for structured melody generation follow an end-to-end left-to-right
note-by-note generative paradigm and treat each note equally. Here, we present
WuYun, a knowledge-enhanced deep learning architecture for improving the
structure of generated melodies, which first generates the most structurally
important notes to construct a melodic skeleton and subsequently infills it
with dynamically decorative notes into a full-fledged melody. Specifically, we
use music domain knowledge to extract melodic skeletons and employ sequence
learning to reconstruct them, which serve as additional knowledge to provide
auxiliary guidance for the melody generation process. We demonstrate that WuYun
can generate melodies with better long-term structure and musicality and
outperforms other state-of-the-art methods by 0.51 on average on all subjective
evaluation metrics. Our study provides a multidisciplinary lens to design
melodic hierarchical structures and bridge the gap between data-driven and
knowledge-based approaches for numerous music generation tasks
ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships
Lyric-to-melody generation, which generates melody according to given lyrics,
is one of the most important automatic music composition tasks. With the rapid
development of deep learning, previous works address this task with end-to-end
neural network models. However, deep learning models cannot well capture the
strict but subtle relationships between lyrics and melodies, which compromises
the harmony between lyrics and generated melodies. In this paper, we propose
ReLyMe, a method that incorporates Relationships between Lyrics and Melodies
from music theory to ensure the harmony between lyrics and melodies.
Specifically, we first introduce several principles that lyrics and melodies
should follow in terms of tone, rhythm, and structure relationships. These
principles are then integrated into neural network lyric-to-melody models by
adding corresponding constraints during the decoding process to improve the
harmony between lyrics and melodies. We use a series of objective and
subjective metrics to evaluate the generated melodies. Experiments on both
English and Chinese song datasets show the effectiveness of ReLyMe,
demonstrating the superiority of incorporating lyric-melody relationships from
the music domain into neural lyric-to-melody generation.Comment: Accepted by ACMMM 2022, ora