Background music (BGM) can enhance the video's emotion. However, selecting an
appropriate BGM often requires domain knowledge. This has led to the
development of video-music retrieval techniques. Most existing approaches
utilize pretrained video/music feature extractors trained with different target
sets to obtain average video/music-level embeddings. The drawbacks are
two-fold. One is that different target sets for video/music pretraining may
cause the generated embeddings difficult to match. The second is that the
underlying temporal correlation between video and music is ignored. In this
paper, our proposed approach leverages a unified target set to perform
video/music pretraining and produces clip-level embeddings to preserve temporal
information. The downstream cross-modal matching is based on the clip-level
features with embedded music rhythm and optical flow information. Experiments
demonstrate that our proposed method can achieve superior performance over the
state-of-the-art methods by a significant margin