Movie highlights stand out of the screenplay for efficient browsing and play
a crucial role on social media platforms. Based on existing efforts, this work
has two observations: (1) For different annotators, labeling highlight has
uncertainty, which leads to inaccurate and time-consuming annotations. (2)
Besides previous supervised or unsupervised settings, some existing video
corpora can be useful, e.g., trailers, but they are often noisy and incomplete
to cover the full highlights. In this work, we study a more practical and
promising setting, i.e., reformulating highlight detection as "learning with
noisy labels". This setting does not require time-consuming manual annotations
and can fully utilize existing abundant video corpora. First, based on movie
trailers, we leverage scene segmentation to obtain complete shots, which are
regarded as noisy labels. Then, we propose a Collaborative noisy Label Cleaner
(CLC) framework to learn from noisy highlight moments. CLC consists of two
modules: augmented cross-propagation (ACP) and multi-modality cleaning (MMC).
The former aims to exploit the closely related audio-visual signals and fuse
them to learn unified multi-modal representations. The latter aims to achieve
cleaner highlight labels by observing the changes in losses among different
modalities. To verify the effectiveness of CLC, we further collect a
large-scale highlight dataset named MovieLights. Comprehensive experiments on
MovieLights and YouTube Highlights datasets demonstrate the effectiveness of
our approach. Code has been made available at:
https://github.com/TencentYoutuResearch/HighlightDetection-CLCComment: Accepted to CVPR202