1 research outputs found
Multi-Face: Self-supervised Multiview Adaptation for Robust Face Clustering in Videos
Robust face clustering is a key step towards computational understanding of
visual character portrayals in media. Face clustering for long-form content
such as movies is challenging because of variations in appearance and lack of
large-scale labeled video resources. However, local face tracking in videos can
be used to mine samples belonging to same/different persons by examining the
faces co-occurring in a video frame. In this work, we use this idea of
self-supervision to harvest large amounts of weakly labeled face tracks in
movies. We propose a nearest-neighbor search in the embedding space to mine
hard examples from the face tracks followed by domain adaptation using
multiview shared subspace learning. Our benchmarking on movie datasets
demonstrate the robustness of multiview adaptation for face verification and
clustering. We hope that the large-scale data resources developed in this work
can further advance automatic character labeling in videos