2 research outputs found
A Unified Multi-Faceted Video Summarization System
This paper addresses automatic summarization and search in visual data
comprising of videos, live streams and image collections in a unified manner.
In particular, we propose a framework for multi-faceted summarization which
extracts key-frames (image summaries), skims (video summaries) and entity
summaries (summarization at the level of entities like objects, scenes, humans
and faces in the video). The user can either view these as extractive
summarization, or query focused summarization. Our approach first pre-processes
the video or image collection once, to extract all important visual features,
following which we provide an interactive mechanism to the user to summarize
the video based on their choice. We investigate several diversity, coverage and
representation models for all these problems, and argue the utility of these
different mod- els depending on the application. While most of the prior work
on submodular summarization approaches has focused on combining several models
and learning weighted mixtures, we focus on the explain-ability of different
the diversity, coverage and representation models and their scalability. Most
importantly, we also show that we can summarize hours of video data in a few
seconds, and our system allows the user to generate summaries of various
lengths and types interactively on the fly.Comment: 18 pages, 11 Figure
Demystifying Multi-Faceted Video Summarization: Tradeoff Between Diversity,Representation, Coverage and Importance
This paper addresses automatic summarization of videos in a unified manner.
In particular, we propose a framework for multi-faceted summarization for
extractive, query base and entity summarization (summarization at the level of
entities like objects, scenes, humans and faces in the video). We investigate
several summarization models which capture notions of diversity, coverage,
representation and importance, and argue the utility of these different models
depending on the application. While most of the prior work on submodular
summarization approaches has focused oncombining several models and learning
weighted mixtures, we focus on the explainability of different models and
featurizations, and how they apply to different domains. We also provide
implementation details on summarization systems and the different modalities
involved. We hope that the study from this paper will give insights into
practitioners to appropriately choose the right summarization models for the
problems at hand.Comment: Accepted to WACV 2019. arXiv admin note: substantial text overlap
with arXiv:1704.01466, arXiv:1809.0884