11 research outputs found
Improving Sequential Determinantal Point Processes for Supervised Video Summarization
It is now much easier than ever before to produce videos. While the
ubiquitous video data is a great source for information discovery and
extraction, the computational challenges are unparalleled. Automatically
summarizing the videos has become a substantial need for browsing, searching,
and indexing visual content. This paper is in the vein of supervised video
summarization using sequential determinantal point process (SeqDPP), which
models diversity by a probabilistic distribution. We improve this model in two
folds. In terms of learning, we propose a large-margin algorithm to address the
exposure bias problem in SeqDPP. In terms of modeling, we design a new
probabilistic distribution such that, when it is integrated into SeqDPP, the
resulting model accepts user input about the expected length of the summary.
Moreover, we also significantly extend a popular video summarization dataset by
1) more egocentric videos, 2) dense user annotations, and 3) a refined
evaluation scheme. We conduct extensive experiments on this dataset (about 60
hours of videos in total) and compare our approach to several competitive
baselines
Summarizing Videos with Attention
In this work we propose a novel method for supervised, keyshots based video
summarization by applying a conceptually simple and computationally efficient
soft, self-attention mechanism. Current state of the art methods leverage
bi-directional recurrent networks such as BiLSTM combined with attention. These
networks are complex to implement and computationally demanding compared to
fully connected networks. To that end we propose a simple, self-attention based
network for video summarization which performs the entire sequence to sequence
transformation in a single feed forward pass and single backward pass during
training. Our method sets a new state of the art results on two benchmarks
TvSum and SumMe, commonly used in this domain.Comment: Presented at ACCV2018 AIU2018 worksho
Towards key-frame extraction methods for 3D video: a review
The increasing rate of creation and use of 3D video content leads to a pressing need for methods capable of lowering
the cost of 3D video searching, browsing and indexing operations, with improved content selection performance.
Video summarisation methods specifically tailored for 3D video content fulfil these requirements. This paper presents
a review of the state-of-the-art of a crucial component of 3D video summarisation algorithms: the key-frame
extraction methods. The methods reviewed cover 3D video key-frame extraction as well as shot boundary detection
methods specific for use in 3D video. The performance metrics used to evaluate the key-frame extraction methods
and the summaries derived from those key-frames are presented and discussed. The applications of these methods
are also presented and discussed, followed by an exposition about current research challenges on 3D video
summarisation methods